2024-04-25T00:52:01.377 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T00:52:01.377 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672295 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/ffsb}} --owner scheduled_teuthology@teuthology --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672295/orig.config.yaml 2024-04-25T00:52:01.435 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T00:52:01.523 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672295 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/ffsb}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672295' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd heartbeat grace: 60 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - SLOW_OPS - slow request sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFT+lZ0fdbw3rN7cTHsfMkIXZ2kNBwDQ1trPKeOklU/XuNJnRZHbbq0iV4IYXhQoxo0cybbhfwRMV66sxr6gkIY= smithi040.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF4cwHvZ9atxUonpm7JKx9mfuj95ava/L3CdhfOBCmP7lgPF2UBi0jXMPo28EdAQwiLlzVsuqXL5thRsLoB8T9M= smithi169.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKmuRDv0oUw4I9RqWmLnVjfNixnnCuritObbyr8ca+pwu3brwBrDmzusbUyIb+ehWGWC3t+WWEua7WbYsW3tRIU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T00:52:01.523 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-25T00:52:01.524 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-25T00:52:01.524 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T00:52:01.526 INFO:teuthology.task.internal:Checking packages... 2024-04-25T00:52:01.549 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-25T00:52:01.549 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T00:52:01.550 INFO:teuthology.packaging:ref: None 2024-04-25T00:52:01.550 INFO:teuthology.packaging:tag: None 2024-04-25T00:52:01.550 INFO:teuthology.packaging:branch: squid 2024-04-25T00:52:01.550 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:52:01.550 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-25T00:52:01.752 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445.gd2c45d97 2024-04-25T00:52:01.754 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T00:52:01.761 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T00:52:01.761 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T00:52:01.767 INFO:teuthology.task.internal:Saving configuration 2024-04-25T00:52:01.788 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T00:52:01.794 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T00:52:01.818 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672295', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 00:46:52.783015', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFT+lZ0fdbw3rN7cTHsfMkIXZ2kNBwDQ1trPKeOklU/XuNJnRZHbbq0iV4IYXhQoxo0cybbhfwRMV66sxr6gkIY='} 2024-04-25T00:52:01.844 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672295', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 00:46:52.784579', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF4cwHvZ9atxUonpm7JKx9mfuj95ava/L3CdhfOBCmP7lgPF2UBi0jXMPo28EdAQwiLlzVsuqXL5thRsLoB8T9M='} 2024-04-25T00:52:01.865 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672295', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 00:46:52.785910', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKmuRDv0oUw4I9RqWmLnVjfNixnnCuritObbyr8ca+pwu3brwBrDmzusbUyIb+ehWGWC3t+WWEua7WbYsW3tRIU='} 2024-04-25T00:52:01.866 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T00:52:01.872 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T00:52:01.872 INFO:teuthology.task.internal:roles: ubuntu@smithi040.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T00:52:01.872 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T00:52:01.872 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T00:52:01.967 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f921a072160>, signals=[15]) 2024-04-25T00:52:01.968 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T00:52:01.974 INFO:teuthology.task.internal:Opening connections... 2024-04-25T00:52:01.974 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T00:52:01.976 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:52:02.054 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi040.front.sepia.ceph.com 2024-04-25T00:52:02.055 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:52:02.130 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T00:52:02.131 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:52:02.204 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T00:52:02.210 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2024-04-25T00:52:02.229 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2024-04-25T00:52:02.229 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2024-04-25T00:52:02.288 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="CentOS Stream" 2024-04-25T00:52:02.288 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="9" 2024-04-25T00:52:02.288 INFO:teuthology.orchestra.run.smithi031.stdout:ID="centos" 2024-04-25T00:52:02.288 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="rhel fedora" 2024-04-25T00:52:02.288 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="9" 2024-04-25T00:52:02.288 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el9" 2024-04-25T00:52:02.288 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T00:52:02.288 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2024-04-25T00:52:02.288 INFO:teuthology.orchestra.run.smithi031.stdout:LOGO="fedora-logo-icon" 2024-04-25T00:52:02.288 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T00:52:02.288 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://centos.org/" 2024-04-25T00:52:02.289 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T00:52:02.289 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T00:52:02.289 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T00:52:02.289 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2024-04-25T00:52:02.308 DEBUG:teuthology.orchestra.run.smithi040:> uname -m 2024-04-25T00:52:02.327 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64 2024-04-25T00:52:02.327 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release 2024-04-25T00:52:02.385 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="CentOS Stream" 2024-04-25T00:52:02.385 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="9" 2024-04-25T00:52:02.385 INFO:teuthology.orchestra.run.smithi040.stdout:ID="centos" 2024-04-25T00:52:02.385 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE="rhel fedora" 2024-04-25T00:52:02.385 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="9" 2024-04-25T00:52:02.385 INFO:teuthology.orchestra.run.smithi040.stdout:PLATFORM_ID="platform:el9" 2024-04-25T00:52:02.385 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T00:52:02.385 INFO:teuthology.orchestra.run.smithi040.stdout:ANSI_COLOR="0;31" 2024-04-25T00:52:02.385 INFO:teuthology.orchestra.run.smithi040.stdout:LOGO="fedora-logo-icon" 2024-04-25T00:52:02.386 INFO:teuthology.orchestra.run.smithi040.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T00:52:02.386 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://centos.org/" 2024-04-25T00:52:02.386 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T00:52:02.386 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T00:52:02.386 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T00:52:02.386 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server 2024-04-25T00:52:02.408 DEBUG:teuthology.orchestra.run.smithi169:> uname -m 2024-04-25T00:52:02.426 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2024-04-25T00:52:02.426 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2024-04-25T00:52:02.483 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream" 2024-04-25T00:52:02.483 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="9" 2024-04-25T00:52:02.483 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos" 2024-04-25T00:52:02.483 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora" 2024-04-25T00:52:02.483 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="9" 2024-04-25T00:52:02.483 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el9" 2024-04-25T00:52:02.483 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T00:52:02.484 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31" 2024-04-25T00:52:02.484 INFO:teuthology.orchestra.run.smithi169.stdout:LOGO="fedora-logo-icon" 2024-04-25T00:52:02.484 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T00:52:02.484 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/" 2024-04-25T00:52:02.484 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T00:52:02.484 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T00:52:02.484 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T00:52:02.484 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2024-04-25T00:52:02.500 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T00:52:02.512 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T00:52:02.579 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T00:52:02.579 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2024-04-25T00:52:02.581 DEBUG:teuthology.orchestra.run.smithi040:> test '!' -e /home/ubuntu/cephtest 2024-04-25T00:52:02.584 DEBUG:teuthology.orchestra.run.smithi169:> test '!' -e /home/ubuntu/cephtest 2024-04-25T00:52:02.600 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T00:52:02.606 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T00:52:02.606 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2024-04-25T00:52:02.640 DEBUG:teuthology.orchestra.run.smithi040:> test -z $(ls -A /var/lib/ceph) 2024-04-25T00:52:02.642 DEBUG:teuthology.orchestra.run.smithi169:> test -z $(ls -A /var/lib/ceph) 2024-04-25T00:52:02.735 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T00:52:02.807 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T00:52:02.821 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-04-25T00:52:02.822 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-25T00:52:02.822 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-25T00:52:02.822 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T00:52:02.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:52:02.840 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-25T00:52:03.081 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T00:52:03.081 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T00:52:03.081 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-04-25T00:52:03.100 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-437.el9.x86_64 2024-04-25T00:52:03.100 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T00:52:03.100 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T00:52:03.101 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum upgrade -y linux-firmware 2024-04-25T00:52:05.927 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 6.6 MB/s | 8.0 MB 00:01 2024-04-25T00:52:08.503 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-04-25T00:52:14.295 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 6.2 MB 00:02 2024-04-25T00:52:15.712 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 16 kB 00:00 2024-04-25T00:52:17.649 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-25T00:52:22.855 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 46 kB/s | 1.7 kB 00:00 2024-04-25T00:52:24.918 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-25T00:52:24.919 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-04-25T00:52:24.919 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-25T00:52:25.012 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T00:52:25.013 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T00:52:25.359 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-25T00:52:25.360 DEBUG:teuthology.orchestra.run.smithi031:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-25T00:52:25.362 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi031.front.sepia.ceph.com to complete... 2024-04-25T00:52:38.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25 00:52:38 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-25T00:52:38.638 INFO:teuthology.orchestra.run.smithi031.stderr:FINISHED --2024-04-25 00:52:38-- 2024-04-25T00:52:38.638 INFO:teuthology.orchestra.run.smithi031.stderr:Total wall clock time: 13s 2024-04-25T00:52:38.638 INFO:teuthology.orchestra.run.smithi031.stderr:Downloaded: 1 files, 341M in 12s (28.6 MB/s) 2024-04-25T00:52:38.641 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T00:52:38.641 DEBUG:teuthology.orchestra.run.smithi031:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-25T00:52:40.838 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying... ######################################## 2024-04-25T00:52:40.989 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing... ######################################## 2024-04-25T00:52:41.095 INFO:teuthology.orchestra.run.smithi031.stdout:Updating / installing... 2024-04-25T00:52:49.411 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-25T00:53:29.236 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi031.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-25T00:53:29.237 DEBUG:teuthology.orchestra.run.smithi031:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-25T00:53:29.318 INFO:teuthology.orchestra.run.smithi031.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T00:53:29.318 INFO:teuthology.orchestra.run.smithi031.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T00:53:29.318 INFO:teuthology.orchestra.run.smithi031.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T00:53:29.318 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-25T00:53:29.319 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-25T00:53:29.319 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-25T00:53:29.319 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-25T00:53:29.319 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-25T00:53:29.319 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-25T00:53:29.319 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T00:53:29.319 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T00:53:29.319 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T00:53:29.319 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T00:53:29.319 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T00:53:29.319 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T00:53:29.319 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T00:53:29.319 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-25T00:53:29.320 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T00:53:29.320 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T00:53:29.320 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T00:53:29.320 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T00:53:29.320 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T00:53:29.320 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T00:53:29.320 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T00:53:29.320 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T00:53:29.320 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T00:53:29.320 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T00:53:29.320 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T00:53:29.320 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T00:53:29.320 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-25T00:53:29.320 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-25T00:53:29.321 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-25T00:53:29.321 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T00:53:29.321 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-25T00:53:29.321 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T00:53:29.321 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T00:53:29.321 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T00:53:29.321 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-25T00:53:29.321 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T00:53:29.321 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T00:53:29.321 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T00:53:29.321 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-25T00:53:29.321 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T00:53:29.321 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T00:53:29.321 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T00:53:29.322 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T00:53:29.322 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T00:53:29.322 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T00:53:29.322 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T00:53:29.322 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T00:53:29.322 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T00:53:29.322 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T00:53:29.322 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T00:53:29.322 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T00:53:29.322 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T00:53:29.322 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T00:53:29.322 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T00:53:29.322 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T00:53:29.323 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T00:53:29.323 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T00:53:29.323 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T00:53:29.323 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T00:53:29.323 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T00:53:29.323 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T00:53:29.323 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T00:53:29.323 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T00:53:29.323 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T00:53:29.323 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T00:53:29.323 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T00:53:29.323 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T00:53:29.323 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T00:53:29.324 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T00:53:29.324 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T00:53:29.324 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T00:53:29.324 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T00:53:29.324 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T00:53:29.324 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T00:53:29.324 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T00:53:29.324 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T00:53:29.324 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T00:53:29.324 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T00:53:29.324 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T00:53:29.324 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T00:53:29.324 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T00:53:29.324 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T00:53:29.325 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T00:53:29.325 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T00:53:29.325 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-25T00:53:29.325 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-25T00:53:29.325 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T00:53:29.325 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T00:53:29.325 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T00:53:29.325 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T00:53:29.325 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T00:53:29.325 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T00:53:29.325 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T00:53:29.325 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-25T00:53:29.325 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T00:53:29.325 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T00:53:29.325 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-25T00:53:29.326 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T00:53:29.326 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T00:53:29.326 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T00:53:29.326 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T00:53:29.326 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T00:53:29.326 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T00:53:29.326 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T00:53:29.326 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T00:53:29.326 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T00:53:29.326 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T00:53:29.326 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T00:53:29.326 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T00:53:29.326 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T00:53:29.326 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T00:53:29.327 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T00:53:29.327 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T00:53:29.327 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T00:53:29.327 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T00:53:29.327 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T00:53:29.327 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T00:53:29.327 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T00:53:29.327 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T00:53:29.327 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T00:53:29.327 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T00:53:29.327 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T00:53:29.327 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T00:53:29.327 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T00:53:29.328 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T00:53:29.328 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T00:53:29.328 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T00:53:29.328 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T00:53:29.328 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T00:53:29.328 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T00:53:29.328 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T00:53:29.328 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T00:53:29.328 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-25T00:53:29.328 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T00:53:29.328 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T00:53:29.328 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T00:53:29.328 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T00:53:29.328 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T00:53:29.329 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T00:53:29.329 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T00:53:29.329 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T00:53:29.329 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T00:53:29.329 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T00:53:29.329 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T00:53:29.329 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T00:53:29.329 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T00:53:29.329 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T00:53:29.329 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T00:53:29.329 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T00:53:29.329 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T00:53:29.329 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-25T00:53:29.329 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T00:53:29.329 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-25T00:53:29.330 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-25T00:53:29.330 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-25T00:53:29.330 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-25T00:53:29.330 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-25T00:53:29.330 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T00:53:29.330 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-25T00:53:29.330 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-25T00:53:29.330 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T00:53:29.330 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-25T00:53:29.330 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-25T00:53:29.330 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T00:53:29.330 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T00:53:29.330 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-25T00:53:29.330 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-25T00:53:29.331 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T00:53:29.331 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T00:53:29.331 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-25T00:53:29.331 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T00:53:29.331 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T00:53:29.331 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T00:53:29.331 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-25T00:53:29.331 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-25T00:53:29.331 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T00:53:29.331 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-25T00:53:29.331 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-25T00:53:29.331 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T00:53:29.331 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-25T00:53:29.331 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T00:53:29.332 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-25T00:53:29.332 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T00:53:29.332 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T00:53:29.332 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T00:53:29.332 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T00:53:29.332 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T00:53:29.332 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-25T00:53:29.332 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T00:53:29.332 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T00:53:29.332 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T00:53:29.332 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T00:53:29.332 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T00:53:29.332 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T00:53:29.332 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T00:53:29.333 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-25T00:53:29.333 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T00:53:29.333 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T00:53:29.333 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T00:53:29.333 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-25T00:53:29.333 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-25T00:53:29.333 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T00:53:29.333 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T00:53:29.333 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T00:53:29.333 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-25T00:53:29.333 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-25T00:53:29.333 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-25T00:53:29.333 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-25T00:53:29.333 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-25T00:53:29.334 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-25T00:53:29.334 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T00:53:29.334 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T00:53:29.334 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-25T00:53:29.334 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-25T00:53:29.334 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T00:53:29.334 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-25T00:53:29.334 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-25T00:53:29.334 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T00:53:29.334 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T00:53:29.334 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-25T00:53:29.334 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-25T00:53:29.334 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-25T00:53:29.334 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T00:53:29.335 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-25T00:53:29.335 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T00:53:29.335 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T00:53:29.335 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T00:53:29.335 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-25T00:53:29.335 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T00:53:29.335 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-25T00:53:29.335 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T00:53:29.335 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-25T00:53:29.335 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-25T00:53:29.335 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-25T00:53:29.335 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-25T00:53:29.335 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T00:53:29.336 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-25T00:53:29.336 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T00:53:29.336 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-25T00:53:29.336 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T00:53:29.336 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T00:53:29.336 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T00:53:29.336 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T00:53:29.336 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T00:53:29.336 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T00:53:29.336 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T00:53:29.336 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T00:53:29.336 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T00:53:29.336 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T00:53:29.336 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T00:53:29.337 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T00:53:29.337 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T00:53:29.337 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-25T00:53:29.337 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-25T00:53:29.337 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T00:53:29.337 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-25T00:53:29.337 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T00:53:29.337 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T00:53:29.337 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-25T00:53:29.337 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T00:53:29.337 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T00:53:29.337 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T00:53:29.337 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T00:53:29.337 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T00:53:29.338 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T00:53:29.338 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T00:53:29.338 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T00:53:29.338 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T00:53:29.338 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T00:53:29.338 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T00:53:29.338 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T00:53:29.338 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T00:53:29.338 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T00:53:29.338 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T00:53:29.338 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T00:53:29.338 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T00:53:29.338 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T00:53:29.338 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T00:53:29.339 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T00:53:29.339 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T00:53:29.339 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T00:53:29.339 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T00:53:29.339 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T00:53:29.339 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T00:53:29.339 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T00:53:29.339 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T00:53:29.339 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T00:53:29.339 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T00:53:29.339 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T00:53:29.339 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T00:53:29.339 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T00:53:29.339 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T00:53:29.340 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T00:53:29.340 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T00:53:29.340 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T00:53:29.340 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T00:53:29.340 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T00:53:29.340 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T00:53:29.340 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T00:53:29.340 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T00:53:29.340 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T00:53:29.340 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T00:53:29.340 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T00:53:29.340 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T00:53:29.340 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T00:53:29.340 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T00:53:29.341 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T00:53:29.341 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T00:53:29.341 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T00:53:29.341 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T00:53:29.341 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T00:53:29.341 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T00:53:29.341 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T00:53:29.341 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T00:53:29.341 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T00:53:29.341 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T00:53:29.341 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T00:53:29.341 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T00:53:29.341 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T00:53:29.341 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T00:53:29.342 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T00:53:29.342 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T00:53:29.342 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T00:53:29.342 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-25T00:53:29.342 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T00:53:29.342 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T00:53:29.342 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T00:53:29.342 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-25T00:53:29.342 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T00:53:29.342 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T00:53:29.342 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T00:53:29.342 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T00:53:29.342 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T00:53:29.342 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T00:53:29.343 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T00:53:29.343 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T00:53:29.343 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T00:53:29.343 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T00:53:29.343 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T00:53:29.343 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T00:53:29.343 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T00:53:29.343 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T00:53:29.343 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T00:53:29.343 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T00:53:29.343 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T00:53:29.343 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T00:53:29.343 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T00:53:29.343 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T00:53:29.343 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T00:53:29.344 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T00:53:29.344 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T00:53:29.344 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T00:53:29.344 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T00:53:29.344 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T00:53:29.344 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T00:53:29.344 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T00:53:29.344 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T00:53:29.344 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T00:53:29.344 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T00:53:29.344 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T00:53:29.344 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T00:53:29.344 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T00:53:29.345 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T00:53:29.345 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T00:53:29.345 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T00:53:29.345 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T00:53:29.345 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T00:53:29.345 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T00:53:29.345 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T00:53:29.345 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T00:53:29.345 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T00:53:29.345 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T00:53:29.345 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T00:53:29.345 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T00:53:29.345 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T00:53:29.345 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T00:53:29.346 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T00:53:29.346 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T00:53:29.346 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T00:53:29.346 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T00:53:29.346 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T00:53:29.346 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T00:53:29.346 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T00:53:29.346 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T00:53:29.346 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T00:53:29.346 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T00:53:29.346 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T00:53:29.346 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T00:53:29.346 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T00:53:29.347 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T00:53:29.347 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T00:53:29.347 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T00:53:29.347 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T00:53:29.347 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T00:53:29.347 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T00:53:29.347 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T00:53:29.347 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T00:53:29.347 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T00:53:29.347 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T00:53:29.347 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T00:53:29.347 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T00:53:29.347 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T00:53:29.347 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T00:53:29.348 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T00:53:29.348 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T00:53:29.348 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T00:53:29.348 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T00:53:29.348 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T00:53:29.348 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T00:53:29.348 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T00:53:29.348 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T00:53:29.348 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T00:53:29.348 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T00:53:29.348 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T00:53:29.348 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T00:53:29.348 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T00:53:29.348 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-25T00:53:29.349 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T00:53:29.349 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T00:53:29.349 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T00:53:29.349 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T00:53:29.349 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T00:53:29.349 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T00:53:29.349 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T00:53:29.349 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T00:53:29.349 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T00:53:29.349 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T00:53:29.349 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T00:53:29.349 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T00:53:29.349 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T00:53:29.349 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T00:53:29.349 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T00:53:29.350 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T00:53:29.350 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T00:53:29.350 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T00:53:29.350 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T00:53:29.350 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T00:53:29.350 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T00:53:29.350 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T00:53:29.350 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T00:53:29.350 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T00:53:29.350 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T00:53:29.350 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T00:53:29.350 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T00:53:29.350 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T00:53:29.351 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T00:53:29.351 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T00:53:29.351 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T00:53:29.351 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T00:53:29.351 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T00:53:29.351 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T00:53:29.351 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T00:53:29.351 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T00:53:29.351 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T00:53:29.351 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T00:53:29.351 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T00:53:29.351 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T00:53:29.351 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T00:53:29.351 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-25T00:53:29.352 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-25T00:53:29.352 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T00:53:29.352 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T00:53:29.352 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-25T00:53:29.352 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T00:53:29.352 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T00:53:29.352 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T00:53:29.352 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T00:53:29.352 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T00:53:29.352 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T00:53:29.352 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T00:53:29.352 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T00:53:29.352 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T00:53:29.352 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T00:53:29.353 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T00:53:29.353 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T00:53:29.353 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T00:53:29.353 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T00:53:29.353 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T00:53:29.353 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T00:53:29.353 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T00:53:29.353 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T00:53:29.353 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T00:53:29.353 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T00:53:29.353 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T00:53:29.353 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T00:53:29.353 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T00:53:29.353 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-25T00:53:29.354 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T00:53:29.354 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-25T00:53:29.354 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T00:53:29.354 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T00:53:29.354 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-25T00:53:29.354 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-25T00:53:29.354 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T00:53:29.354 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T00:53:29.354 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T00:53:29.354 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T00:53:29.354 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T00:53:29.354 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T00:53:29.354 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T00:53:29.354 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-25T00:53:29.355 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-25T00:53:29.355 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T00:53:29.355 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-25T00:53:29.355 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T00:53:29.355 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-25T00:53:29.355 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T00:53:29.355 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-25T00:53:29.355 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T00:53:29.355 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-25T00:53:29.355 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-25T00:53:29.355 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T00:53:29.355 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-25T00:53:29.355 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T00:53:29.355 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-25T00:53:29.356 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T00:53:29.356 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-25T00:53:29.356 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T00:53:29.356 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-25T00:53:29.356 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T00:53:29.356 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-25T00:53:29.356 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-25T00:53:29.356 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T00:53:29.356 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T00:53:29.356 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T00:53:29.356 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T00:53:29.356 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-25T00:53:29.356 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T00:53:29.356 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-25T00:53:29.356 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T00:53:29.357 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T00:53:29.357 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T00:53:29.357 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T00:53:29.357 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T00:53:29.357 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T00:53:29.357 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T00:53:29.357 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T00:53:29.357 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-25T00:53:29.357 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T00:53:29.357 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-25T00:53:29.357 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T00:53:29.357 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T00:53:29.357 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T00:53:29.357 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T00:53:29.358 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T00:53:29.358 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-25T00:53:29.358 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T00:53:29.358 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T00:53:29.358 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T00:53:29.358 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T00:53:29.358 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-25T00:53:29.358 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T00:53:29.358 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T00:53:29.358 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T00:53:29.358 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T00:53:29.358 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T00:53:29.358 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T00:53:29.358 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T00:53:29.359 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T00:53:29.359 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T00:53:29.359 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-25T00:53:29.359 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-25T00:53:29.359 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T00:53:29.359 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T00:53:29.359 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T00:53:29.359 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T00:53:29.359 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T00:53:29.359 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T00:53:29.359 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T00:53:29.359 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T00:53:29.359 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T00:53:29.359 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T00:53:29.360 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T00:53:29.360 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T00:53:29.360 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T00:53:29.360 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T00:53:29.360 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T00:53:29.360 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T00:53:29.360 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-25T00:53:29.360 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T00:53:29.360 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T00:53:29.360 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T00:53:29.360 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T00:53:29.360 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T00:53:29.360 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-25T00:53:29.361 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-25T00:53:29.361 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T00:53:29.361 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T00:53:29.361 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T00:53:29.361 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T00:53:29.361 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T00:53:29.361 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T00:53:29.361 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T00:53:29.361 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-25T00:53:29.361 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T00:53:29.361 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T00:53:29.361 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-25T00:53:29.361 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T00:53:29.362 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T00:53:29.362 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T00:53:29.362 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T00:53:29.362 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-25T00:53:29.362 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-25T00:53:29.362 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T00:53:29.362 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T00:53:29.362 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T00:53:29.362 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T00:53:29.362 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-25T00:53:29.362 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T00:53:29.362 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-25T00:53:29.362 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T00:53:29.362 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T00:53:29.363 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T00:53:29.363 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T00:53:29.363 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T00:53:29.363 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T00:53:29.363 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T00:53:29.363 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-25T00:53:29.363 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T00:53:29.363 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T00:53:29.363 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-25T00:53:29.363 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T00:53:29.363 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T00:53:29.363 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T00:53:29.363 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-25T00:53:29.363 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-25T00:53:29.364 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-25T00:53:29.364 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-25T00:53:29.364 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-25T00:53:29.364 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-25T00:53:29.364 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T00:53:29.364 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T00:53:29.364 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T00:53:29.364 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T00:53:29.364 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-25T00:53:29.364 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T00:53:29.364 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-25T00:53:29.364 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-25T00:53:29.364 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T00:53:29.364 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-25T00:53:29.365 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-25T00:53:29.365 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T00:53:29.365 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T00:53:29.365 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T00:53:29.365 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-25T00:53:29.365 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T00:53:29.365 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T00:53:29.365 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T00:53:29.365 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-25T00:53:29.365 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-25T00:53:29.365 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T00:53:29.365 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T00:53:29.365 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T00:53:29.365 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T00:53:29.366 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-25T00:53:29.366 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T00:53:29.366 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T00:53:29.366 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T00:53:29.366 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T00:53:29.366 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-25T00:53:29.366 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T00:53:29.366 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-25T00:53:29.366 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T00:53:29.366 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-25T00:53:29.366 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T00:53:29.366 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-25T00:53:29.366 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T00:53:29.366 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-25T00:53:29.367 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T00:53:29.367 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T00:53:29.367 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T00:53:29.367 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-25T00:53:29.367 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-25T00:53:29.367 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T00:53:29.367 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-25T00:53:29.367 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T00:53:29.367 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-25T00:53:29.367 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T00:53:29.367 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-25T00:53:29.367 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T00:53:29.367 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T00:53:29.368 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-25T00:53:29.368 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T00:53:29.368 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T00:53:29.368 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-25T00:53:29.368 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T00:53:29.368 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T00:53:29.368 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-25T00:53:29.368 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T00:53:29.368 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T00:53:29.368 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-25T00:53:29.368 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-25T00:53:29.368 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T00:53:29.368 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-25T00:53:29.369 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-25T00:53:29.369 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T00:53:29.369 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-25T00:53:29.369 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T00:53:29.369 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-25T00:53:29.369 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T00:53:29.369 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-25T00:53:29.369 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T00:53:29.369 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-25T00:53:29.369 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T00:53:29.369 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-25T00:53:29.369 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-25T00:53:29.369 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T00:53:29.369 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-25T00:53:29.370 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-25T00:53:29.370 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T00:53:29.370 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T00:53:29.370 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T00:53:29.370 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T00:53:29.370 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T00:53:29.370 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T00:53:29.370 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-25T00:53:29.370 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T00:53:29.370 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T00:53:29.370 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T00:53:29.370 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-25T00:53:29.370 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-25T00:53:29.371 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T00:53:29.371 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T00:53:29.371 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T00:53:29.371 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-25T00:53:29.371 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T00:53:29.371 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-25T00:53:29.371 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T00:53:29.371 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-25T00:53:29.371 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T00:53:29.371 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-25T00:53:29.371 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T00:53:29.371 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-25T00:53:29.371 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T00:53:29.372 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-25T00:53:29.372 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T00:53:29.372 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-25T00:53:29.372 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T00:53:29.372 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-25T00:53:29.372 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T00:53:29.372 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-25T00:53:29.372 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T00:53:29.372 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-25T00:53:29.372 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T00:53:29.372 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T00:53:29.372 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-25T00:53:29.372 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T00:53:29.372 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T00:53:29.372 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T00:53:29.373 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-25T00:53:29.373 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-25T00:53:29.373 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T00:53:29.373 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T00:53:29.373 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-25T00:53:29.373 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-25T00:53:29.373 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T00:53:29.373 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-25T00:53:29.373 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T00:53:29.373 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-25T00:53:29.373 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-25T00:53:29.373 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T00:53:29.373 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-25T00:53:29.373 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T00:53:29.373 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T00:53:29.374 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-25T00:53:29.374 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T00:53:29.374 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-25T00:53:29.374 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T00:53:29.374 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-25T00:53:29.374 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T00:53:29.374 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T00:53:29.374 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-25T00:53:29.374 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-25T00:53:29.374 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T00:53:29.374 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T00:53:29.374 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-25T00:53:29.374 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T00:53:29.374 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-25T00:53:29.375 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T00:53:29.375 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-25T00:53:29.375 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T00:53:29.375 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T00:53:29.375 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T00:53:29.375 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T00:53:29.375 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-25T00:53:29.375 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-25T00:53:29.375 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-25T00:53:29.375 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T00:53:29.375 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-25T00:53:29.375 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T00:53:29.375 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-25T00:53:29.375 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T00:53:29.376 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T00:53:29.376 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-25T00:53:29.376 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T00:53:29.376 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T00:53:29.376 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T00:53:29.376 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T00:53:29.376 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-25T00:53:29.376 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-25T00:53:29.376 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T00:53:29.376 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T00:53:29.376 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-25T00:53:29.376 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T00:53:29.376 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T00:53:29.376 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T00:53:29.377 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T00:53:29.377 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-25T00:53:29.377 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T00:53:29.377 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-25T00:53:29.377 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T00:53:29.377 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-25T00:53:29.377 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T00:53:29.377 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T00:53:29.377 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-25T00:53:29.377 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T00:53:29.377 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T00:53:29.377 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-25T00:53:29.377 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T00:53:29.378 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T00:53:29.378 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-25T00:53:29.378 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T00:53:29.378 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-25T00:53:29.378 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T00:53:29.378 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T00:53:29.378 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-25T00:53:29.378 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T00:53:29.378 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T00:53:29.378 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T00:53:29.378 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-25T00:53:29.378 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T00:53:29.378 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T00:53:29.379 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T00:53:29.379 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T00:53:29.379 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T00:53:29.379 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-25T00:53:29.379 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T00:53:29.379 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T00:53:29.379 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-25T00:53:29.379 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T00:53:29.379 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T00:53:29.379 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-25T00:53:29.379 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T00:53:29.379 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T00:53:29.379 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-25T00:53:29.379 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T00:53:29.380 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T00:53:29.380 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T00:53:29.380 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T00:53:29.380 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T00:53:29.380 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-25T00:53:29.380 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T00:53:29.380 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T00:53:29.380 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T00:53:29.380 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T00:53:29.380 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T00:53:29.380 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T00:53:29.380 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T00:53:29.380 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T00:53:29.380 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T00:53:29.381 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T00:53:29.381 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T00:53:29.381 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T00:53:29.381 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-25T00:53:29.381 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T00:53:29.381 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T00:53:29.381 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T00:53:29.381 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-25T00:53:29.381 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T00:53:29.381 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T00:53:29.381 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T00:53:29.381 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T00:53:29.381 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T00:53:29.381 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-25T00:53:29.381 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-25T00:53:29.382 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T00:53:29.382 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T00:53:29.382 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T00:53:29.382 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-25T00:53:29.382 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T00:53:29.382 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T00:53:29.382 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T00:53:29.382 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T00:53:29.382 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T00:53:29.382 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T00:53:29.382 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-25T00:53:29.382 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T00:53:29.382 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-25T00:53:29.383 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-25T00:53:29.383 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T00:53:29.383 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-25T00:53:29.383 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T00:53:29.383 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-25T00:53:29.383 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T00:53:29.383 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-25T00:53:29.383 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-25T00:53:29.383 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T00:53:29.383 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-25T00:53:29.383 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-25T00:53:29.383 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T00:53:29.383 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T00:53:29.383 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-25T00:53:29.383 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T00:53:29.384 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T00:53:29.384 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T00:53:29.384 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T00:53:29.384 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-25T00:53:29.384 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-25T00:53:29.384 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T00:53:29.384 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-25T00:53:29.384 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-25T00:53:29.384 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T00:53:29.384 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-25T00:53:29.384 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T00:53:29.384 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T00:53:29.384 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T00:53:29.385 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-25T00:53:29.385 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-25T00:53:29.385 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T00:53:29.385 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-25T00:53:29.385 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-25T00:53:29.385 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T00:53:29.385 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T00:53:29.385 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T00:53:29.385 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T00:53:29.385 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T00:53:29.385 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T00:53:29.385 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T00:53:29.385 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T00:53:29.385 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T00:53:29.386 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T00:53:29.386 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T00:53:29.386 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T00:53:29.386 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-25T00:53:29.386 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T00:53:29.386 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T00:53:29.386 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T00:53:29.386 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T00:53:29.386 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T00:53:29.386 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T00:53:29.386 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T00:53:29.386 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T00:53:29.386 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T00:53:29.387 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T00:53:29.387 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T00:53:29.387 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T00:53:29.387 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-25T00:53:29.387 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T00:53:29.387 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T00:53:29.387 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-25T00:53:29.387 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-25T00:53:29.387 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T00:53:29.387 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T00:53:29.387 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T00:53:29.387 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T00:53:29.387 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-25T00:53:29.387 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T00:53:29.388 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T00:53:29.388 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-25T00:53:29.388 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-25T00:53:29.388 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-25T00:53:29.388 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T00:53:29.388 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T00:53:29.388 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-25T00:53:29.388 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T00:53:29.388 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T00:53:29.388 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T00:53:29.388 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T00:53:29.388 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T00:53:29.388 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T00:53:29.388 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T00:53:29.388 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T00:53:29.389 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T00:53:29.389 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T00:53:29.389 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T00:53:29.389 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T00:53:29.389 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T00:53:29.389 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T00:53:29.389 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T00:53:29.389 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T00:53:29.389 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T00:53:29.389 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T00:53:29.389 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T00:53:29.389 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T00:53:29.389 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T00:53:29.389 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T00:53:29.389 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T00:53:29.390 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T00:53:29.390 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T00:53:29.390 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T00:53:29.390 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T00:53:29.390 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T00:53:29.390 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T00:53:29.390 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T00:53:29.390 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T00:53:29.390 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-25T00:53:29.390 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T00:53:29.390 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T00:53:29.390 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T00:53:29.390 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T00:53:29.390 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T00:53:29.391 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-25T00:53:29.391 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T00:53:29.391 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T00:53:29.391 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-25T00:53:29.391 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T00:53:29.391 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T00:53:29.391 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T00:53:29.391 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-25T00:53:29.391 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T00:53:29.391 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T00:53:29.391 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-25T00:53:29.391 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T00:53:29.391 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-25T00:53:29.391 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T00:53:29.392 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-25T00:53:29.392 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T00:53:29.392 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-25T00:53:29.392 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T00:53:29.392 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T00:53:29.392 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-25T00:53:29.392 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T00:53:29.392 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-25T00:53:29.392 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-25T00:53:29.392 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T00:53:29.392 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-25T00:53:29.392 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T00:53:29.392 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T00:53:29.392 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T00:53:29.393 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-25T00:53:29.393 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-25T00:53:29.393 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-25T00:53:29.393 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-25T00:53:29.393 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-25T00:53:29.393 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T00:53:29.393 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-25T00:53:29.393 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T00:53:29.393 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T00:53:29.393 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-25T00:53:29.393 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T00:53:29.393 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T00:53:29.393 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-25T00:53:29.393 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T00:53:29.394 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T00:53:29.394 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T00:53:29.394 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T00:53:29.394 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T00:53:29.394 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T00:53:29.394 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T00:53:29.394 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-25T00:53:29.394 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T00:53:29.394 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T00:53:29.394 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T00:53:29.394 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-25T00:53:29.394 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T00:53:29.394 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T00:53:29.394 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T00:53:29.395 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-25T00:53:29.395 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T00:53:29.395 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-25T00:53:29.395 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T00:53:29.395 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-25T00:53:29.395 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T00:53:29.395 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T00:53:29.395 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T00:53:29.395 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T00:53:29.395 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-25T00:53:29.395 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T00:53:29.395 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-25T00:53:29.395 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T00:53:29.396 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T00:53:29.396 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-25T00:53:29.396 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T00:53:29.396 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T00:53:29.396 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T00:53:29.396 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-25T00:53:29.396 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T00:53:29.396 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T00:53:29.396 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-25T00:53:29.396 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T00:53:29.396 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T00:53:29.396 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T00:53:29.396 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T00:53:29.396 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T00:53:29.397 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T00:53:29.397 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T00:53:29.397 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T00:53:29.397 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T00:53:29.397 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T00:53:29.397 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T00:53:29.397 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-25T00:53:29.397 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T00:53:29.397 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T00:53:29.397 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T00:53:29.397 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T00:53:29.397 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-25T00:53:29.397 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T00:53:29.398 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-25T00:53:29.398 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-25T00:53:29.398 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T00:53:29.398 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-25T00:53:29.398 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T00:53:29.398 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T00:53:29.398 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T00:53:29.398 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T00:53:29.398 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T00:53:29.398 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T00:53:29.398 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-25T00:53:29.398 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T00:53:29.398 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-25T00:53:29.398 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-25T00:53:29.399 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-25T00:53:29.399 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T00:53:29.399 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T00:53:29.399 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T00:53:29.399 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T00:53:29.399 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T00:53:29.399 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T00:53:29.399 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T00:53:29.399 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T00:53:29.399 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T00:53:29.399 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T00:53:29.399 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T00:53:29.399 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T00:53:29.399 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T00:53:29.400 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T00:53:29.400 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T00:53:29.400 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T00:53:29.400 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T00:53:29.400 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T00:53:29.400 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-25T00:53:29.400 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-25T00:53:29.400 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T00:53:29.400 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T00:53:29.400 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T00:53:29.400 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T00:53:29.400 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-25T00:53:29.400 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-25T00:53:29.401 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T00:53:29.401 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T00:53:29.401 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-25T00:53:29.401 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T00:53:29.401 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-25T00:53:29.401 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-25T00:53:29.401 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T00:53:29.401 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-25T00:53:29.401 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T00:53:29.401 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-25T00:53:29.401 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T00:53:29.401 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T00:53:29.401 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T00:53:29.401 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T00:53:29.401 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T00:53:29.402 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-25T00:53:29.402 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-25T00:53:29.402 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T00:53:29.402 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T00:53:29.402 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T00:53:29.402 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T00:53:29.402 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-25T00:53:29.402 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T00:53:29.402 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-25T00:53:29.402 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T00:53:29.402 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T00:53:29.402 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-25T00:53:29.402 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T00:53:29.402 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T00:53:29.403 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T00:53:29.403 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T00:53:29.403 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T00:53:29.403 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T00:53:29.403 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T00:53:29.403 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T00:53:29.403 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T00:53:29.403 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T00:53:29.403 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-25T00:53:29.403 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T00:53:29.403 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T00:53:29.403 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T00:53:29.403 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T00:53:29.403 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T00:53:29.404 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T00:53:29.404 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T00:53:29.404 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-25T00:53:29.404 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T00:53:29.404 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-25T00:53:29.404 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T00:53:29.404 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T00:53:29.404 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T00:53:29.404 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T00:53:29.404 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T00:53:29.404 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T00:53:29.404 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T00:53:29.404 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T00:53:29.405 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T00:53:29.405 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T00:53:29.405 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T00:53:29.405 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T00:53:29.405 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T00:53:29.405 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T00:53:29.405 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T00:53:29.405 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T00:53:29.405 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T00:53:29.405 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T00:53:29.405 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T00:53:29.405 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T00:53:29.405 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T00:53:29.405 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T00:53:29.406 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T00:53:29.406 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T00:53:29.406 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T00:53:29.406 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T00:53:29.406 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T00:53:29.406 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T00:53:29.406 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T00:53:29.406 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T00:53:29.406 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T00:53:29.406 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T00:53:29.406 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T00:53:29.406 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T00:53:29.406 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T00:53:29.406 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T00:53:29.407 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T00:53:29.407 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T00:53:29.407 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T00:53:29.407 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T00:53:29.407 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T00:53:29.407 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T00:53:29.407 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T00:53:29.407 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T00:53:29.407 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T00:53:29.407 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-25T00:53:29.407 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T00:53:29.407 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T00:53:29.407 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T00:53:29.407 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T00:53:29.407 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T00:53:29.408 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T00:53:29.408 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T00:53:29.408 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T00:53:29.408 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T00:53:29.408 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T00:53:29.408 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T00:53:29.408 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T00:53:29.408 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T00:53:29.408 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T00:53:29.408 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T00:53:29.408 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-25T00:53:29.408 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T00:53:29.408 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T00:53:29.408 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T00:53:29.409 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-25T00:53:29.409 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-25T00:53:29.409 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T00:53:29.409 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T00:53:29.409 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T00:53:29.409 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T00:53:29.409 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-25T00:53:29.409 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T00:53:29.409 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T00:53:29.409 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T00:53:29.409 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T00:53:29.409 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-25T00:53:29.409 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-25T00:53:29.409 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T00:53:29.410 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T00:53:29.410 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T00:53:29.410 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-25T00:53:29.410 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-25T00:53:29.410 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-25T00:53:29.410 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-25T00:53:29.410 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-25T00:53:29.410 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-25T00:53:29.410 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-25T00:53:29.410 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-25T00:53:29.410 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-25T00:53:29.410 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-25T00:53:29.410 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T00:53:29.410 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T00:53:29.411 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-25T00:53:29.411 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T00:53:29.411 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T00:53:29.411 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T00:53:29.411 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T00:53:29.411 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T00:53:29.411 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T00:53:29.411 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T00:53:29.411 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T00:53:29.411 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T00:53:29.411 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T00:53:29.411 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T00:53:29.411 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T00:53:29.412 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-25T00:53:29.412 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T00:53:29.412 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T00:53:29.412 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T00:53:29.412 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T00:53:29.412 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T00:53:29.412 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T00:53:29.412 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T00:53:29.412 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T00:53:29.412 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T00:53:29.412 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T00:53:29.412 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T00:53:29.412 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T00:53:29.412 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T00:53:29.413 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T00:53:29.413 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T00:53:29.413 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T00:53:29.413 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T00:53:29.413 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T00:53:29.413 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T00:53:29.413 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T00:53:29.413 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T00:53:29.413 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T00:53:29.413 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T00:53:29.413 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T00:53:29.413 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T00:53:29.413 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T00:53:29.413 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-25T00:53:29.414 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-25T00:53:29.414 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T00:53:29.414 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T00:53:29.414 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T00:53:29.414 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-25T00:53:29.414 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T00:53:29.414 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T00:53:29.414 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-25T00:53:29.414 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T00:53:29.414 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-25T00:53:29.414 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-25T00:53:29.414 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T00:53:29.414 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-25T00:53:29.414 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T00:53:29.415 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-25T00:53:29.415 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T00:53:29.415 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T00:53:29.415 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-25T00:53:29.415 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T00:53:29.415 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-25T00:53:29.415 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T00:53:29.415 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-25T00:53:29.415 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T00:53:29.415 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-25T00:53:29.415 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T00:53:29.415 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-25T00:53:29.415 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T00:53:29.415 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-25T00:53:29.416 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T00:53:29.416 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-25T00:53:29.416 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-25T00:53:29.416 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-25T00:53:29.416 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T00:53:29.416 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-25T00:53:29.416 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-25T00:53:29.416 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T00:53:29.416 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-25T00:53:29.416 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-25T00:53:29.416 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T00:53:29.416 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-25T00:53:29.416 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-25T00:53:29.417 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T00:53:29.417 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-25T00:53:29.417 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T00:53:29.417 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-25T00:53:29.417 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T00:53:29.417 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-25T00:53:29.417 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T00:53:29.417 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-25T00:53:29.417 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T00:53:29.417 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-25T00:53:29.417 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T00:53:29.417 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-25T00:53:29.417 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T00:53:29.417 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-25T00:53:29.418 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T00:53:29.418 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-25T00:53:29.418 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-25T00:53:29.418 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T00:53:29.418 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-25T00:53:29.418 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T00:53:29.418 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-25T00:53:29.418 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T00:53:29.418 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T00:53:29.418 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T00:53:29.418 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T00:53:29.418 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-25T00:53:29.418 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T00:53:29.418 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T00:53:29.419 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-25T00:53:29.419 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T00:53:29.419 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-25T00:53:29.419 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T00:53:29.419 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-25T00:53:29.419 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T00:53:29.419 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T00:53:29.419 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T00:53:29.419 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T00:53:29.419 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T00:53:29.419 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T00:53:29.419 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T00:53:29.419 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T00:53:29.419 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T00:53:29.419 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T00:53:29.420 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T00:53:29.420 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T00:53:29.420 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T00:53:29.420 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T00:53:29.420 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T00:53:29.420 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T00:53:29.420 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T00:53:29.420 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T00:53:29.420 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T00:53:29.420 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T00:53:29.420 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T00:53:29.420 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T00:53:29.420 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T00:53:29.420 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T00:53:29.421 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T00:53:29.421 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T00:53:29.421 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T00:53:29.421 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T00:53:29.421 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T00:53:29.421 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T00:53:29.421 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T00:53:29.421 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T00:53:29.421 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T00:53:29.421 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T00:53:29.421 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T00:53:29.421 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T00:53:29.421 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T00:53:29.421 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T00:53:29.422 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T00:53:29.422 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T00:53:29.422 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T00:53:29.422 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T00:53:29.422 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T00:53:29.422 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T00:53:29.422 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T00:53:29.422 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T00:53:29.422 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T00:53:29.422 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T00:53:29.422 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T00:53:29.422 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T00:53:29.422 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-25T00:53:29.422 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-25T00:53:29.423 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T00:53:29.423 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-25T00:53:29.423 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T00:53:29.423 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-25T00:53:29.423 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T00:53:29.423 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T00:53:29.423 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T00:53:29.423 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T00:53:29.423 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T00:53:29.423 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-25T00:53:29.423 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T00:53:29.423 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-25T00:53:29.423 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T00:53:29.423 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-25T00:53:29.423 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T00:53:29.424 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-25T00:53:29.424 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T00:53:29.424 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-25T00:53:29.424 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-25T00:53:29.424 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T00:53:29.424 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-25T00:53:29.424 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T00:53:29.424 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T00:53:29.424 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-25T00:53:29.424 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T00:53:29.424 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-25T00:53:29.424 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T00:53:29.424 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-25T00:53:29.424 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T00:53:29.425 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-25T00:53:29.425 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T00:53:29.425 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-25T00:53:29.425 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T00:53:29.425 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-25T00:53:29.425 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-25T00:53:29.425 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-25T00:53:29.425 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T00:53:29.425 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-25T00:53:29.425 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-25T00:53:29.425 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-25T00:53:29.425 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-25T00:53:29.425 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T00:53:29.425 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T00:53:29.426 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-25T00:53:29.426 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T00:53:29.426 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T00:53:29.426 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T00:53:29.426 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T00:53:29.426 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-25T00:53:29.426 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T00:53:29.426 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T00:53:29.426 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-25T00:53:29.426 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T00:53:29.426 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-25T00:53:29.426 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T00:53:29.426 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-25T00:53:29.427 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T00:53:29.427 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T00:53:29.427 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T00:53:29.427 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T00:53:29.427 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-25T00:53:29.427 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-25T00:53:29.427 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-25T00:53:29.427 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T00:53:29.427 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-25T00:53:29.427 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-25T00:53:29.427 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T00:53:29.427 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T00:53:29.427 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T00:53:29.427 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T00:53:29.428 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T00:53:29.428 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T00:53:29.428 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T00:53:29.428 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T00:53:29.428 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-25T00:53:29.428 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T00:53:29.428 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T00:53:29.428 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T00:53:29.428 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T00:53:29.428 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T00:53:29.428 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-25T00:53:29.428 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T00:53:29.428 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-25T00:53:29.428 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T00:53:29.428 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T00:53:29.429 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T00:53:29.429 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T00:53:29.429 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-25T00:53:29.429 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T00:53:29.429 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-25T00:53:29.429 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T00:53:29.429 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T00:53:29.429 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-25T00:53:29.429 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T00:53:29.429 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T00:53:29.429 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T00:53:29.429 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T00:53:29.429 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T00:53:29.429 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T00:53:29.430 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T00:53:29.430 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T00:53:29.430 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T00:53:29.430 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T00:53:29.430 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T00:53:29.430 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T00:53:29.430 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T00:53:29.430 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T00:53:29.430 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T00:53:29.430 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T00:53:29.430 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T00:53:29.430 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T00:53:29.430 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T00:53:29.430 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T00:53:29.431 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T00:53:29.431 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-25T00:53:29.431 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T00:53:29.431 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T00:53:29.431 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-25T00:53:29.431 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T00:53:29.431 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-25T00:53:29.431 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-25T00:53:29.431 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T00:53:29.431 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-25T00:53:29.431 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T00:53:29.431 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T00:53:29.431 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T00:53:29.431 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T00:53:29.432 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-25T00:53:29.432 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-25T00:53:29.432 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-25T00:53:29.432 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T00:53:29.432 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T00:53:29.432 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T00:53:29.432 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T00:53:29.432 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T00:53:29.432 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-25T00:53:29.432 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-25T00:53:29.432 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T00:53:29.432 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T00:53:29.432 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T00:53:29.432 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-25T00:53:29.433 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T00:53:29.433 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T00:53:29.433 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T00:53:29.433 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T00:53:29.433 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T00:53:29.433 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T00:53:29.433 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T00:53:29.433 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T00:53:29.433 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T00:53:29.433 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T00:53:29.433 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-25T00:53:29.433 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T00:53:29.433 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T00:53:29.433 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T00:53:29.434 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T00:53:29.434 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T00:53:29.434 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T00:53:29.434 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T00:53:29.434 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T00:53:29.434 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T00:53:29.434 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T00:53:29.434 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T00:53:29.434 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T00:53:29.434 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T00:53:29.434 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T00:53:29.434 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T00:53:29.434 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T00:53:29.435 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T00:53:29.435 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T00:53:29.435 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T00:53:29.435 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T00:53:29.435 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T00:53:29.435 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T00:53:29.435 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T00:53:29.435 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T00:53:29.435 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T00:53:29.435 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T00:53:29.435 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T00:53:29.435 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T00:53:29.435 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T00:53:29.435 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T00:53:29.436 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T00:53:29.436 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T00:53:29.436 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T00:53:29.436 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T00:53:29.436 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T00:53:29.436 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T00:53:29.436 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T00:53:29.436 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T00:53:29.436 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-25T00:53:29.436 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T00:53:29.436 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T00:53:29.436 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T00:53:29.436 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T00:53:29.436 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T00:53:29.437 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T00:53:29.437 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T00:53:29.437 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-25T00:53:29.437 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T00:53:29.437 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T00:53:29.437 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T00:53:29.437 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T00:53:29.437 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T00:53:29.437 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T00:53:29.437 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T00:53:29.437 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T00:53:29.437 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T00:53:29.437 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T00:53:29.437 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T00:53:29.438 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T00:53:29.438 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T00:53:29.438 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T00:53:29.438 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T00:53:29.438 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T00:53:29.438 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T00:53:29.438 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T00:53:29.438 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T00:53:29.438 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T00:53:29.438 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T00:53:29.438 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T00:53:29.438 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T00:53:29.438 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T00:53:29.438 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T00:53:29.439 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T00:53:29.439 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T00:53:29.439 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-25T00:53:29.439 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T00:53:29.439 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-25T00:53:29.439 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T00:53:29.439 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T00:53:29.439 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T00:53:29.439 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T00:53:29.439 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T00:53:29.439 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T00:53:29.439 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T00:53:29.439 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-25T00:53:29.439 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T00:53:29.440 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-25T00:53:29.440 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T00:53:29.440 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-25T00:53:29.440 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T00:53:29.440 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-25T00:53:29.440 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-25T00:53:29.440 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T00:53:29.440 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T00:53:29.440 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T00:53:29.440 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T00:53:29.440 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T00:53:29.440 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T00:53:29.440 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T00:53:29.440 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T00:53:29.441 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T00:53:29.441 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T00:53:29.441 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T00:53:29.441 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T00:53:29.441 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T00:53:29.441 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T00:53:29.441 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T00:53:29.441 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T00:53:29.441 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-25T00:53:29.441 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T00:53:29.441 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T00:53:29.441 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T00:53:29.441 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T00:53:29.441 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T00:53:29.442 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T00:53:29.442 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T00:53:29.442 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T00:53:29.442 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T00:53:29.442 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T00:53:29.442 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T00:53:29.442 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T00:53:29.442 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T00:53:29.442 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T00:53:29.442 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T00:53:29.442 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T00:53:29.442 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T00:53:29.442 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T00:53:29.442 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T00:53:29.443 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T00:53:29.443 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T00:53:29.443 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T00:53:29.443 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T00:53:29.443 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T00:53:29.443 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T00:53:29.443 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T00:53:29.443 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T00:53:29.443 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T00:53:29.443 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T00:53:29.443 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T00:53:29.443 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T00:53:29.443 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T00:53:29.444 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T00:53:29.444 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T00:53:29.444 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T00:53:29.444 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T00:53:29.444 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T00:53:29.444 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T00:53:29.444 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T00:53:29.444 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T00:53:29.444 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T00:53:29.444 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T00:53:29.444 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T00:53:29.444 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T00:53:29.444 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T00:53:29.444 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T00:53:29.444 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T00:53:29.445 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T00:53:29.445 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T00:53:29.445 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T00:53:29.445 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T00:53:29.445 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T00:53:29.445 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T00:53:29.445 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T00:53:29.445 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T00:53:29.445 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T00:53:29.445 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T00:53:29.445 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T00:53:29.445 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-25T00:53:29.445 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T00:53:29.445 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T00:53:29.446 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T00:53:29.446 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T00:53:29.446 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-25T00:53:29.446 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T00:53:29.446 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T00:53:29.446 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T00:53:29.446 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T00:53:29.446 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T00:53:29.446 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T00:53:29.446 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T00:53:29.446 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T00:53:29.446 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T00:53:29.446 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T00:53:29.447 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T00:53:29.447 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T00:53:29.447 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T00:53:29.447 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T00:53:29.447 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T00:53:29.447 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T00:53:29.447 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T00:53:29.447 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T00:53:29.447 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T00:53:29.447 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T00:53:29.447 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T00:53:29.447 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T00:53:29.447 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T00:53:29.447 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T00:53:29.447 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T00:53:29.448 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T00:53:29.448 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T00:53:29.448 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T00:53:29.448 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T00:53:29.448 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T00:53:29.448 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T00:53:29.448 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T00:53:29.448 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T00:53:29.448 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T00:53:29.448 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T00:53:29.448 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T00:53:29.448 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T00:53:29.448 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T00:53:29.448 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T00:53:29.448 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T00:53:29.449 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T00:53:29.449 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T00:53:29.449 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T00:53:29.449 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T00:53:29.449 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T00:53:29.449 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T00:53:29.449 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T00:53:29.449 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T00:53:29.449 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T00:53:29.449 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T00:53:29.449 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T00:53:29.449 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T00:53:29.449 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T00:53:29.449 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T00:53:29.450 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T00:53:29.450 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T00:53:29.450 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T00:53:29.450 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T00:53:29.450 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T00:53:29.450 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T00:53:29.450 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T00:53:29.450 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T00:53:29.450 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T00:53:29.450 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T00:53:29.450 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T00:53:29.450 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T00:53:29.450 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T00:53:29.450 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T00:53:29.451 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T00:53:29.451 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T00:53:29.451 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T00:53:29.451 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-25T00:53:29.451 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T00:53:29.451 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-25T00:53:29.451 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T00:53:29.451 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-25T00:53:29.451 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T00:53:29.451 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T00:53:29.451 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T00:53:29.451 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T00:53:29.451 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-25T00:53:29.451 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T00:53:29.452 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-25T00:53:29.452 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T00:53:29.452 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T00:53:29.452 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T00:53:29.452 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-25T00:53:29.452 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T00:53:29.452 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T00:53:29.452 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T00:53:29.452 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T00:53:29.452 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T00:53:29.452 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T00:53:29.452 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T00:53:29.452 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T00:53:29.452 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T00:53:29.452 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T00:53:29.453 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T00:53:29.453 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T00:53:29.453 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T00:53:29.453 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T00:53:29.453 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T00:53:29.453 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T00:53:29.453 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T00:53:29.453 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T00:53:29.453 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T00:53:29.453 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T00:53:29.453 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T00:53:29.453 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T00:53:29.453 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T00:53:29.453 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T00:53:29.454 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T00:53:29.454 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T00:53:29.454 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T00:53:29.454 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T00:53:29.454 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T00:53:29.454 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T00:53:29.454 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T00:53:29.454 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T00:53:29.454 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T00:53:29.454 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T00:53:29.454 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T00:53:29.454 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T00:53:29.454 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T00:53:29.454 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T00:53:29.455 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T00:53:29.455 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T00:53:29.455 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T00:53:29.455 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T00:53:29.455 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T00:53:29.455 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-25T00:53:29.455 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T00:53:29.455 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T00:53:29.455 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-25T00:53:29.455 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-25T00:53:29.455 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T00:53:29.455 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T00:53:29.455 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T00:53:29.455 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-25T00:53:29.456 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T00:53:29.456 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-25T00:53:29.456 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T00:53:29.456 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T00:53:29.456 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-25T00:53:29.456 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T00:53:29.456 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-25T00:53:29.456 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T00:53:29.456 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T00:53:29.456 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T00:53:29.456 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T00:53:29.456 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T00:53:29.456 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-25T00:53:29.456 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T00:53:29.456 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-25T00:53:29.457 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-25T00:53:29.457 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-25T00:53:29.457 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-25T00:53:29.457 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-25T00:53:29.457 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-25T00:53:29.457 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T00:53:29.457 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T00:53:29.457 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T00:53:29.457 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T00:53:29.457 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T00:53:29.457 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T00:53:29.457 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T00:53:29.457 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T00:53:29.457 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T00:53:29.458 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T00:53:29.458 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T00:53:29.458 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T00:53:29.458 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T00:53:29.458 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T00:53:29.462 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T00:53:29.462 DEBUG:teuthology.orchestra.run.smithi031:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-25T00:53:29.489 INFO:teuthology.orchestra.run.smithi031.stdout:postinstall scriptlet (using /bin/sh): 2024-04-25T00:53:29.489 INFO:teuthology.orchestra.run.smithi031.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T00:53:29.489 INFO:teuthology.orchestra.run.smithi031.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T00:53:29.489 INFO:teuthology.orchestra.run.smithi031.stdout:fi 2024-04-25T00:53:29.489 INFO:teuthology.orchestra.run.smithi031.stdout:for file in vmlinuz System.map config; do 2024-04-25T00:53:29.489 INFO:teuthology.orchestra.run.smithi031.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-25T00:53:29.489 INFO:teuthology.orchestra.run.smithi031.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-25T00:53:29.489 INFO:teuthology.orchestra.run.smithi031.stdout: fi 2024-04-25T00:53:29.489 INFO:teuthology.orchestra.run.smithi031.stdout:done 2024-04-25T00:53:29.489 INFO:teuthology.orchestra.run.smithi031.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-25T00:53:29.490 INFO:teuthology.orchestra.run.smithi031.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-25T00:53:29.490 INFO:teuthology.orchestra.run.smithi031.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T00:53:29.490 INFO:teuthology.orchestra.run.smithi031.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T00:53:29.490 INFO:teuthology.orchestra.run.smithi031.stdout:fi 2024-04-25T00:53:29.490 INFO:teuthology.orchestra.run.smithi031.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-25T00:53:29.490 INFO:teuthology.orchestra.run.smithi031.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-25T00:53:29.490 INFO:teuthology.orchestra.run.smithi031.stdout:fi 2024-04-25T00:53:29.490 INFO:teuthology.orchestra.run.smithi031.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-25T00:53:29.490 INFO:teuthology.orchestra.run.smithi031.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T00:53:29.490 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-25T00:53:29.490 INFO:teuthology.orchestra.run.smithi031.stdout:fi 2024-04-25T00:53:29.491 DEBUG:teuthology.orchestra.run.smithi031:> sudo rpm -qi grub2-tools 2024-04-25T00:53:29.568 INFO:teuthology.orchestra.run.smithi031.stdout:Name : grub2-tools 2024-04-25T00:53:29.568 INFO:teuthology.orchestra.run.smithi031.stdout:Epoch : 1 2024-04-25T00:53:29.568 INFO:teuthology.orchestra.run.smithi031.stdout:Version : 2.06 2024-04-25T00:53:29.568 INFO:teuthology.orchestra.run.smithi031.stdout:Release : 68.el9 2024-04-25T00:53:29.568 INFO:teuthology.orchestra.run.smithi031.stdout:Architecture: x86_64 2024-04-25T00:53:29.568 INFO:teuthology.orchestra.run.smithi031.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-25T00:53:29.568 INFO:teuthology.orchestra.run.smithi031.stdout:Group : Unspecified 2024-04-25T00:53:29.568 INFO:teuthology.orchestra.run.smithi031.stdout:Size : 8264254 2024-04-25T00:53:29.568 INFO:teuthology.orchestra.run.smithi031.stdout:License : GPLv3+ 2024-04-25T00:53:29.568 INFO:teuthology.orchestra.run.smithi031.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-25T00:53:29.568 INFO:teuthology.orchestra.run.smithi031.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-25T00:53:29.568 INFO:teuthology.orchestra.run.smithi031.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-25T00:53:29.569 INFO:teuthology.orchestra.run.smithi031.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-25T00:53:29.569 INFO:teuthology.orchestra.run.smithi031.stdout:Packager : builder@centos.org 2024-04-25T00:53:29.569 INFO:teuthology.orchestra.run.smithi031.stdout:Vendor : CentOS 2024-04-25T00:53:29.569 INFO:teuthology.orchestra.run.smithi031.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-25T00:53:29.569 INFO:teuthology.orchestra.run.smithi031.stdout:Summary : Support tools for GRUB. 2024-04-25T00:53:29.569 INFO:teuthology.orchestra.run.smithi031.stdout:Description : 2024-04-25T00:53:29.569 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:53:29.569 INFO:teuthology.orchestra.run.smithi031.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-25T00:53:29.569 INFO:teuthology.orchestra.run.smithi031.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-25T00:53:29.569 INFO:teuthology.orchestra.run.smithi031.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-25T00:53:29.569 INFO:teuthology.orchestra.run.smithi031.stdout:hardware devices. 2024-04-25T00:53:29.569 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T00:53:29.569 INFO:teuthology.orchestra.run.smithi031.stdout:This subpackage provides tools for support of all platforms. 2024-04-25T00:53:29.570 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-25T00:53:29.570 INFO:teuthology.task.kernel:Updating grub on smithi031 to boot 6.9.0-rc3-g806ec36b2112 2024-04-25T00:53:29.570 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-25T00:53:30.130 INFO:teuthology.orchestra.run.smithi031.stderr:Generating grub configuration file ... 2024-04-25T00:53:31.584 INFO:teuthology.orchestra.run.smithi031.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-25T00:53:31.619 INFO:teuthology.orchestra.run.smithi031.stderr:done 2024-04-25T00:53:31.621 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-04-25T00:53:31.655 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.z5Tm9awe0B 2024-04-25T00:53:31.655 DEBUG:teuthology.orchestra.run.smithi031:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.z5Tm9awe0B 2024-04-25T00:53:31.724 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.z5Tm9awe0B 2024-04-25T00:53:31.863 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.z5Tm9awe0B is 6KB 2024-04-25T00:53:31.912 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.z5Tm9awe0B 2024-04-25T00:53:31.927 DEBUG:teuthology.orchestra.run.smithi031:> sudo /bin/ls /boot/loader/entries || true 2024-04-25T00:53:31.989 INFO:teuthology.orchestra.run.smithi031.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-25T00:53:31.989 INFO:teuthology.orchestra.run.smithi031.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-25T00:53:31.990 INFO:teuthology.orchestra.run.smithi031.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-25T00:53:31.990 INFO:teuthology.orchestra.run.smithi031.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-25T00:53:31.991 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-25T00:53:32.135 DEBUG:teuthology.orchestra.run.smithi031:> sudo shutdown -r now 2024-04-25T00:54:02.139 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-25T00:54:02.139 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-04-25T00:54:02.140 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:54:20.703 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-04-25T00:54:29.704 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-04-25T00:54:29.705 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:55:29.711 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T00:55:41.712 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-04-25T00:55:41.714 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:55:42.116 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-04-25T00:55:42.798 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-04-25T00:55:42.798 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T00:55:42.799 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-04-25T00:55:42.857 INFO:teuthology.orchestra.run.smithi031.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T00:55:42.857 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T00:55:42.857 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T00:55:42.857 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-25T00:55:43.858 DEBUG:teuthology.parallel:result is None 2024-04-25T00:55:43.858 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T00:55:43.866 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T00:55:43.866 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T00:55:43.874 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T00:55:43.877 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T00:55:43.926 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T00:55:43.933 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T00:55:43.941 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T00:55:43.941 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T00:55:43.949 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T00:55:43.951 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T00:55:44.016 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T00:55:44.023 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T00:55:44.023 DEBUG:teuthology.orchestra.run.smithi040:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T00:55:44.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:55:44.042 DEBUG:teuthology.orchestra.run.smithi169:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T00:55:44.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:55:44.060 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T00:55:44.067 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T00:55:44.085 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T00:55:44.131 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:55:44.140 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:55:44.148 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:55:44.166 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:55:44.200 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:55:44.253 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:55:44.259 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T00:55:44.267 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T00:55:44.268 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T00:55:44.275 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T00:55:44.279 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T00:55:44.360 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T00:55:44.370 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T00:55:44.370 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T00:55:44.378 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T00:55:44.381 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T00:55:44.428 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:55:44.512 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:55:44.609 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:55:44.695 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:55:44.784 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T00:55:44.784 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T00:55:44.872 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:55:44.918 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:55:44.973 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:55:45.060 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:55:45.104 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T00:55:45.104 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T00:55:45.172 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:55:45.215 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:55:45.288 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:55:45.332 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:55:45.373 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T00:55:45.374 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T00:55:45.439 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2024-04-25T00:55:45.447 DEBUG:teuthology.orchestra.run.smithi040:> sudo service rsyslog restart 2024-04-25T00:55:45.450 DEBUG:teuthology.orchestra.run.smithi169:> sudo service rsyslog restart 2024-04-25T00:55:45.487 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:55:45.516 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:55:45.560 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:55:46.007 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T00:55:46.019 INFO:teuthology.task.internal:Starting timer... 2024-04-25T00:55:46.019 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T00:55:46.051 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T00:55:46.061 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T00:55:46.061 DEBUG:teuthology.orchestra.run.smithi031:> sudo service auditd rotate 2024-04-25T00:55:46.274 INFO:teuthology.orchestra.run.smithi031.stdout:Rotating logs: 2024-04-25T00:55:46.280 DEBUG:teuthology.orchestra.run.smithi040:> sudo service auditd rotate 2024-04-25T00:55:46.347 INFO:teuthology.orchestra.run.smithi040.stdout:Rotating logs: 2024-04-25T00:55:46.349 DEBUG:teuthology.orchestra.run.smithi169:> sudo service auditd rotate 2024-04-25T00:55:46.404 INFO:teuthology.orchestra.run.smithi169.stdout:Rotating logs: 2024-04-25T00:55:46.407 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T00:55:46.407 DEBUG:teuthology.orchestra.run.smithi031:> /usr/sbin/getenforce 2024-04-25T00:55:46.466 INFO:teuthology.orchestra.run.smithi031.stdout:Permissive 2024-04-25T00:55:46.466 DEBUG:teuthology.orchestra.run.smithi040:> /usr/sbin/getenforce 2024-04-25T00:55:46.513 INFO:teuthology.orchestra.run.smithi040.stdout:Permissive 2024-04-25T00:55:46.514 DEBUG:teuthology.orchestra.run.smithi169:> /usr/sbin/getenforce 2024-04-25T00:55:46.537 INFO:teuthology.orchestra.run.smithi169.stdout:Permissive 2024-04-25T00:55:46.538 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi031.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi040.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi169.front.sepia.ceph.com': 'permissive'} 2024-04-25T00:55:46.538 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:55:46.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:55:46.616 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:55:46.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:55:46.644 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:55:46.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:55:46.669 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T00:55:46.669 DEBUG:teuthology.orchestra.run.smithi031:> sudo /usr/sbin/setenforce permissive 2024-04-25T00:55:46.745 DEBUG:teuthology.orchestra.run.smithi040:> sudo /usr/sbin/setenforce permissive 2024-04-25T00:55:46.771 DEBUG:teuthology.orchestra.run.smithi169:> sudo /usr/sbin/setenforce permissive 2024-04-25T00:55:46.806 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T00:55:46.819 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T00:55:46.824 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T00:55:46.951 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T00:55:46.969 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T00:55:46.971 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi031.front.sepia.ceph.com,smithi040.front.sepia.ceph.com,smithi169.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T01:00:58.853 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi031.front.sepia.ceph.com'), Remote(name='ubuntu@smithi040.front.sepia.ceph.com'), Remote(name='ubuntu@smithi169.front.sepia.ceph.com')] 2024-04-25T01:00:58.855 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-04-25T01:00:58.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T01:00:58.967 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-04-25T01:00:59.125 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-04-25T01:00:59.126 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-04-25T01:00:59.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T01:00:59.203 DEBUG:teuthology.orchestra.run.smithi040:> true 2024-04-25T01:00:59.284 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-04-25T01:00:59.284 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-25T01:00:59.285 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T01:00:59.358 DEBUG:teuthology.orchestra.run.smithi169:> true 2024-04-25T01:00:59.434 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-25T01:00:59.435 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T01:00:59.454 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T01:00:59.454 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T01:00:59.455 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:00:59.462 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T01:00:59.462 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:00:59.464 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T01:00:59.465 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:00:59.496 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T01:00:59.504 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T01:00:59.511 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T01:00:59.519 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T01:00:59.537 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: ntpd: command not found 2024-04-25T01:00:59.542 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: ntpd: command not found 2024-04-25T01:00:59.548 INFO:teuthology.orchestra.run.smithi040.stdout:506 Cannot talk to daemon 2024-04-25T01:00:59.553 INFO:teuthology.orchestra.run.smithi169.stdout:506 Cannot talk to daemon 2024-04-25T01:00:59.555 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T01:00:59.563 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T01:00:59.566 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T01:00:59.577 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T01:00:59.579 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T01:00:59.603 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T01:00:59.636 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-04-25T01:00:59.640 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T01:00:59.640 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-04-25T01:00:59.640 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:00:59.640 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:00:59.640 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:00:59.640 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:00:59.641 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 1: ntpq: command not found 2024-04-25T01:00:59.644 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T01:00:59.645 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2024-04-25T01:00:59.645 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:00:59.645 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:00:59.645 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:00:59.645 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:00:59.680 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: ntpd: command not found 2024-04-25T01:00:59.713 INFO:teuthology.orchestra.run.smithi031.stdout:506 Cannot talk to daemon 2024-04-25T01:00:59.754 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T01:00:59.799 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T01:00:59.953 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-04-25T01:00:59.962 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T01:00:59.962 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-04-25T01:00:59.962 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:00:59.962 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:00:59.962 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:00:59.962 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:00:59.963 INFO:teuthology.run_tasks:Running task install... 2024-04-25T01:00:59.972 DEBUG:teuthology.task.install:project ceph 2024-04-25T01:00:59.972 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-25T01:00:59.972 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-25T01:00:59.972 INFO:teuthology.task.install:Using flavor: default 2024-04-25T01:00:59.978 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T01:00:59.978 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T01:00:59.979 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-25T01:00:59.979 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:00:59.981 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-25T01:00:59.982 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:00:59.983 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-25T01:00:59.983 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:01:00.131 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-25T01:01:00.131 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-25T01:01:00.210 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-25T01:01:00.210 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-25T01:01:00.214 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-25T01:01:00.214 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-25T01:01:00.488 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T01:01:00.488 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T01:01:00.489 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T01:01:00.520 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T01:01:00.520 DEBUG:teuthology.orchestra.run.smithi040:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T01:01:00.569 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T01:01:00.569 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T01:01:00.569 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T01:01:00.581 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T01:01:00.581 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T01:01:00.581 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T01:01:00.590 DEBUG:teuthology.orchestra.run.smithi040:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T01:01:00.610 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T01:01:00.610 DEBUG:teuthology.orchestra.run.smithi169:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T01:01:00.665 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T01:01:00.665 DEBUG:teuthology.orchestra.run.smithi031:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T01:01:00.670 DEBUG:teuthology.orchestra.run.smithi040:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T01:01:00.679 DEBUG:teuthology.orchestra.run.smithi169:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T01:01:00.738 INFO:teuthology.orchestra.run.smithi040.stdout:check_obsoletes = 1 2024-04-25T01:01:00.739 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum clean all 2024-04-25T01:01:00.756 DEBUG:teuthology.orchestra.run.smithi169:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T01:01:00.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T01:01:00.821 INFO:teuthology.orchestra.run.smithi169.stdout:check_obsoletes = 1 2024-04-25T01:01:00.822 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum clean all 2024-04-25T01:01:00.912 DEBUG:teuthology.orchestra.run.smithi031:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T01:01:00.981 INFO:teuthology.orchestra.run.smithi040.stdout:48 files removed 2024-04-25T01:01:00.999 INFO:teuthology.orchestra.run.smithi031.stdout:check_obsoletes = 1 2024-04-25T01:01:01.003 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum clean all 2024-04-25T01:01:01.009 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T01:01:01.048 INFO:teuthology.orchestra.run.smithi169.stdout:48 files removed 2024-04-25T01:01:01.076 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T01:01:01.338 INFO:teuthology.orchestra.run.smithi031.stdout:48 files removed 2024-04-25T01:01:01.376 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T01:01:02.036 INFO:teuthology.orchestra.run.smithi040.stdout:ceph packages for x86_64 111 kB/s | 83 kB 00:00 2024-04-25T01:01:02.208 INFO:teuthology.orchestra.run.smithi169.stdout:ceph packages for x86_64 97 kB/s | 83 kB 00:00 2024-04-25T01:01:02.674 INFO:teuthology.orchestra.run.smithi031.stdout:ceph packages for x86_64 98 kB/s | 83 kB 00:00 2024-04-25T01:01:02.836 INFO:teuthology.orchestra.run.smithi040.stdout:ceph noarch packages 15 kB/s | 11 kB 00:00 2024-04-25T01:01:02.935 INFO:teuthology.orchestra.run.smithi169.stdout:ceph noarch packages 17 kB/s | 11 kB 00:00 2024-04-25T01:01:03.579 INFO:teuthology.orchestra.run.smithi040.stdout:ceph source packages 2.7 kB/s | 1.9 kB 00:00 2024-04-25T01:01:03.599 INFO:teuthology.orchestra.run.smithi031.stdout:ceph noarch packages 14 kB/s | 11 kB 00:00 2024-04-25T01:01:03.667 INFO:teuthology.orchestra.run.smithi169.stdout:ceph source packages 2.7 kB/s | 1.9 kB 00:00 2024-04-25T01:01:04.508 INFO:teuthology.orchestra.run.smithi031.stdout:ceph source packages 2.3 kB/s | 1.9 kB 00:00 2024-04-25T01:01:05.192 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - BaseOS 5.4 MB/s | 8.0 MB 00:01 2024-04-25T01:01:06.912 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - BaseOS 2.4 MB/s | 8.0 MB 00:03 2024-04-25T01:01:07.497 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 2.8 MB/s | 8.0 MB 00:02 2024-04-25T01:01:11.560 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - AppStream 3.6 MB/s | 19 MB 00:05 2024-04-25T01:01:12.931 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 4.5 MB/s | 19 MB 00:04 2024-04-25T01:01:15.829 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - AppStream 2.4 MB/s | 19 MB 00:07 2024-04-25T01:01:18.481 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.2 MB 00:01 2024-04-25T01:01:20.038 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-25T01:01:20.416 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-25T01:01:21.706 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-04-25T01:01:22.148 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - CRB 904 kB/s | 6.2 MB 00:06 2024-04-25T01:01:23.073 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 8.5 MB/s | 21 MB 00:02 2024-04-25T01:01:24.410 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-04-25T01:01:24.819 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 7.0 MB/s | 21 MB 00:02 2024-04-25T01:01:26.687 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 9.7 MB/s | 21 MB 00:02 2024-04-25T01:01:29.258 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-25T01:01:30.157 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-25T01:01:31.695 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-04-25T01:01:31.812 INFO:teuthology.orchestra.run.smithi031.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T01:01:31.814 INFO:teuthology.orchestra.run.smithi031.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T01:01:31.836 INFO:teuthology.orchestra.run.smithi031.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T01:01:31.837 INFO:teuthology.orchestra.run.smithi031.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T01:01:31.838 INFO:teuthology.orchestra.run.smithi031.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T01:01:31.840 INFO:teuthology.orchestra.run.smithi031.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T01:01:31.842 INFO:teuthology.orchestra.run.smithi031.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T01:01:31.846 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T01:01:31.847 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T01:01:31.924 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-25T01:01:31.946 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-04-25T01:01:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-04-25T01:01:31.946 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-04-25T01:01:31.946 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-25T01:01:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T01:01:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-25T01:01:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-25T01:01:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-25T01:01:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-25T01:01:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-25T01:01:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-25T01:01:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-25T01:01:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-25T01:01:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-25T01:01:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-25T01:01:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-25T01:01:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-25T01:01:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-25T01:01:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-25T01:01:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-25T01:01:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-25T01:01:31.948 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T01:01:31.948 INFO:teuthology.orchestra.run.smithi031.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T01:01:31.948 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T01:01:31.948 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-25T01:01:31.948 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-25T01:01:31.948 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-25T01:01:31.948 INFO:teuthology.orchestra.run.smithi031.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T01:01:31.948 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T01:01:31.948 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T01:01:31.948 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T01:01:31.948 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T01:01:31.948 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T01:01:31.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-25T01:01:31.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-25T01:01:31.949 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-25T01:01:31.949 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-25T01:01:31.949 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-25T01:01:31.949 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T01:01:31.949 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-25T01:01:31.949 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T01:01:31.949 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrading: 2024-04-25T01:01:31.949 INFO:teuthology.orchestra.run.smithi031.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-25T01:01:31.949 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T01:01:31.949 INFO:teuthology.orchestra.run.smithi031.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T01:01:31.949 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T01:01:31.949 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-04-25T01:01:31.949 INFO:teuthology.orchestra.run.smithi031.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T01:01:31.950 INFO:teuthology.orchestra.run.smithi031.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T01:01:31.950 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T01:01:31.950 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-25T01:01:31.950 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-25T01:01:31.950 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-25T01:01:31.950 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-25T01:01:31.950 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-25T01:01:31.950 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-25T01:01:31.950 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-25T01:01:31.950 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-25T01:01:31.950 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T01:01:31.950 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T01:01:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T01:01:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T01:01:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T01:01:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T01:01:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T01:01:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T01:01:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-25T01:01:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T01:01:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T01:01:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T01:01:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T01:01:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T01:01:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-25T01:01:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T01:01:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-25T01:01:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T01:01:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T01:01:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T01:01:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T01:01:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T01:01:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T01:01:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T01:01:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T01:01:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T01:01:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T01:01:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T01:01:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T01:01:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T01:01:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T01:01:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T01:01:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T01:01:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T01:01:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-25T01:01:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-25T01:01:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T01:01:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T01:01:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T01:01:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T01:01:31.954 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T01:01:31.954 INFO:teuthology.orchestra.run.smithi031.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T01:01:31.954 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T01:01:31.954 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T01:01:31.954 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T01:01:31.954 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T01:01:31.954 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T01:01:31.954 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T01:01:31.954 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T01:01:31.954 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T01:01:31.954 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T01:01:31.954 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T01:01:31.955 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T01:01:31.955 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T01:01:31.955 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T01:01:31.955 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T01:01:31.955 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T01:01:31.955 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T01:01:31.955 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T01:01:31.955 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T01:01:31.955 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T01:01:31.955 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T01:01:31.955 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T01:01:31.955 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T01:01:31.955 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T01:01:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T01:01:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T01:01:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T01:01:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T01:01:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T01:01:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T01:01:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T01:01:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T01:01:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T01:01:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T01:01:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T01:01:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T01:01:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T01:01:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T01:01:31.957 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T01:01:31.957 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T01:01:31.957 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T01:01:31.957 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T01:01:31.957 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T01:01:31.957 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T01:01:31.957 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T01:01:31.957 INFO:teuthology.orchestra.run.smithi031.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T01:01:31.957 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T01:01:31.957 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T01:01:31.957 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T01:01:31.957 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T01:01:31.958 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T01:01:31.958 INFO:teuthology.orchestra.run.smithi031.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T01:01:31.958 INFO:teuthology.orchestra.run.smithi031.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T01:01:31.958 INFO:teuthology.orchestra.run.smithi031.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T01:01:31.958 INFO:teuthology.orchestra.run.smithi031.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T01:01:31.958 INFO:teuthology.orchestra.run.smithi031.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T01:01:31.958 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T01:01:31.958 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T01:01:31.958 INFO:teuthology.orchestra.run.smithi031.stdout:Installing weak dependencies: 2024-04-25T01:01:31.958 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T01:01:31.958 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:01:31.958 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-25T01:01:31.958 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-04-25T01:01:31.959 INFO:teuthology.orchestra.run.smithi031.stdout:Install 148 Packages 2024-04-25T01:01:31.959 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrade 4 Packages 2024-04-25T01:01:31.959 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:01:31.959 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 220 M 2024-04-25T01:01:31.959 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-25T01:01:32.434 INFO:teuthology.orchestra.run.smithi040.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T01:01:32.435 INFO:teuthology.orchestra.run.smithi040.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T01:01:32.446 INFO:teuthology.orchestra.run.smithi040.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T01:01:32.447 INFO:teuthology.orchestra.run.smithi040.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T01:01:32.448 INFO:teuthology.orchestra.run.smithi040.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T01:01:32.449 INFO:teuthology.orchestra.run.smithi040.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T01:01:32.451 INFO:teuthology.orchestra.run.smithi040.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T01:01:32.453 INFO:teuthology.orchestra.run.smithi040.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T01:01:32.454 INFO:teuthology.orchestra.run.smithi040.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T01:01:32.519 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-25T01:01:32.539 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-04-25T01:01:32.539 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-04-25T01:01:32.539 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-04-25T01:01:32.539 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-04-25T01:01:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T01:01:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-25T01:01:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-25T01:01:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-25T01:01:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-25T01:01:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-25T01:01:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-25T01:01:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-25T01:01:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-25T01:01:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-25T01:01:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-25T01:01:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-25T01:01:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-25T01:01:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-25T01:01:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-25T01:01:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-25T01:01:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-25T01:01:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T01:01:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T01:01:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T01:01:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-25T01:01:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-25T01:01:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-25T01:01:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T01:01:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T01:01:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T01:01:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T01:01:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T01:01:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T01:01:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-25T01:01:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-25T01:01:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-25T01:01:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-25T01:01:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-25T01:01:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T01:01:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-25T01:01:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T01:01:32.542 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrading: 2024-04-25T01:01:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-25T01:01:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T01:01:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T01:01:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T01:01:32.543 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-04-25T01:01:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T01:01:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T01:01:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T01:01:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-25T01:01:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-25T01:01:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-25T01:01:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-25T01:01:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-25T01:01:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-25T01:01:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-25T01:01:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-25T01:01:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T01:01:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T01:01:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T01:01:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T01:01:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T01:01:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T01:01:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T01:01:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T01:01:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-25T01:01:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T01:01:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T01:01:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T01:01:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T01:01:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T01:01:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-25T01:01:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T01:01:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-25T01:01:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T01:01:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T01:01:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T01:01:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T01:01:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T01:01:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T01:01:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T01:01:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T01:01:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T01:01:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T01:01:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T01:01:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T01:01:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T01:01:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T01:01:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T01:01:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T01:01:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T01:01:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-25T01:01:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-25T01:01:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T01:01:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T01:01:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T01:01:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T01:01:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T01:01:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T01:01:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T01:01:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T01:01:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T01:01:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T01:01:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T01:01:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T01:01:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T01:01:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T01:01:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T01:01:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T01:01:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T01:01:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T01:01:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T01:01:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T01:01:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T01:01:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T01:01:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T01:01:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T01:01:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T01:01:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T01:01:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T01:01:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T01:01:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T01:01:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T01:01:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T01:01:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T01:01:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T01:01:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T01:01:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T01:01:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T01:01:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T01:01:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T01:01:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T01:01:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T01:01:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T01:01:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T01:01:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T01:01:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T01:01:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T01:01:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T01:01:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T01:01:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T01:01:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T01:01:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T01:01:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T01:01:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T01:01:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T01:01:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T01:01:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T01:01:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T01:01:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T01:01:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T01:01:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T01:01:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T01:01:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T01:01:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T01:01:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T01:01:32.551 INFO:teuthology.orchestra.run.smithi040.stdout:Installing weak dependencies: 2024-04-25T01:01:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T01:01:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T01:01:32.552 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-25T01:01:32.552 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-04-25T01:01:32.552 INFO:teuthology.orchestra.run.smithi040.stdout:Install 148 Packages 2024-04-25T01:01:32.552 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrade 4 Packages 2024-04-25T01:01:32.552 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T01:01:32.553 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 220 M 2024-04-25T01:01:32.553 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-04-25T01:01:33.160 INFO:teuthology.orchestra.run.smithi031.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-04-25T01:01:33.361 INFO:teuthology.orchestra.run.smithi031.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 4.4 MB/s | 890 kB 00:00 2024-04-25T01:01:33.410 INFO:teuthology.orchestra.run.smithi031.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 2.9 MB/s | 145 kB 00:00 2024-04-25T01:01:33.795 INFO:teuthology.orchestra.run.smithi169.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T01:01:33.796 INFO:teuthology.orchestra.run.smithi169.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T01:01:33.802 INFO:teuthology.orchestra.run.smithi031.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 6.0 MB/s | 2.4 MB 00:00 2024-04-25T01:01:33.806 INFO:teuthology.orchestra.run.smithi169.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T01:01:33.807 INFO:teuthology.orchestra.run.smithi169.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T01:01:33.808 INFO:teuthology.orchestra.run.smithi169.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T01:01:33.809 INFO:teuthology.orchestra.run.smithi169.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T01:01:33.810 INFO:teuthology.orchestra.run.smithi169.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T01:01:33.813 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T01:01:33.813 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T01:01:33.872 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-04-25T01:01:33.903 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-04-25T01:01:33.903 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-04-25T01:01:33.903 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-04-25T01:01:33.903 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-04-25T01:01:33.903 INFO:teuthology.orchestra.run.smithi169.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T01:01:33.903 INFO:teuthology.orchestra.run.smithi169.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-25T01:01:33.903 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-25T01:01:33.903 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-25T01:01:33.904 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-25T01:01:33.904 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-25T01:01:33.904 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-25T01:01:33.904 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-25T01:01:33.904 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-25T01:01:33.904 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-25T01:01:33.904 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-25T01:01:33.904 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-25T01:01:33.904 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-25T01:01:33.904 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-25T01:01:33.904 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-25T01:01:33.904 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-25T01:01:33.904 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-25T01:01:33.904 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T01:01:33.905 INFO:teuthology.orchestra.run.smithi169.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T01:01:33.905 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T01:01:33.905 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-25T01:01:33.905 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-25T01:01:33.905 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-25T01:01:33.905 INFO:teuthology.orchestra.run.smithi169.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T01:01:33.905 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T01:01:33.905 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T01:01:33.905 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T01:01:33.905 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T01:01:33.905 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T01:01:33.905 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-25T01:01:33.905 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-25T01:01:33.905 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-25T01:01:33.906 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-25T01:01:33.906 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-25T01:01:33.906 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T01:01:33.906 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-25T01:01:33.906 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T01:01:33.906 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrading: 2024-04-25T01:01:33.906 INFO:teuthology.orchestra.run.smithi169.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-25T01:01:33.906 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T01:01:33.906 INFO:teuthology.orchestra.run.smithi169.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T01:01:33.906 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T01:01:33.906 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-04-25T01:01:33.906 INFO:teuthology.orchestra.run.smithi169.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T01:01:33.907 INFO:teuthology.orchestra.run.smithi169.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T01:01:33.907 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T01:01:33.907 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-25T01:01:33.907 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-25T01:01:33.907 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-25T01:01:33.907 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-25T01:01:33.907 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-25T01:01:33.907 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-25T01:01:33.907 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-25T01:01:33.907 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-25T01:01:33.907 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T01:01:33.907 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T01:01:33.907 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T01:01:33.908 INFO:teuthology.orchestra.run.smithi169.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T01:01:33.908 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T01:01:33.908 INFO:teuthology.orchestra.run.smithi169.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T01:01:33.908 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T01:01:33.908 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T01:01:33.908 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-25T01:01:33.908 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T01:01:33.908 INFO:teuthology.orchestra.run.smithi169.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T01:01:33.908 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T01:01:33.908 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T01:01:33.908 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T01:01:33.908 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-25T01:01:33.908 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T01:01:33.908 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-25T01:01:33.909 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T01:01:33.909 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T01:01:33.909 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T01:01:33.909 INFO:teuthology.orchestra.run.smithi169.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T01:01:33.909 INFO:teuthology.orchestra.run.smithi169.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T01:01:33.909 INFO:teuthology.orchestra.run.smithi169.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T01:01:33.909 INFO:teuthology.orchestra.run.smithi169.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T01:01:33.909 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T01:01:33.909 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T01:01:33.909 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T01:01:33.909 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T01:01:33.909 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T01:01:33.909 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T01:01:33.909 INFO:teuthology.orchestra.run.smithi169.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T01:01:33.910 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T01:01:33.910 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T01:01:33.910 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T01:01:33.910 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-25T01:01:33.910 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-25T01:01:33.910 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T01:01:33.910 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T01:01:33.910 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T01:01:33.910 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T01:01:33.910 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T01:01:33.910 INFO:teuthology.orchestra.run.smithi169.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T01:01:33.910 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T01:01:33.910 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T01:01:33.911 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T01:01:33.911 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T01:01:33.911 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T01:01:33.911 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T01:01:33.911 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T01:01:33.911 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T01:01:33.911 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T01:01:33.911 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T01:01:33.911 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T01:01:33.911 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T01:01:33.911 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T01:01:33.911 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T01:01:33.911 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T01:01:33.911 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T01:01:33.912 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T01:01:33.912 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T01:01:33.912 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T01:01:33.912 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T01:01:33.912 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T01:01:33.912 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T01:01:33.912 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T01:01:33.912 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T01:01:33.912 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T01:01:33.912 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T01:01:33.912 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T01:01:33.912 INFO:teuthology.orchestra.run.smithi169.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T01:01:33.912 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T01:01:33.912 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T01:01:33.913 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T01:01:33.913 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T01:01:33.913 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T01:01:33.913 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T01:01:33.913 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T01:01:33.913 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T01:01:33.913 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T01:01:33.913 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T01:01:33.913 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T01:01:33.913 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T01:01:33.913 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T01:01:33.913 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T01:01:33.913 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T01:01:33.913 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T01:01:33.914 INFO:teuthology.orchestra.run.smithi169.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T01:01:33.914 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T01:01:33.914 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T01:01:33.914 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T01:01:33.914 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T01:01:33.914 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T01:01:33.914 INFO:teuthology.orchestra.run.smithi169.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T01:01:33.914 INFO:teuthology.orchestra.run.smithi169.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T01:01:33.914 INFO:teuthology.orchestra.run.smithi169.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T01:01:33.914 INFO:teuthology.orchestra.run.smithi169.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T01:01:33.914 INFO:teuthology.orchestra.run.smithi169.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T01:01:33.914 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T01:01:33.914 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T01:01:33.914 INFO:teuthology.orchestra.run.smithi169.stdout:Installing weak dependencies: 2024-04-25T01:01:33.915 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T01:01:33.915 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T01:01:33.915 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-04-25T01:01:33.915 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-04-25T01:01:33.915 INFO:teuthology.orchestra.run.smithi169.stdout:Install 148 Packages 2024-04-25T01:01:33.915 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrade 4 Packages 2024-04-25T01:01:33.915 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T01:01:33.915 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 220 M 2024-04-25T01:01:33.916 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-04-25T01:01:33.919 INFO:teuthology.orchestra.run.smithi031.stdout:(5/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 5.2 MB/s | 5.4 MB 00:01 2024-04-25T01:01:34.011 INFO:teuthology.orchestra.run.smithi031.stdout:(6/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 7.2 MB/s | 1.5 MB 00:00 2024-04-25T01:01:34.636 INFO:teuthology.orchestra.run.smithi031.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 6.6 MB/s | 4.7 MB 00:00 2024-04-25T01:01:35.212 INFO:teuthology.orchestra.run.smithi031.stdout:(8/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x86 14 MB/s | 17 MB 00:01 2024-04-25T01:01:35.262 INFO:teuthology.orchestra.run.smithi031.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 516 kB/s | 25 kB 00:00 2024-04-25T01:01:37.954 INFO:teuthology.orchestra.run.smithi031.stdout:(10/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 18 MB/s | 48 MB 00:02 2024-04-25T01:01:38.146 INFO:teuthology.orchestra.run.smithi031.stdout:(11/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 2.9 MB/s | 10 MB 00:03 2024-04-25T01:01:38.179 INFO:teuthology.orchestra.run.smithi031.stdout:(12/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 992 kB/s | 222 kB 00:00 2024-04-25T01:01:38.215 INFO:teuthology.orchestra.run.smithi031.stdout:(13/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 412 kB/s | 27 kB 00:00 2024-04-25T01:01:38.238 INFO:teuthology.orchestra.run.smithi031.stdout:(14/152): libcephfs-devel-19.0.0-2445.gd2c45d97 548 kB/s | 32 kB 00:00 2024-04-25T01:01:38.497 INFO:teuthology.orchestra.run.smithi031.stdout:(15/152): ceph-common-19.0.0-2445.gd2c45d97.el9 3.7 MB/s | 21 MB 00:05 2024-04-25T01:01:38.530 INFO:teuthology.orchestra.run.smithi031.stdout:(16/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 561 kB/s | 163 kB 00:00 2024-04-25T01:01:38.580 INFO:teuthology.orchestra.run.smithi031.stdout:(17/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 2.0 MB/s | 745 kB 00:00 2024-04-25T01:01:38.622 INFO:teuthology.orchestra.run.smithi031.stdout:(18/152): libradosstriper1-19.0.0-2445.gd2c45d9 5.4 MB/s | 501 kB 00:00 2024-04-25T01:01:38.673 INFO:teuthology.orchestra.run.smithi031.stdout:(19/152): librados-devel-19.0.0-2445.gd2c45d97. 727 kB/s | 127 kB 00:00 2024-04-25T01:01:38.689 INFO:teuthology.orchestra.run.smithi031.stdout:(20/152): python3-ceph-argparse-19.0.0-2445.gd2 681 kB/s | 45 kB 00:00 2024-04-25T01:01:38.740 INFO:teuthology.orchestra.run.smithi031.stdout:(21/152): python3-ceph-common-19.0.0-2445.gd2c4 2.0 MB/s | 133 kB 00:00 2024-04-25T01:01:38.765 INFO:teuthology.orchestra.run.smithi031.stdout:(22/152): python3-cephfs-19.0.0-2445.gd2c45d97. 2.1 MB/s | 162 kB 00:00 2024-04-25T01:01:38.957 INFO:teuthology.orchestra.run.smithi031.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 1.5 MB/s | 299 kB 00:00 2024-04-25T01:01:39.036 INFO:teuthology.orchestra.run.smithi031.stdout:(24/152): python3-rados-19.0.0-2445.gd2c45d97.e 1.1 MB/s | 321 kB 00:00 2024-04-25T01:01:39.074 INFO:teuthology.orchestra.run.smithi031.stdout:(25/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 853 kB/s | 99 kB 00:00 2024-04-25T01:01:39.099 INFO:teuthology.orchestra.run.smithi031.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-25T01:01:39.183 INFO:teuthology.orchestra.run.smithi031.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 2.0 MB/s | 171 kB 00:00 2024-04-25T01:01:39.275 INFO:teuthology.orchestra.run.smithi031.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 349 kB/s | 32 kB 00:00 2024-04-25T01:01:39.342 INFO:teuthology.orchestra.run.smithi031.stdout:(29/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 12 MB/s | 3.1 MB 00:00 2024-04-25T01:01:39.367 INFO:teuthology.orchestra.run.smithi031.stdout:(30/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.5 MB/s | 141 kB 00:00 2024-04-25T01:01:39.717 INFO:teuthology.orchestra.run.smithi031.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 9.4 MB/s | 3.5 MB 00:00 2024-04-25T01:01:39.776 INFO:teuthology.orchestra.run.smithi031.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 4.2 MB/s | 249 kB 00:00 2024-04-25T01:01:39.851 INFO:teuthology.orchestra.run.smithi031.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 666 kB/s | 49 kB 00:00 2024-04-25T01:01:39.901 INFO:teuthology.orchestra.run.smithi031.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 313 kB/s | 15 kB 00:00 2024-04-25T01:01:39.977 INFO:teuthology.orchestra.run.smithi031.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 3.5 MB/s | 264 kB 00:00 2024-04-25T01:01:40.086 INFO:teuthology.orchestra.run.smithi031.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 6.9 MB/s | 762 kB 00:00 2024-04-25T01:01:40.216 INFO:teuthology.orchestra.run.smithi031.stdout:(37/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 3.3 MB/s | 5.3 MB 00:01 2024-04-25T01:01:40.244 INFO:teuthology.orchestra.run.smithi031.stdout:(38/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 116 kB/s | 18 kB 00:00 2024-04-25T01:01:40.353 INFO:teuthology.orchestra.run.smithi031.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 592 kB/s | 80 kB 00:00 2024-04-25T01:01:40.436 INFO:teuthology.orchestra.run.smithi031.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 4.1 MB/s | 803 kB 00:00 2024-04-25T01:01:40.462 INFO:teuthology.orchestra.run.smithi031.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2024-04-25T01:01:40.495 INFO:teuthology.orchestra.run.smithi031.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.3 MB/s | 253 kB 00:00 2024-04-25T01:01:40.539 INFO:teuthology.orchestra.run.smithi031.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-04-25T01:01:40.610 INFO:teuthology.orchestra.run.smithi031.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 8.3 MB/s | 1.2 MB 00:00 2024-04-25T01:01:40.638 INFO:teuthology.orchestra.run.smithi031.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-04-25T01:01:40.672 INFO:teuthology.orchestra.run.smithi031.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.0 MB/s | 125 kB 00:00 2024-04-25T01:01:40.705 INFO:teuthology.orchestra.run.smithi031.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 215 kB 00:00 2024-04-25T01:01:41.058 INFO:teuthology.orchestra.run.smithi031.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 689 kB 00:00 2024-04-25T01:01:41.097 INFO:teuthology.orchestra.run.smithi031.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 2.4 MB/s | 943 kB 00:00 2024-04-25T01:01:41.256 INFO:teuthology.orchestra.run.smithi031.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 136 kB/s | 21 kB 00:00 2024-04-25T01:01:41.281 INFO:teuthology.orchestra.run.smithi031.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 477 kB/s | 107 kB 00:00 2024-04-25T01:01:41.423 INFO:teuthology.orchestra.run.smithi031.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 232 kB/s | 33 kB 00:00 2024-04-25T01:01:41.557 INFO:teuthology.orchestra.run.smithi031.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 1.0 MB/s | 313 kB 00:00 2024-04-25T01:01:41.699 INFO:teuthology.orchestra.run.smithi031.stdout:(54/152): ceph-mgr-diskprediction-local-19.0.0- 3.2 MB/s | 7.4 MB 00:02 2024-04-25T01:01:41.799 INFO:teuthology.orchestra.run.smithi031.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 75 kB/s | 18 kB 00:00 2024-04-25T01:01:41.815 INFO:teuthology.orchestra.run.smithi031.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 96 kB/s | 11 kB 00:00 2024-04-25T01:01:41.891 INFO:teuthology.orchestra.run.smithi040.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-04-25T01:01:42.041 INFO:teuthology.orchestra.run.smithi031.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.8 MB/s | 3.0 MB 00:00 2024-04-25T01:01:42.068 INFO:teuthology.orchestra.run.smithi031.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 180 kB/s | 45 kB 00:00 2024-04-25T01:01:42.093 INFO:teuthology.orchestra.run.smithi031.stdout:(59/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 546 kB/s | 160 kB 00:00 2024-04-25T01:01:42.133 INFO:teuthology.orchestra.run.smithi040.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 3.6 MB/s | 890 kB 00:00 2024-04-25T01:01:42.208 INFO:teuthology.orchestra.run.smithi040.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 1.9 MB/s | 145 kB 00:00 2024-04-25T01:01:42.260 INFO:teuthology.orchestra.run.smithi031.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 153 kB/s | 25 kB 00:00 2024-04-25T01:01:42.377 INFO:teuthology.orchestra.run.smithi031.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 1.8 MB/s | 582 kB 00:00 2024-04-25T01:01:42.545 INFO:teuthology.orchestra.run.smithi031.stdout:(62/152): librdkafka-1.6.1-102.el9.x86_64.rpm 1.3 MB/s | 662 kB 00:00 2024-04-25T01:01:42.578 INFO:teuthology.orchestra.run.smithi031.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 150 kB/s | 47 kB 00:00 2024-04-25T01:01:42.592 INFO:teuthology.orchestra.run.smithi040.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 6.2 MB/s | 2.4 MB 00:00 2024-04-25T01:01:42.709 INFO:teuthology.orchestra.run.smithi040.stdout:(5/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 5.2 MB/s | 5.4 MB 00:01 2024-04-25T01:01:42.778 INFO:teuthology.orchestra.run.smithi031.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 177 kB/s | 35 kB 00:00 2024-04-25T01:01:42.812 INFO:teuthology.orchestra.run.smithi031.stdout:(65/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 675 kB/s | 292 kB 00:00 2024-04-25T01:01:42.837 INFO:teuthology.orchestra.run.smithi031.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 647 kB/s | 188 kB 00:00 2024-04-25T01:01:42.984 INFO:teuthology.orchestra.run.smithi040.stdout:(6/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 3.8 MB/s | 1.5 MB 00:00 2024-04-25T01:01:42.995 INFO:teuthology.orchestra.run.smithi169.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-04-25T01:01:43.304 INFO:teuthology.orchestra.run.smithi031.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 3.4 MB/s | 1.6 MB 00:00 2024-04-25T01:01:43.571 INFO:teuthology.orchestra.run.smithi031.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 6.0 MB/s | 4.4 MB 00:00 2024-04-25T01:01:43.659 INFO:teuthology.orchestra.run.smithi040.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 4.9 MB/s | 4.7 MB 00:00 2024-04-25T01:01:43.679 INFO:teuthology.orchestra.run.smithi031.stdout:(69/152): openblas-openmp-0.3.21-2.el9.x86_64.r 5.3 MB/s | 4.7 MB 00:00 2024-04-25T01:01:43.796 INFO:teuthology.orchestra.run.smithi031.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 1.8 MB/s | 911 kB 00:00 2024-04-25T01:01:43.896 INFO:teuthology.orchestra.run.smithi031.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 411 kB/s | 133 kB 00:00 2024-04-25T01:01:44.122 INFO:teuthology.orchestra.run.smithi031.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 390 kB/s | 87 kB 00:00 2024-04-25T01:01:44.213 INFO:teuthology.orchestra.run.smithi031.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 2.4 MB/s | 1.0 MB 00:00 2024-04-25T01:01:44.470 INFO:teuthology.orchestra.run.smithi169.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 603 kB/s | 890 kB 00:01 2024-04-25T01:01:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 6.4 MB/s | 5.8 MB 00:00 2024-04-25T01:01:44.631 INFO:teuthology.orchestra.run.smithi031.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 587 kB/s | 244 kB 00:00 2024-04-25T01:01:44.737 INFO:teuthology.orchestra.run.smithi169.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 543 kB/s | 145 kB 00:00 2024-04-25T01:01:44.797 INFO:teuthology.orchestra.run.smithi031.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 155 kB/s | 26 kB 00:00 2024-04-25T01:01:44.906 INFO:teuthology.orchestra.run.smithi031.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 764 kB/s | 248 kB 00:00 2024-04-25T01:01:44.931 INFO:teuthology.orchestra.run.smithi031.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 144 kB/s | 19 kB 00:00 2024-04-25T01:01:45.046 INFO:teuthology.orchestra.run.smithi169.stdout:(4/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 2.2 MB/s | 5.4 MB 00:02 2024-04-25T01:01:45.098 INFO:teuthology.orchestra.run.smithi031.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 209 kB/s | 35 kB 00:00 2024-04-25T01:01:45.173 INFO:teuthology.orchestra.run.smithi031.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 647 kB/s | 172 kB 00:00 2024-04-25T01:01:45.407 INFO:teuthology.orchestra.run.smithi031.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 4.6 MB/s | 6.0 MB 00:01 2024-04-25T01:01:45.529 INFO:teuthology.orchestra.run.smithi169.stdout:(5/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 3.1 MB/s | 1.5 MB 00:00 2024-04-25T01:01:45.574 INFO:teuthology.orchestra.run.smithi031.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 627 kB/s | 250 kB 00:00 2024-04-25T01:01:45.707 INFO:teuthology.orchestra.run.smithi031.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 743 kB/s | 222 kB 00:00 2024-04-25T01:01:45.791 INFO:teuthology.orchestra.run.smithi031.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 357 kB/s | 77 kB 00:00 2024-04-25T01:01:45.794 INFO:teuthology.orchestra.run.smithi040.stdout:(8/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el9 4.8 MB/s | 10 MB 00:02 2024-04-25T01:01:45.868 INFO:teuthology.orchestra.run.smithi040.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 336 kB/s | 25 kB 00:00 2024-04-25T01:01:45.899 INFO:teuthology.orchestra.run.smithi031.stdout:(85/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.4 MB/s | 5.1 MB 00:00 2024-04-25T01:01:45.925 INFO:teuthology.orchestra.run.smithi031.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 195 kB/s | 42 kB 00:00 2024-04-25T01:01:46.125 INFO:teuthology.orchestra.run.smithi031.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 708 kB/s | 159 kB 00:00 2024-04-25T01:01:46.159 INFO:teuthology.orchestra.run.smithi031.stdout:(88/152): python3-protobuf-3.14.0-13.el9.noarch 735 kB/s | 269 kB 00:00 2024-04-25T01:01:46.209 INFO:teuthology.orchestra.run.smithi031.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 985 kB/s | 279 kB 00:00 2024-04-25T01:01:46.326 INFO:teuthology.orchestra.run.smithi031.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 256 kB/s | 51 kB 00:00 2024-04-25T01:01:46.351 INFO:teuthology.orchestra.run.smithi031.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 280 kB/s | 54 kB 00:00 2024-04-25T01:01:46.535 INFO:teuthology.orchestra.run.smithi031.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 200 kB/s | 42 kB 00:00 2024-04-25T01:01:46.560 INFO:teuthology.orchestra.run.smithi031.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 212 kB/s | 44 kB 00:00 2024-04-25T01:01:46.627 INFO:teuthology.orchestra.run.smithi040.stdout:(10/152): ceph-common-19.0.0-2445.gd2c45d97.el9 4.2 MB/s | 21 MB 00:04 2024-04-25T01:01:46.727 INFO:teuthology.orchestra.run.smithi031.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 216 kB/s | 36 kB 00:00 2024-04-25T01:01:46.768 INFO:teuthology.orchestra.run.smithi040.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 1.5 MB/s | 222 kB 00:00 2024-04-25T01:01:46.827 INFO:teuthology.orchestra.run.smithi031.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 758 kB/s | 221 kB 00:00 2024-04-25T01:01:46.843 INFO:teuthology.orchestra.run.smithi040.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 361 kB/s | 27 kB 00:00 2024-04-25T01:01:46.896 INFO:teuthology.orchestra.run.smithi169.stdout:(6/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 3.4 MB/s | 4.7 MB 00:01 2024-04-25T01:01:46.911 INFO:teuthology.orchestra.run.smithi040.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 475 kB/s | 32 kB 00:00 2024-04-25T01:01:46.911 INFO:teuthology.orchestra.run.smithi031.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 260 kB/s | 47 kB 00:00 2024-04-25T01:01:47.111 INFO:teuthology.orchestra.run.smithi040.stdout:(14/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 4.1 MB/s | 17 MB 00:04 2024-04-25T01:01:47.145 INFO:teuthology.orchestra.run.smithi040.stdout:(15/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 3.1 MB/s | 745 kB 00:00 2024-04-25T01:01:47.170 INFO:teuthology.orchestra.run.smithi040.stdout:(16/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 2.7 MB/s | 163 kB 00:00 2024-04-25T01:01:47.191 INFO:teuthology.orchestra.run.smithi031.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 207 kB/s | 58 kB 00:00 2024-04-25T01:01:47.212 INFO:teuthology.orchestra.run.smithi040.stdout:(17/152): librados-devel-19.0.0-2445.gd2c45d97. 1.8 MB/s | 127 kB 00:00 2024-04-25T01:01:47.228 INFO:teuthology.orchestra.run.smithi031.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 763 kB/s | 305 kB 00:00 2024-04-25T01:01:47.321 INFO:teuthology.orchestra.run.smithi040.stdout:(18/152): libradosstriper1-19.0.0-2445.gd2c45d9 3.3 MB/s | 501 kB 00:00 2024-04-25T01:01:47.388 INFO:teuthology.orchestra.run.smithi040.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 678 kB/s | 45 kB 00:00 2024-04-25T01:01:47.411 INFO:teuthology.orchestra.run.smithi031.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 349 kB/s | 64 kB 00:00 2024-04-25T01:01:47.421 INFO:teuthology.orchestra.run.smithi169.stdout:(7/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 901 kB/s | 2.4 MB 00:02 2024-04-25T01:01:47.437 INFO:teuthology.orchestra.run.smithi031.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 383 kB/s | 94 kB 00:00 2024-04-25T01:01:47.463 INFO:teuthology.orchestra.run.smithi040.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.7 MB/s | 133 kB 00:00 2024-04-25T01:01:47.546 INFO:teuthology.orchestra.run.smithi040.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 1.9 MB/s | 162 kB 00:00 2024-04-25T01:01:47.585 INFO:teuthology.orchestra.run.smithi031.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 152 kB/s | 22 kB 00:00 2024-04-25T01:01:47.613 INFO:teuthology.orchestra.run.smithi031.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 224 kB/s | 45 kB 00:00 2024-04-25T01:01:47.688 INFO:teuthology.orchestra.run.smithi040.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 2.2 MB/s | 321 kB 00:00 2024-04-25T01:01:47.754 INFO:teuthology.orchestra.run.smithi031.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 5.0 MB/s | 863 kB 00:00 2024-04-25T01:01:47.830 INFO:teuthology.orchestra.run.smithi040.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 2.1 MB/s | 299 kB 00:00 2024-04-25T01:01:47.929 INFO:teuthology.orchestra.run.smithi031.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-04-25T01:01:47.988 INFO:teuthology.orchestra.run.smithi031.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 338 kB/s | 19 kB 00:00 2024-04-25T01:01:48.022 INFO:teuthology.orchestra.run.smithi031.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.1 MB/s | 309 kB 00:00 2024-04-25T01:01:48.072 INFO:teuthology.orchestra.run.smithi031.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 530 kB/s | 26 kB 00:00 2024-04-25T01:01:48.105 INFO:teuthology.orchestra.run.smithi040.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 362 kB/s | 99 kB 00:00 2024-04-25T01:01:48.122 INFO:teuthology.orchestra.run.smithi031.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 989 kB/s | 49 kB 00:00 2024-04-25T01:01:48.181 INFO:teuthology.orchestra.run.smithi040.stdout:(25/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 5.5 MB/s | 5.3 MB 00:00 2024-04-25T01:01:48.189 INFO:teuthology.orchestra.run.smithi031.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-25T01:01:48.198 INFO:teuthology.orchestra.run.smithi040.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 926 kB/s | 85 kB 00:00 2024-04-25T01:01:48.273 INFO:teuthology.orchestra.run.smithi040.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 2.2 MB/s | 171 kB 00:00 2024-04-25T01:01:48.273 INFO:teuthology.orchestra.run.smithi031.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-25T01:01:48.307 INFO:teuthology.orchestra.run.smithi031.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-25T01:01:48.348 INFO:teuthology.orchestra.run.smithi040.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 423 kB/s | 32 kB 00:00 2024-04-25T01:01:48.381 INFO:teuthology.orchestra.run.smithi031.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.6 MB/s | 838 kB 00:00 2024-04-25T01:01:48.432 INFO:teuthology.orchestra.run.smithi040.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.7 MB/s | 141 kB 00:00 2024-04-25T01:01:48.440 INFO:teuthology.orchestra.run.smithi031.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-04-25T01:01:48.474 INFO:teuthology.orchestra.run.smithi031.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 475 kB/s | 43 kB 00:00 2024-04-25T01:01:48.633 INFO:teuthology.orchestra.run.smithi031.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 168 kB/s | 32 kB 00:00 2024-04-25T01:01:48.658 INFO:teuthology.orchestra.run.smithi031.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 82 kB/s | 15 kB 00:00 2024-04-25T01:01:48.732 INFO:teuthology.orchestra.run.smithi040.stdout:(30/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 5.6 MB/s | 3.1 MB 00:00 2024-04-25T01:01:48.884 INFO:teuthology.orchestra.run.smithi031.stdout:(117/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.1 MB/s | 16 MB 00:02 2024-04-25T01:01:48.917 INFO:teuthology.orchestra.run.smithi031.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 607 kB/s | 172 kB 00:00 2024-04-25T01:01:48.950 INFO:teuthology.orchestra.run.smithi031.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 1.2 MB/s | 358 kB 00:00 2024-04-25T01:01:48.976 INFO:teuthology.orchestra.run.smithi031.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 612 kB/s | 35 kB 00:00 2024-04-25T01:01:49.026 INFO:teuthology.orchestra.run.smithi031.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-04-25T01:01:49.152 INFO:teuthology.orchestra.run.smithi031.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-25T01:01:49.185 INFO:teuthology.orchestra.run.smithi031.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 915 kB/s | 144 kB 00:00 2024-04-25T01:01:49.210 INFO:teuthology.orchestra.run.smithi031.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 186 kB/s | 11 kB 00:00 2024-04-25T01:01:49.244 INFO:teuthology.orchestra.run.smithi031.stdout:(125/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 844 kB/s | 303 kB 00:00 2024-04-25T01:01:49.269 INFO:teuthology.orchestra.run.smithi031.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 212 kB/s | 18 kB 00:00 2024-04-25T01:01:49.294 INFO:teuthology.orchestra.run.smithi031.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 278 kB/s | 23 kB 00:00 2024-04-25T01:01:49.320 INFO:teuthology.orchestra.run.smithi031.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 260 kB/s | 19 kB 00:00 2024-04-25T01:01:49.345 INFO:teuthology.orchestra.run.smithi031.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 263 kB/s | 20 kB 00:00 2024-04-25T01:01:49.378 INFO:teuthology.orchestra.run.smithi031.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 800 kB/s | 46 kB 00:00 2024-04-25T01:01:49.403 INFO:teuthology.orchestra.run.smithi031.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-25T01:01:49.437 INFO:teuthology.orchestra.run.smithi031.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.0 MB/s | 58 kB 00:00 2024-04-25T01:01:49.449 INFO:teuthology.orchestra.run.smithi040.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 3.5 MB/s | 3.5 MB 00:01 2024-04-25T01:01:49.471 INFO:teuthology.orchestra.run.smithi031.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-04-25T01:01:49.517 INFO:teuthology.orchestra.run.smithi031.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 210 kB/s | 16 kB 00:00 2024-04-25T01:01:49.541 INFO:teuthology.orchestra.run.smithi040.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 2.7 MB/s | 249 kB 00:00 2024-04-25T01:01:49.546 INFO:teuthology.orchestra.run.smithi031.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-25T01:01:49.572 INFO:teuthology.orchestra.run.smithi031.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 526 kB/s | 28 kB 00:00 2024-04-25T01:01:49.605 INFO:teuthology.orchestra.run.smithi031.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 531 kB/s | 31 kB 00:00 2024-04-25T01:01:49.625 INFO:teuthology.orchestra.run.smithi040.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 591 kB/s | 49 kB 00:00 2024-04-25T01:01:49.638 INFO:teuthology.orchestra.run.smithi031.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-04-25T01:01:49.664 INFO:teuthology.orchestra.run.smithi031.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-25T01:01:49.698 INFO:teuthology.orchestra.run.smithi031.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 616 kB/s | 36 kB 00:00 2024-04-25T01:01:49.700 INFO:teuthology.orchestra.run.smithi040.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 205 kB/s | 15 kB 00:00 2024-04-25T01:01:49.723 INFO:teuthology.orchestra.run.smithi031.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 880 kB/s | 51 kB 00:00 2024-04-25T01:01:49.765 INFO:teuthology.orchestra.run.smithi031.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-04-25T01:01:49.783 INFO:teuthology.orchestra.run.smithi040.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 3.1 MB/s | 264 kB 00:00 2024-04-25T01:01:49.807 INFO:teuthology.orchestra.run.smithi031.stdout:(143/152): python3-kubernetes-26.1.0-2.el9.noar 2.0 MB/s | 1.0 MB 00:00 2024-04-25T01:01:49.833 INFO:teuthology.orchestra.run.smithi031.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 830 kB/s | 90 kB 00:00 2024-04-25T01:01:49.874 INFO:teuthology.orchestra.run.smithi031.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.9 MB/s | 427 kB 00:00 2024-04-25T01:01:49.924 INFO:teuthology.orchestra.run.smithi031.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 172 kB/s | 20 kB 00:00 2024-04-25T01:01:49.958 INFO:teuthology.orchestra.run.smithi031.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-04-25T01:01:49.967 INFO:teuthology.orchestra.run.smithi040.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 4.1 MB/s | 762 kB 00:00 2024-04-25T01:01:50.021 INFO:teuthology.orchestra.run.smithi169.stdout:(8/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 2.8 MB/s | 21 MB 00:07 2024-04-25T01:01:50.050 INFO:teuthology.orchestra.run.smithi031.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 9.1 MB/s | 1.6 MB 00:00 2024-04-25T01:01:50.051 INFO:teuthology.orchestra.run.smithi040.stdout:(37/152): ceph-mgr-diskprediction-local-19.0.0- 5.6 MB/s | 7.4 MB 00:01 2024-04-25T01:01:50.067 INFO:teuthology.orchestra.run.smithi040.stdout:(38/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 182 kB/s | 18 kB 00:00 2024-04-25T01:01:50.080 INFO:teuthology.orchestra.run.smithi169.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 434 kB/s | 25 kB 00:00 2024-04-25T01:01:50.168 INFO:teuthology.orchestra.run.smithi040.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 688 kB/s | 80 kB 00:00 2024-04-25T01:01:50.225 INFO:teuthology.orchestra.run.smithi031.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 7.7 MB/s | 1.3 MB 00:00 2024-04-25T01:01:50.235 INFO:teuthology.orchestra.run.smithi040.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 4.7 MB/s | 803 kB 00:00 2024-04-25T01:01:50.260 INFO:teuthology.orchestra.run.smithi040.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-04-25T01:01:50.286 INFO:teuthology.orchestra.run.smithi040.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-04-25T01:01:50.328 INFO:teuthology.orchestra.run.smithi040.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-04-25T01:01:50.333 INFO:teuthology.orchestra.run.smithi031.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-04-25T01:01:50.361 INFO:teuthology.orchestra.run.smithi040.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-04-25T01:01:50.387 INFO:teuthology.orchestra.run.smithi040.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-25T01:01:50.412 INFO:teuthology.orchestra.run.smithi040.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-25T01:01:50.437 INFO:teuthology.orchestra.run.smithi040.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T01:01:50.559 INFO:teuthology.orchestra.run.smithi031.stdout:(151/152): librados2-19.0.0-2445.gd2c45d97.el9. 5.3 MB/s | 3.4 MB 00:00 2024-04-25T01:01:50.696 INFO:teuthology.orchestra.run.smithi040.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-04-25T01:01:50.738 INFO:teuthology.orchestra.run.smithi040.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 943 kB 00:00 2024-04-25T01:01:50.763 INFO:teuthology.orchestra.run.smithi040.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-25T01:01:50.797 INFO:teuthology.orchestra.run.smithi040.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 367 kB/s | 21 kB 00:00 2024-04-25T01:01:50.822 INFO:teuthology.orchestra.run.smithi040.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-04-25T01:01:50.864 INFO:teuthology.orchestra.run.smithi040.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-04-25T01:01:50.923 INFO:teuthology.orchestra.run.smithi040.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-04-25T01:01:50.981 INFO:teuthology.orchestra.run.smithi040.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-04-25T01:01:51.040 INFO:teuthology.orchestra.run.smithi040.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-25T01:01:51.097 INFO:teuthology.orchestra.run.smithi169.stdout:(10/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 4.0 MB/s | 17 MB 00:04 2024-04-25T01:01:51.123 INFO:teuthology.orchestra.run.smithi040.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 545 kB/s | 45 kB 00:00 2024-04-25T01:01:51.166 INFO:teuthology.orchestra.run.smithi169.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 3.2 MB/s | 222 kB 00:00 2024-04-25T01:01:51.190 INFO:teuthology.orchestra.run.smithi040.stdout:(58/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.1 MB/s | 3.0 MB 00:00 2024-04-25T01:01:51.224 INFO:teuthology.orchestra.run.smithi169.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 465 kB/s | 27 kB 00:00 2024-04-25T01:01:51.266 INFO:teuthology.orchestra.run.smithi040.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-04-25T01:01:51.283 INFO:teuthology.orchestra.run.smithi169.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 541 kB/s | 32 kB 00:00 2024-04-25T01:01:51.293 INFO:teuthology.orchestra.run.smithi031.stdout:(152/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 2.3 MB/s | 3.1 MB 00:01 2024-04-25T01:01:51.298 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:01:51.299 INFO:teuthology.orchestra.run.smithi031.stdout:Total 11 MB/s | 220 MB 00:19 2024-04-25T01:01:51.324 INFO:teuthology.orchestra.run.smithi040.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 4.3 MB/s | 582 kB 00:00 2024-04-25T01:01:51.349 INFO:teuthology.orchestra.run.smithi040.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 305 kB/s | 25 kB 00:00 2024-04-25T01:01:51.383 INFO:teuthology.orchestra.run.smithi040.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-04-25T01:01:51.408 INFO:teuthology.orchestra.run.smithi040.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-25T01:01:51.442 INFO:teuthology.orchestra.run.smithi040.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-04-25T01:01:51.467 INFO:teuthology.orchestra.run.smithi040.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 603 kB/s | 35 kB 00:00 2024-04-25T01:01:51.475 INFO:teuthology.orchestra.run.smithi169.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 3.8 MB/s | 745 kB 00:00 2024-04-25T01:01:51.542 INFO:teuthology.orchestra.run.smithi169.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 2.4 MB/s | 163 kB 00:00 2024-04-25T01:01:51.634 INFO:teuthology.orchestra.run.smithi169.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.4 MB/s | 127 kB 00:00 2024-04-25T01:01:51.768 INFO:teuthology.orchestra.run.smithi040.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-04-25T01:01:51.776 INFO:teuthology.orchestra.run.smithi169.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 3.5 MB/s | 501 kB 00:00 2024-04-25T01:01:51.835 INFO:teuthology.orchestra.run.smithi040.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-25T01:01:51.910 INFO:teuthology.orchestra.run.smithi040.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T01:01:51.944 INFO:teuthology.orchestra.run.smithi040.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 8.2 MB/s | 911 kB 00:00 2024-04-25T01:01:51.969 INFO:teuthology.orchestra.run.smithi040.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-25T01:01:52.071 INFO:teuthology.orchestra.run.smithi040.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 9.9 MB/s | 1.0 MB 00:00 2024-04-25T01:01:52.138 INFO:teuthology.orchestra.run.smithi040.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-25T01:01:52.318 INFO:teuthology.orchestra.run.smithi169.stdout:(18/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 2.1 MB/s | 10 MB 00:04 2024-04-25T01:01:52.368 INFO:teuthology.orchestra.run.smithi169.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 910 kB/s | 45 kB 00:00 2024-04-25T01:01:52.427 INFO:teuthology.orchestra.run.smithi169.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 2.2 MB/s | 133 kB 00:00 2024-04-25T01:01:52.502 INFO:teuthology.orchestra.run.smithi169.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 2.1 MB/s | 162 kB 00:00 2024-04-25T01:01:52.639 INFO:teuthology.orchestra.run.smithi040.stdout:(73/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 7.1 MB/s | 48 MB 00:06 2024-04-25T01:01:52.660 INFO:teuthology.orchestra.run.smithi169.stdout:(22/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 6.0 MB/s | 5.3 MB 00:00 2024-04-25T01:01:52.686 INFO:teuthology.orchestra.run.smithi169.stdout:(23/152): python3-rados-19.0.0-2445.gd2c45d97.e 1.7 MB/s | 321 kB 00:00 2024-04-25T01:01:52.722 INFO:teuthology.orchestra.run.smithi040.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 7.4 MB/s | 5.8 MB 00:00 2024-04-25T01:01:52.736 INFO:teuthology.orchestra.run.smithi169.stdout:(24/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 3.9 MB/s | 299 kB 00:00 2024-04-25T01:01:52.761 INFO:teuthology.orchestra.run.smithi169.stdout:(25/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.3 MB/s | 99 kB 00:00 2024-04-25T01:01:52.787 INFO:teuthology.orchestra.run.smithi169.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.7 MB/s | 85 kB 00:00 2024-04-25T01:01:52.822 INFO:teuthology.orchestra.run.smithi040.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 8.7 MB/s | 6.0 MB 00:00 2024-04-25T01:01:52.848 INFO:teuthology.orchestra.run.smithi040.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.9 MB/s | 248 kB 00:00 2024-04-25T01:01:52.881 INFO:teuthology.orchestra.run.smithi040.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 440 kB/s | 26 kB 00:00 2024-04-25T01:01:52.906 INFO:teuthology.orchestra.run.smithi040.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 327 kB/s | 19 kB 00:00 2024-04-25T01:01:52.912 INFO:teuthology.orchestra.run.smithi169.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 1.3 MB/s | 171 kB 00:00 2024-04-25T01:01:52.940 INFO:teuthology.orchestra.run.smithi040.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-25T01:01:52.962 INFO:teuthology.orchestra.run.smithi169.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 638 kB/s | 32 kB 00:00 2024-04-25T01:01:52.965 INFO:teuthology.orchestra.run.smithi040.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 594 kB/s | 35 kB 00:00 2024-04-25T01:01:52.990 INFO:teuthology.orchestra.run.smithi040.stdout:(81/152): python3-devel-3.9.18-3.el9.x86_64.rpm 694 kB/s | 244 kB 00:00 2024-04-25T01:01:53.021 INFO:teuthology.orchestra.run.smithi169.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 2.4 MB/s | 141 kB 00:00 2024-04-25T01:01:53.024 INFO:teuthology.orchestra.run.smithi040.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-04-25T01:01:53.091 INFO:teuthology.orchestra.run.smithi040.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-04-25T01:01:53.116 INFO:teuthology.orchestra.run.smithi040.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.7 MB/s | 222 kB 00:00 2024-04-25T01:01:53.150 INFO:teuthology.orchestra.run.smithi040.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 722 kB/s | 42 kB 00:00 2024-04-25T01:01:53.225 INFO:teuthology.orchestra.run.smithi040.stdout:(86/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 5.1 MB 00:00 2024-04-25T01:01:53.250 INFO:teuthology.orchestra.run.smithi040.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 2.0 MB/s | 269 kB 00:00 2024-04-25T01:01:53.276 INFO:teuthology.orchestra.run.smithi040.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-04-25T01:01:53.301 INFO:teuthology.orchestra.run.smithi040.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-04-25T01:01:53.330 INFO:teuthology.orchestra.run.smithi040.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-04-25T01:01:53.352 INFO:teuthology.orchestra.run.smithi040.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 710 kB/s | 54 kB 00:00 2024-04-25T01:01:53.363 INFO:teuthology.orchestra.run.smithi169.stdout:(30/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 5.1 MB/s | 3.1 MB 00:00 2024-04-25T01:01:53.385 INFO:teuthology.orchestra.run.smithi040.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 713 kB/s | 42 kB 00:00 2024-04-25T01:01:53.411 INFO:teuthology.orchestra.run.smithi040.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 755 kB/s | 44 kB 00:00 2024-04-25T01:01:53.430 INFO:teuthology.orchestra.run.smithi169.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 8.6 MB/s | 3.5 MB 00:00 2024-04-25T01:01:53.461 INFO:teuthology.orchestra.run.smithi040.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 720 kB/s | 36 kB 00:00 2024-04-25T01:01:53.480 INFO:teuthology.orchestra.run.smithi169.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 4.9 MB/s | 249 kB 00:00 2024-04-25T01:01:53.486 INFO:teuthology.orchestra.run.smithi040.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-04-25T01:01:53.520 INFO:teuthology.orchestra.run.smithi040.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 808 kB/s | 47 kB 00:00 2024-04-25T01:01:53.530 INFO:teuthology.orchestra.run.smithi169.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 993 kB/s | 49 kB 00:00 2024-04-25T01:01:53.581 INFO:teuthology.orchestra.run.smithi169.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 307 kB/s | 15 kB 00:00 2024-04-25T01:01:53.586 INFO:teuthology.orchestra.run.smithi040.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-04-25T01:01:53.612 INFO:teuthology.orchestra.run.smithi040.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 629 kB/s | 58 kB 00:00 2024-04-25T01:01:53.639 INFO:teuthology.orchestra.run.smithi169.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 4.4 MB/s | 264 kB 00:00 2024-04-25T01:01:53.654 INFO:teuthology.orchestra.run.smithi040.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-04-25T01:01:53.671 INFO:teuthology.orchestra.run.smithi040.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-25T01:01:53.704 INFO:teuthology.orchestra.run.smithi040.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 888 kB/s | 45 kB 00:00 2024-04-25T01:01:53.722 INFO:teuthology.orchestra.run.smithi040.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 443 kB/s | 22 kB 00:00 2024-04-25T01:01:53.748 INFO:teuthology.orchestra.run.smithi169.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 6.9 MB/s | 762 kB 00:00 2024-04-25T01:01:53.798 INFO:teuthology.orchestra.run.smithi169.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 366 kB/s | 18 kB 00:00 2024-04-25T01:01:54.014 INFO:teuthology.orchestra.run.smithi040.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-04-25T01:01:54.047 INFO:teuthology.orchestra.run.smithi040.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-04-25T01:01:54.164 INFO:teuthology.orchestra.run.smithi040.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 167 kB/s | 19 kB 00:00 2024-04-25T01:01:54.166 INFO:teuthology.orchestra.run.smithi169.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 219 kB/s | 80 kB 00:00 2024-04-25T01:01:54.249 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-25T01:01:54.256 INFO:teuthology.orchestra.run.smithi040.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-04-25T01:01:54.298 INFO:teuthology.orchestra.run.smithi040.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 952 kB/s | 548 kB 00:00 2024-04-25T01:01:54.323 INFO:teuthology.orchestra.run.smithi040.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 388 kB/s | 26 kB 00:00 2024-04-25T01:01:54.449 INFO:teuthology.orchestra.run.smithi040.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 538 kB/s | 67 kB 00:00 2024-04-25T01:01:54.491 INFO:teuthology.orchestra.run.smithi040.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 252 kB/s | 49 kB 00:00 2024-04-25T01:01:54.507 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-25T01:01:54.507 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-25T01:01:54.516 INFO:teuthology.orchestra.run.smithi040.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-04-25T01:01:54.516 INFO:teuthology.orchestra.run.smithi169.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 6.4 MB/s | 7.4 MB 00:01 2024-04-25T01:01:54.699 INFO:teuthology.orchestra.run.smithi040.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-04-25T01:01:54.708 INFO:teuthology.orchestra.run.smithi169.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 803 kB 00:00 2024-04-25T01:01:54.766 INFO:teuthology.orchestra.run.smithi040.stdout:(113/152): libarrow-9.0.0-10.el9.x86_64.rpm 7.3 MB/s | 4.4 MB 00:00 2024-04-25T01:01:54.792 INFO:teuthology.orchestra.run.smithi040.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 473 kB/s | 43 kB 00:00 2024-04-25T01:01:54.817 INFO:teuthology.orchestra.run.smithi040.stdout:(115/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.5 MB/s | 838 kB 00:00 2024-04-25T01:01:54.842 INFO:teuthology.orchestra.run.smithi040.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 297 kB/s | 15 kB 00:00 2024-04-25T01:01:54.849 INFO:teuthology.orchestra.run.smithi169.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-04-25T01:01:54.867 INFO:teuthology.orchestra.run.smithi040.stdout:(117/152): python3-cachetools-4.2.4-1.el9.noarc 319 kB/s | 32 kB 00:00 2024-04-25T01:01:54.984 INFO:teuthology.orchestra.run.smithi040.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.0 MB/s | 172 kB 00:00 2024-04-25T01:01:55.010 INFO:teuthology.orchestra.run.smithi040.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.1 MB/s | 303 kB 00:00 2024-04-25T01:01:55.035 INFO:teuthology.orchestra.run.smithi040.stdout:(120/152): python3-cherrypy-18.6.1-2.el9.noarch 1.8 MB/s | 358 kB 00:00 2024-04-25T01:01:55.060 INFO:teuthology.orchestra.run.smithi040.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 469 kB/s | 35 kB 00:00 2024-04-25T01:01:55.127 INFO:teuthology.orchestra.run.smithi040.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-04-25T01:01:55.177 INFO:teuthology.orchestra.run.smithi040.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-04-25T01:01:55.191 INFO:teuthology.orchestra.run.smithi169.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 289 kB/s | 195 kB 00:00 2024-04-25T01:01:55.203 INFO:teuthology.orchestra.run.smithi040.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-04-25T01:01:55.233 INFO:teuthology.orchestra.run.smithi169.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 3.2 MB/s | 1.2 MB 00:00 2024-04-25T01:01:55.236 INFO:teuthology.orchestra.run.smithi040.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 301 kB/s | 18 kB 00:00 2024-04-25T01:01:55.262 INFO:teuthology.orchestra.run.smithi040.stdout:(126/152): python3-jaraco-8.2.1-3.el9.noarch.rp 80 kB/s | 11 kB 00:00 2024-04-25T01:01:55.287 INFO:teuthology.orchestra.run.smithi040.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 276 kB/s | 23 kB 00:00 2024-04-25T01:01:55.312 INFO:teuthology.orchestra.run.smithi040.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-04-25T01:01:55.317 INFO:teuthology.orchestra.run.smithi169.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 850 kB/s | 106 kB 00:00 2024-04-25T01:01:55.338 INFO:teuthology.orchestra.run.smithi040.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-04-25T01:01:55.367 INFO:teuthology.orchestra.run.smithi169.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.0 MB/s | 135 kB 00:00 2024-04-25T01:01:55.379 INFO:teuthology.orchestra.run.smithi040.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 691 kB/s | 46 kB 00:00 2024-04-25T01:01:55.405 INFO:teuthology.orchestra.run.smithi040.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.1 MB/s | 79 kB 00:00 2024-04-25T01:01:55.438 INFO:teuthology.orchestra.run.smithi040.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 6.7 MB/s | 1.0 MB 00:00 2024-04-25T01:01:55.442 INFO:teuthology.orchestra.run.smithi169.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 996 kB/s | 125 kB 00:00 2024-04-25T01:01:55.464 INFO:teuthology.orchestra.run.smithi040.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 688 kB/s | 58 kB 00:00 2024-04-25T01:01:55.489 INFO:teuthology.orchestra.run.smithi040.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-25T01:01:55.501 INFO:teuthology.orchestra.run.smithi169.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.6 MB/s | 215 kB 00:00 2024-04-25T01:01:55.514 INFO:teuthology.orchestra.run.smithi040.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-04-25T01:01:55.539 INFO:teuthology.orchestra.run.smithi040.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-25T01:01:55.565 INFO:teuthology.orchestra.run.smithi040.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-04-25T01:01:55.590 INFO:teuthology.orchestra.run.smithi040.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-04-25T01:01:55.615 INFO:teuthology.orchestra.run.smithi040.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-25T01:01:55.641 INFO:teuthology.orchestra.run.smithi040.stdout:(140/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-25T01:01:55.652 INFO:teuthology.orchestra.run.smithi169.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 6.1 MB/s | 943 kB 00:00 2024-04-25T01:01:55.666 INFO:teuthology.orchestra.run.smithi040.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 473 kB/s | 36 kB 00:00 2024-04-25T01:01:55.691 INFO:teuthology.orchestra.run.smithi040.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-04-25T01:01:55.716 INFO:teuthology.orchestra.run.smithi040.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-04-25T01:01:55.750 INFO:teuthology.orchestra.run.smithi040.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 7.1 MB/s | 427 kB 00:00 2024-04-25T01:01:55.775 INFO:teuthology.orchestra.run.smithi040.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-04-25T01:01:55.777 INFO:teuthology.orchestra.run.smithi169.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 857 kB/s | 107 kB 00:00 2024-04-25T01:01:55.801 INFO:teuthology.orchestra.run.smithi040.stdout:(146/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-04-25T01:01:55.826 INFO:teuthology.orchestra.run.smithi040.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-25T01:01:55.885 INFO:teuthology.orchestra.run.smithi169.stdout:(50/152): automake-1.16.2-8.el9.noarch.rpm 1.5 MB/s | 689 kB 00:00 2024-04-25T01:01:55.910 INFO:teuthology.orchestra.run.smithi169.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 161 kB/s | 21 kB 00:00 2024-04-25T01:01:55.926 INFO:teuthology.orchestra.run.smithi040.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T01:01:56.011 INFO:teuthology.orchestra.run.smithi169.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-04-25T01:01:56.036 INFO:teuthology.orchestra.run.smithi169.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 261 kB/s | 33 kB 00:00 2024-04-25T01:01:56.076 INFO:teuthology.orchestra.run.smithi040.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 1.3 MB 00:00 2024-04-25T01:01:56.145 INFO:teuthology.orchestra.run.smithi169.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-04-25T01:01:56.160 INFO:teuthology.orchestra.run.smithi040.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 29 MB/s | 2.4 MB 00:00 2024-04-25T01:01:56.236 INFO:teuthology.orchestra.run.smithi040.stdout:(151/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 7.7 MB/s | 3.1 MB 00:00 2024-04-25T01:01:56.253 INFO:teuthology.orchestra.run.smithi169.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-04-25T01:01:56.335 INFO:teuthology.orchestra.run.smithi040.stdout:(152/152): librados2-19.0.0-2445.gd2c45d97.el9. 6.3 MB/s | 3.4 MB 00:00 2024-04-25T01:01:56.338 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:01:56.338 INFO:teuthology.orchestra.run.smithi040.stdout:Total 9.3 MB/s | 220 MB 00:23 2024-04-25T01:01:56.362 INFO:teuthology.orchestra.run.smithi169.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-04-25T01:01:56.412 INFO:teuthology.orchestra.run.smithi169.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.5 MB/s | 3.0 MB 00:00 2024-04-25T01:01:56.471 INFO:teuthology.orchestra.run.smithi169.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-04-25T01:01:56.538 INFO:teuthology.orchestra.run.smithi169.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-04-25T01:01:56.613 INFO:teuthology.orchestra.run.smithi169.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 4.0 MB/s | 582 kB 00:00 2024-04-25T01:01:56.672 INFO:teuthology.orchestra.run.smithi169.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 190 kB/s | 25 kB 00:00 2024-04-25T01:01:56.747 INFO:teuthology.orchestra.run.smithi169.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 355 kB/s | 47 kB 00:00 2024-04-25T01:01:56.814 INFO:teuthology.orchestra.run.smithi169.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-04-25T01:01:56.889 INFO:teuthology.orchestra.run.smithi169.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 1.3 MB/s | 188 kB 00:00 2024-04-25T01:01:56.956 INFO:teuthology.orchestra.run.smithi169.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 248 kB/s | 35 kB 00:00 2024-04-25T01:01:57.128 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-25T01:01:57.128 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-25T01:01:57.494 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-04-25T01:01:57.655 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-04-25T01:01:57.655 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-04-25T01:01:58.640 INFO:teuthology.orchestra.run.smithi169.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 2.6 MB/s | 4.4 MB 00:01 2024-04-25T01:01:58.782 INFO:teuthology.orchestra.run.smithi169.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T01:01:58.899 INFO:teuthology.orchestra.run.smithi169.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 7.6 MB/s | 911 kB 00:00 2024-04-25T01:01:59.008 INFO:teuthology.orchestra.run.smithi169.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-04-25T01:01:59.547 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-04-25T01:01:59.592 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-04-25T01:01:59.608 INFO:teuthology.orchestra.run.smithi169.stdout:(70/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 5.1 MB/s | 48 MB 00:09 2024-04-25T01:01:59.725 INFO:teuthology.orchestra.run.smithi169.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 8.1 MB/s | 5.8 MB 00:00 2024-04-25T01:01:59.834 INFO:teuthology.orchestra.run.smithi169.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 807 kB/s | 87 kB 00:00 2024-04-25T01:02:00.359 INFO:teuthology.orchestra.run.smithi169.stdout:(73/152): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-04-25T01:02:00.484 INFO:teuthology.orchestra.run.smithi169.stdout:(74/152): protobuf-3.14.0-13.el9.x86_64.rpm 1.2 MB/s | 1.0 MB 00:00 2024-04-25T01:02:00.509 INFO:teuthology.orchestra.run.smithi169.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.6 MB/s | 244 kB 00:00 2024-04-25T01:02:00.601 INFO:teuthology.orchestra.run.smithi169.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-04-25T01:02:00.626 INFO:teuthology.orchestra.run.smithi169.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 219 kB/s | 26 kB 00:00 2024-04-25T01:02:00.710 INFO:teuthology.orchestra.run.smithi169.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-04-25T01:02:00.744 INFO:teuthology.orchestra.run.smithi169.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-04-25T01:02:00.827 INFO:teuthology.orchestra.run.smithi169.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 297 kB/s | 35 kB 00:00 2024-04-25T01:02:00.944 INFO:teuthology.orchestra.run.smithi169.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-04-25T01:02:01.069 INFO:teuthology.orchestra.run.smithi169.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.7 MB/s | 222 kB 00:00 2024-04-25T01:02:01.136 INFO:teuthology.orchestra.run.smithi169.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-04-25T01:02:01.228 INFO:teuthology.orchestra.run.smithi169.stdout:(84/152): openblas-openmp-0.3.21-2.el9.x86_64.r 1.1 MB/s | 4.7 MB 00:04 2024-04-25T01:02:01.245 INFO:teuthology.orchestra.run.smithi169.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 388 kB/s | 42 kB 00:00 2024-04-25T01:02:01.270 INFO:teuthology.orchestra.run.smithi169.stdout:(86/152): python3-packaging-20.9-5.el9.noarch.r 384 kB/s | 77 kB 00:00 2024-04-25T01:02:01.346 INFO:teuthology.orchestra.run.smithi169.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-04-25T01:02:01.371 INFO:teuthology.orchestra.run.smithi169.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-04-25T01:02:01.404 INFO:teuthology.orchestra.run.smithi169.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.0 MB/s | 279 kB 00:00 2024-04-25T01:02:01.446 INFO:teuthology.orchestra.run.smithi169.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 508 kB/s | 51 kB 00:00 2024-04-25T01:02:01.472 INFO:teuthology.orchestra.run.smithi169.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 533 kB/s | 54 kB 00:00 2024-04-25T01:02:01.547 INFO:teuthology.orchestra.run.smithi169.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-04-25T01:02:01.572 INFO:teuthology.orchestra.run.smithi169.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 440 kB/s | 44 kB 00:00 2024-04-25T01:02:01.665 INFO:teuthology.orchestra.run.smithi169.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-04-25T01:02:01.690 INFO:teuthology.orchestra.run.smithi169.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 306 kB/s | 36 kB 00:00 2024-04-25T01:02:01.782 INFO:teuthology.orchestra.run.smithi169.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 405 kB/s | 47 kB 00:00 2024-04-25T01:02:01.815 INFO:teuthology.orchestra.run.smithi169.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.4 MB/s | 305 kB 00:00 2024-04-25T01:02:01.899 INFO:teuthology.orchestra.run.smithi169.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 495 kB/s | 58 kB 00:00 2024-04-25T01:02:01.933 INFO:teuthology.orchestra.run.smithi169.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 800 kB/s | 94 kB 00:00 2024-04-25T01:02:02.016 INFO:teuthology.orchestra.run.smithi169.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 544 kB/s | 64 kB 00:00 2024-04-25T01:02:02.050 INFO:teuthology.orchestra.run.smithi169.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 383 kB/s | 45 kB 00:00 2024-04-25T01:02:02.142 INFO:teuthology.orchestra.run.smithi169.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-04-25T01:02:02.367 INFO:teuthology.orchestra.run.smithi169.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.7 MB/s | 863 kB 00:00 2024-04-25T01:02:02.401 INFO:teuthology.orchestra.run.smithi169.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 2.1 MB/s | 548 kB 00:00 2024-04-25T01:02:02.459 INFO:teuthology.orchestra.run.smithi169.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 334 kB/s | 19 kB 00:00 2024-04-25T01:02:02.610 INFO:teuthology.orchestra.run.smithi169.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-04-25T01:02:02.676 INFO:teuthology.orchestra.run.smithi169.stdout:(107/152): libarrow-9.0.0-10.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-04-25T01:02:02.702 INFO:teuthology.orchestra.run.smithi169.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-04-25T01:02:02.735 INFO:teuthology.orchestra.run.smithi169.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 827 kB/s | 49 kB 00:00 2024-04-25T01:02:02.761 INFO:teuthology.orchestra.run.smithi169.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-25T01:02:02.794 INFO:teuthology.orchestra.run.smithi169.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-25T01:02:02.861 INFO:teuthology.orchestra.run.smithi169.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-04-25T01:02:02.886 INFO:teuthology.orchestra.run.smithi169.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.5 MB/s | 838 kB 00:00 2024-04-25T01:02:02.912 INFO:teuthology.orchestra.run.smithi169.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 865 kB/s | 43 kB 00:00 2024-04-25T01:02:02.937 INFO:teuthology.orchestra.run.smithi169.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 639 kB/s | 32 kB 00:00 2024-04-25T01:02:02.962 INFO:teuthology.orchestra.run.smithi169.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-04-25T01:02:02.996 INFO:teuthology.orchestra.run.smithi169.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-25T01:02:03.021 INFO:teuthology.orchestra.run.smithi169.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-04-25T01:02:03.055 INFO:teuthology.orchestra.run.smithi169.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-04-25T01:02:03.272 INFO:teuthology.orchestra.run.smithi169.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 141 kB/s | 35 kB 00:00 2024-04-25T01:02:03.314 INFO:teuthology.orchestra.run.smithi169.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 795 kB/s | 206 kB 00:00 2024-04-25T01:02:03.356 INFO:teuthology.orchestra.run.smithi169.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 24 MB/s | 2.0 MB 00:00 2024-04-25T01:02:03.381 INFO:teuthology.orchestra.run.smithi169.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-04-25T01:02:03.423 INFO:teuthology.orchestra.run.smithi169.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 159 kB/s | 11 kB 00:00 2024-04-25T01:02:03.457 INFO:teuthology.orchestra.run.smithi169.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 234 kB/s | 18 kB 00:00 2024-04-25T01:02:03.474 INFO:teuthology.orchestra.run.smithi169.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 460 kB/s | 23 kB 00:00 2024-04-25T01:02:03.507 INFO:teuthology.orchestra.run.smithi169.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-04-25T01:02:03.524 INFO:teuthology.orchestra.run.smithi169.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-04-25T01:02:03.584 INFO:teuthology.orchestra.run.smithi169.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 778 kB/s | 46 kB 00:00 2024-04-25T01:02:03.617 INFO:teuthology.orchestra.run.smithi169.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-04-25T01:02:03.650 INFO:teuthology.orchestra.run.smithi169.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-04-25T01:02:03.667 INFO:teuthology.orchestra.run.smithi169.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-25T01:02:03.709 INFO:teuthology.orchestra.run.smithi169.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-04-25T01:02:03.726 INFO:teuthology.orchestra.run.smithi169.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-04-25T01:02:03.768 INFO:teuthology.orchestra.run.smithi169.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-25T01:02:03.785 INFO:teuthology.orchestra.run.smithi169.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 481 kB/s | 28 kB 00:00 2024-04-25T01:02:03.827 INFO:teuthology.orchestra.run.smithi169.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 525 kB/s | 31 kB 00:00 2024-04-25T01:02:03.835 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-25T01:02:03.869 INFO:teuthology.orchestra.run.smithi169.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-25T01:02:03.894 INFO:teuthology.orchestra.run.smithi169.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 886 kB/s | 59 kB 00:00 2024-04-25T01:02:03.919 INFO:teuthology.orchestra.run.smithi169.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-04-25T01:02:03.945 INFO:teuthology.orchestra.run.smithi169.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-25T01:02:03.949 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T01:02:03.978 INFO:teuthology.orchestra.run.smithi169.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-04-25T01:02:04.003 INFO:teuthology.orchestra.run.smithi169.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-25T01:02:04.046 INFO:teuthology.orchestra.run.smithi169.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 6.3 MB/s | 427 kB 00:00 2024-04-25T01:02:04.086 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T01:02:04.141 INFO:teuthology.orchestra.run.smithi169.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 239 kB/s | 20 kB 00:00 2024-04-25T01:02:04.142 INFO:teuthology.orchestra.run.smithi169.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-25T01:02:04.179 INFO:teuthology.orchestra.run.smithi169.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-25T01:02:04.230 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T01:02:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T01:02:04.323 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T01:02:04.437 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T01:02:04.546 INFO:teuthology.orchestra.run.smithi169.stdout:(148/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 8.6 MB/s | 3.1 MB 00:00 2024-04-25T01:02:04.663 INFO:teuthology.orchestra.run.smithi169.stdout:(149/152): librados2-19.0.0-2445.gd2c45d97.el9. 6.2 MB/s | 3.4 MB 00:00 2024-04-25T01:02:04.899 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T01:02:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:02:05.228 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:02:05.257 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:02:05.322 INFO:teuthology.orchestra.run.smithi169.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 1.7 MB/s | 1.3 MB 00:00 2024-04-25T01:02:05.395 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:02:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T01:02:05.539 INFO:teuthology.orchestra.run.smithi169.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.7 MB/s | 2.4 MB 00:00 2024-04-25T01:02:05.754 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-04-25T01:02:05.821 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T01:02:05.886 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T01:02:05.889 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T01:02:06.022 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T01:02:06.084 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T01:02:06.103 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:02:06.124 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T01:02:06.151 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T01:02:06.194 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T01:02:06.276 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:02:06.303 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T01:02:06.398 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T01:02:06.429 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T01:02:06.512 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T01:02:06.541 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T01:02:06.650 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:02:06.774 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:02:06.801 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:02:06.831 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T01:02:06.889 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T01:02:06.894 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:02:06.922 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T01:02:07.079 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T01:02:07.114 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T01:02:07.128 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T01:02:07.155 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:02:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T01:02:07.260 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:02:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T01:02:07.290 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T01:02:07.337 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T01:02:07.353 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T01:02:07.374 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T01:02:07.399 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T01:02:07.462 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T01:02:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T01:02:07.574 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T01:02:07.592 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T01:02:07.621 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T01:02:07.633 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T01:02:07.661 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T01:02:07.744 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T01:02:07.781 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T01:02:07.861 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T01:02:07.896 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T01:02:07.956 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T01:02:07.973 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T01:02:08.006 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T01:02:08.042 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T01:02:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T01:02:08.108 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-25T01:02:08.117 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T01:02:08.157 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T01:02:08.167 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-25T01:02:08.190 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-25T01:02:08.201 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T01:02:08.233 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T01:02:08.261 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T01:02:08.307 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T01:02:08.333 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T01:02:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T01:02:08.384 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T01:02:08.414 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T01:02:08.450 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T01:02:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T01:02:08.528 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T01:02:08.568 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-25T01:02:08.612 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-25T01:02:08.640 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-25T01:02:08.672 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T01:02:08.714 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T01:02:08.737 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T01:02:08.764 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T01:02:08.842 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T01:02:08.986 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T01:02:09.042 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T01:02:09.439 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T01:02:09.545 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T01:02:09.587 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T01:02:09.604 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T01:02:09.632 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T01:02:09.690 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T01:02:09.700 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-25T01:02:09.724 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T01:02:09.767 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T01:02:09.768 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T01:02:09.805 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-25T01:02:09.847 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T01:02:09.998 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T01:02:10.020 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T01:02:10.050 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T01:02:10.106 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T01:02:10.106 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T01:02:10.179 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T01:02:10.193 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T01:02:10.217 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T01:02:10.253 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T01:02:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T01:02:10.301 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T01:02:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T01:02:10.336 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T01:02:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T01:02:10.380 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T01:02:10.422 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T01:02:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T01:02:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T01:02:10.564 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T01:02:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T01:02:10.610 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T01:02:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T01:02:10.640 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T01:02:10.682 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T01:02:10.779 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T01:02:10.815 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T01:02:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T01:02:10.855 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T01:02:10.880 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T01:02:10.905 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T01:02:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T01:02:10.939 INFO:teuthology.orchestra.run.smithi169.stdout:(152/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.7 MB/s | 16 MB 00:09 2024-04-25T01:02:10.942 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:02:10.942 INFO:teuthology.orchestra.run.smithi169.stdout:Total 5.9 MB/s | 220 MB 00:37 2024-04-25T01:02:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T01:02:10.983 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T01:02:11.006 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T01:02:11.121 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T01:02:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T01:02:11.163 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T01:02:11.191 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T01:02:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T01:02:11.225 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T01:02:11.249 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T01:02:11.264 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T01:02:11.286 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T01:02:11.305 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T01:02:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T01:02:11.334 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T01:02:11.369 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T01:02:11.411 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T01:02:11.453 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T01:02:11.486 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T01:02:11.506 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T01:02:11.577 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T01:02:11.626 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T01:02:11.665 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T01:02:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T01:02:11.713 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T01:02:11.749 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T01:02:11.790 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T01:02:11.822 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T01:02:11.846 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T01:02:11.876 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T01:02:11.914 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T01:02:11.979 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T01:02:11.993 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-04-25T01:02:12.047 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T01:02:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T01:02:12.139 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-04-25T01:02:12.140 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-04-25T01:02:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T01:02:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T01:02:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T01:02:12.427 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T01:02:12.494 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T01:02:12.538 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T01:02:12.611 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T01:02:12.641 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T01:02:12.689 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T01:02:12.722 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T01:02:12.770 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T01:02:12.824 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T01:02:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T01:02:12.881 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T01:02:12.970 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T01:02:13.038 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T01:02:13.219 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T01:02:13.247 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T01:02:13.329 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T01:02:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T01:02:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T01:02:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T01:02:13.987 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-04-25T01:02:13.987 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-04-25T01:02:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T01:02:14.178 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T01:02:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T01:02:14.298 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T01:02:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T01:02:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T01:02:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T01:02:14.672 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T01:02:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T01:02:15.598 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T01:02:15.692 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T01:02:15.720 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T01:02:15.753 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T01:02:15.768 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T01:02:15.924 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T01:02:16.083 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T01:02:16.116 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T01:02:16.208 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T01:02:16.238 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-25T01:02:16.274 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T01:02:16.773 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T01:02:16.811 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T01:02:16.882 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T01:02:16.921 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-25T01:02:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T01:02:17.736 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T01:02:17.788 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T01:02:17.858 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T01:02:18.277 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T01:02:18.322 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T01:02:18.446 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T01:02:18.479 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-25T01:02:18.538 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T01:02:19.133 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T01:02:19.167 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T01:02:19.267 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T01:02:19.294 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-25T01:02:19.302 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T01:02:19.334 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T01:02:19.904 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T01:02:19.948 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-04-25T01:02:19.954 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T01:02:20.042 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T01:02:20.045 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T01:02:20.045 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T01:02:20.045 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T01:02:20.045 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T01:02:20.072 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T01:02:20.080 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T01:02:20.134 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T01:02:20.168 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T01:02:20.193 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T01:02:20.236 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T01:02:20.559 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T01:02:20.669 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:02:20.791 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:02:20.829 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:02:20.920 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:02:20.950 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T01:02:21.096 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T01:02:21.147 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T01:02:21.174 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:02:21.269 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:02:21.292 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T01:02:21.338 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T01:02:21.360 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T01:02:21.379 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T01:02:21.576 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T01:02:21.609 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T01:02:21.725 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T01:02:21.811 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T01:02:21.850 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T01:02:21.884 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T01:02:21.950 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T01:02:21.983 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T01:02:22.027 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T01:02:22.060 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T01:02:22.088 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T01:02:22.155 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T01:02:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T01:02:22.209 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T01:02:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T01:02:22.240 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T01:02:22.284 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T01:02:22.359 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T01:02:22.395 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-25T01:02:22.439 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-25T01:02:22.459 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-25T01:02:22.528 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T01:02:22.574 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T01:02:22.646 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T01:02:22.778 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T01:02:22.835 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T01:02:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T01:02:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T01:02:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T01:02:23.117 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T01:02:23.117 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T01:02:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:02:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T01:02:23.350 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T01:02:23.430 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T01:02:23.459 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T01:02:23.494 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T01:02:23.532 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-25T01:02:23.602 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T01:02:23.963 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T01:02:24.026 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T01:02:24.080 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T01:02:24.153 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T01:02:24.185 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T01:02:24.222 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T01:02:24.268 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T01:02:24.305 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T01:02:24.348 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T01:02:24.389 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T01:02:24.525 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T01:02:24.570 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T01:02:24.600 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T01:02:24.642 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T01:02:24.735 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T01:02:24.767 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T01:02:24.808 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T01:02:24.833 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T01:02:24.858 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T01:02:24.923 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T01:02:25.049 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T01:02:25.090 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T01:02:25.118 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T01:02:25.152 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T01:02:25.192 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T01:02:25.223 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T01:02:25.253 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T01:02:25.279 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T01:02:25.313 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T01:02:25.354 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T01:02:25.389 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T01:02:25.476 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T01:02:25.519 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T01:02:25.550 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T01:02:25.597 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T01:02:25.704 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T01:02:25.760 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T01:02:26.468 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T01:02:26.516 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T01:02:26.550 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T01:02:26.646 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T01:02:26.699 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T01:02:26.787 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T01:02:26.835 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T01:02:27.012 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T01:02:27.042 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T01:02:27.118 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T01:02:27.750 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T01:02:27.863 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T01:02:29.069 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T01:02:29.146 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T01:02:29.173 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T01:02:29.207 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T01:02:29.519 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T01:02:29.570 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T01:02:29.678 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T01:02:29.724 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-25T01:02:29.761 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T01:02:30.234 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T01:02:30.265 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T01:02:30.335 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T01:02:30.366 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-25T01:02:32.196 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T01:02:32.196 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /sys 2024-04-25T01:02:32.196 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /proc 2024-04-25T01:02:32.196 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /mnt 2024-04-25T01:02:32.196 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /var/tmp 2024-04-25T01:02:32.196 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /home 2024-04-25T01:02:32.196 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /root 2024-04-25T01:02:32.196 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /tmp 2024-04-25T01:02:32.196 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T01:02:32.360 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T01:02:32.376 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T01:02:32.377 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:32.377 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T01:02:32.377 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T01:02:32.691 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T01:02:32.739 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T01:02:33.290 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T01:02:33.325 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T01:02:33.416 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T01:02:33.416 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T01:02:33.417 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T01:02:33.417 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T01:02:33.443 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T01:02:33.913 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T01:02:33.940 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T01:02:33.940 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:33.941 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T01:02:33.941 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T01:02:33.941 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T01:02:33.941 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T01:02:34.188 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T01:02:34.213 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T01:02:34.213 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:34.213 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T01:02:34.213 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T01:02:34.213 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T01:02:34.213 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T01:02:34.681 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T01:02:34.705 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T01:02:34.705 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:34.705 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T01:02:34.706 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T01:02:34.706 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T01:02:34.706 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T01:02:34.812 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T01:02:34.870 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T01:02:34.933 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T01:02:35.147 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T01:02:35.199 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T01:02:35.279 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T01:02:35.369 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T01:02:35.519 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T01:02:35.566 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T01:02:35.597 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T01:02:35.625 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T01:02:36.076 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T01:02:36.107 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T01:02:36.139 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-25T01:02:36.211 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-25T01:02:36.247 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T01:02:36.668 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T01:02:36.669 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /sys 2024-04-25T01:02:36.669 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /proc 2024-04-25T01:02:36.669 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /mnt 2024-04-25T01:02:36.669 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /var/tmp 2024-04-25T01:02:36.669 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /home 2024-04-25T01:02:36.669 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /root 2024-04-25T01:02:36.669 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /tmp 2024-04-25T01:02:36.669 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:02:36.939 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T01:02:36.991 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T01:02:36.991 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:36.991 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T01:02:36.991 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:02:37.342 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T01:02:37.381 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T01:02:38.399 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T01:02:38.432 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T01:02:38.540 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T01:02:38.706 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T01:02:38.719 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-25T01:02:38.751 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T01:02:38.775 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T01:02:38.775 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:38.775 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T01:02:38.775 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T01:02:38.775 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T01:02:38.775 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T01:02:38.791 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T01:02:38.791 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:38.791 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T01:02:38.791 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T01:02:38.792 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T01:02:38.792 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:02:38.826 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T01:02:38.838 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T01:02:38.939 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-25T01:02:38.993 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-25T01:02:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T01:02:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T01:02:39.127 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:39.127 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T01:02:39.128 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T01:02:39.128 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T01:02:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:02:39.649 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T01:02:39.726 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T01:02:39.726 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:39.727 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T01:02:39.727 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T01:02:39.727 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T01:02:39.727 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:02:39.873 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T01:02:39.932 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T01:02:39.950 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T01:02:39.996 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T01:02:40.007 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T01:02:40.022 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T01:02:40.022 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:40.022 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T01:02:40.022 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:02:40.022 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:02:40.022 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T01:02:40.064 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T01:02:40.094 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T01:02:40.095 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:40.095 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T01:02:40.095 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T01:02:40.147 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T01:02:40.163 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T01:02:40.164 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:40.164 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T01:02:40.164 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T01:02:40.164 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T01:02:40.164 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T01:02:40.276 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T01:02:40.350 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T01:02:40.471 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T01:02:40.481 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T01:02:40.505 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T01:02:40.506 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:40.506 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T01:02:40.506 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T01:02:40.506 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T01:02:40.506 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T01:02:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T01:02:40.772 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T01:02:40.833 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T01:02:40.878 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T01:02:40.909 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T01:02:41.521 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T01:02:41.564 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T01:02:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-25T01:02:41.745 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-25T01:02:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T01:02:43.146 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T01:02:43.190 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T01:02:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T01:02:44.407 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T01:02:44.562 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T01:02:44.771 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-25T01:02:44.800 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T01:02:44.880 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T01:02:44.880 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:44.880 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T01:02:44.880 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T01:02:44.880 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T01:02:44.880 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:02:44.934 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T01:02:44.974 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T01:02:44.992 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T01:02:44.992 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /sys 2024-04-25T01:02:44.992 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /proc 2024-04-25T01:02:44.993 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /mnt 2024-04-25T01:02:44.993 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /var/tmp 2024-04-25T01:02:44.993 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /home 2024-04-25T01:02:44.993 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /root 2024-04-25T01:02:44.993 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /tmp 2024-04-25T01:02:44.993 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T01:02:45.147 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T01:02:45.156 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-25T01:02:45.162 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T01:02:45.162 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:45.162 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T01:02:45.162 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T01:02:45.237 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-25T01:02:45.285 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-25T01:02:45.327 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-25T01:02:45.368 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-25T01:02:45.408 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T01:02:45.444 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T01:02:45.641 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-25T01:02:45.681 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T01:02:45.730 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-25T01:02:46.322 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T01:02:46.362 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T01:02:46.399 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:02:46.435 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T01:02:46.435 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:46.435 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T01:02:46.435 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:02:46.435 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:02:46.436 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:02:46.442 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:02:46.463 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:02:46.479 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T01:02:46.552 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T01:02:46.552 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:46.552 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T01:02:46.552 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:02:46.638 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T01:02:46.648 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T01:02:46.673 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T01:02:46.673 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:46.673 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T01:02:46.673 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T01:02:46.673 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T01:02:46.673 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T01:02:46.722 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T01:02:46.722 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:46.722 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T01:02:46.722 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T01:02:46.722 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T01:02:46.723 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:02:46.908 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T01:02:46.930 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T01:02:46.930 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:46.930 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T01:02:46.930 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T01:02:46.930 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T01:02:46.930 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T01:02:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T01:02:47.213 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T01:02:47.213 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:47.213 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T01:02:47.213 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T01:02:47.214 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T01:02:47.214 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:02:47.370 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-25T01:02:47.376 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T01:02:47.399 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T01:02:47.399 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:47.399 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T01:02:47.399 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T01:02:47.399 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T01:02:47.400 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T01:02:47.472 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T01:02:47.514 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T01:02:47.515 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T01:02:47.563 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T01:02:47.599 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T01:02:47.618 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T01:02:47.635 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T01:02:47.635 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T01:02:47.719 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T01:02:47.752 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T01:02:47.752 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T01:02:47.811 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T01:02:47.834 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T01:02:47.869 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T01:02:47.912 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:02:47.947 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T01:02:48.034 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T01:02:48.173 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T01:02:48.211 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T01:02:48.243 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T01:02:48.271 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T01:02:48.678 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T01:02:48.711 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T01:02:48.742 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-25T01:02:48.804 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-25T01:02:48.842 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T01:02:49.871 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T01:02:49.902 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T01:02:50.868 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T01:02:50.903 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T01:02:51.003 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T01:02:51.157 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-25T01:02:51.180 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T01:02:51.202 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T01:02:51.202 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:51.202 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T01:02:51.203 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T01:02:51.203 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T01:02:51.203 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T01:02:51.247 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T01:02:51.258 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T01:02:51.347 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-25T01:02:51.404 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-25T01:02:52.306 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T01:02:52.342 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T01:02:52.364 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T01:02:52.364 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:52.364 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T01:02:52.364 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:02:52.364 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:02:52.365 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T01:02:52.409 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T01:02:52.430 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T01:02:52.430 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:52.430 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T01:02:52.431 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T01:02:52.476 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T01:02:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-25T01:02:52.498 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T01:02:52.498 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:52.498 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T01:02:52.498 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T01:02:52.498 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T01:02:52.498 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T01:02:52.650 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-25T01:02:52.733 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-25T01:02:52.801 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T01:02:52.807 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T01:02:52.824 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T01:02:52.824 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:02:52.824 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T01:02:52.824 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T01:02:52.824 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T01:02:52.824 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T01:02:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T01:02:53.173 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-25T01:02:53.240 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T01:02:53.259 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-25T01:02:53.655 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:02:53.656 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-25T01:02:53.656 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-25T01:02:53.656 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-25T01:02:53.656 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-25T01:02:53.656 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-25T01:02:53.656 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-25T01:02:53.656 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-25T01:02:53.656 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:02:53.656 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:02:53.657 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T01:02:53.657 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-25T01:02:53.657 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-25T01:02:53.657 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:02:53.657 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-25T01:02:53.657 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-25T01:02:53.657 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-25T01:02:53.657 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-25T01:02:53.657 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-25T01:02:53.657 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-25T01:02:53.657 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-25T01:02:53.657 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-25T01:02:53.657 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-25T01:02:53.658 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-25T01:02:53.658 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-25T01:02:53.658 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-25T01:02:53.658 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-25T01:02:53.658 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-25T01:02:53.658 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-25T01:02:53.658 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-25T01:02:53.658 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-25T01:02:53.671 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-25T01:02:53.671 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-25T01:02:53.671 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-25T01:02:53.671 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-25T01:02:53.672 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-25T01:02:53.672 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-25T01:02:53.672 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-25T01:02:53.672 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-25T01:02:53.672 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-25T01:02:53.672 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T01:02:53.672 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T01:02:53.672 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T01:02:53.672 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T01:02:53.672 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T01:02:53.672 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T01:02:53.672 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T01:02:53.672 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T01:02:53.672 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T01:02:53.673 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T01:02:53.673 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T01:02:53.673 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T01:02:53.673 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T01:02:53.673 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T01:02:53.673 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T01:02:53.673 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T01:02:53.673 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T01:02:53.673 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T01:02:53.673 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T01:02:53.673 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T01:02:53.673 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T01:02:53.673 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T01:02:53.673 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T01:02:53.674 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T01:02:53.674 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T01:02:53.674 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T01:02:53.674 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T01:02:53.674 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T01:02:53.674 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T01:02:53.674 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T01:02:53.674 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T01:02:53.674 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T01:02:53.674 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T01:02:53.674 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T01:02:53.674 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T01:02:53.674 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T01:02:53.674 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T01:02:53.675 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T01:02:53.675 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T01:02:53.675 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T01:02:53.675 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T01:02:53.675 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T01:02:53.675 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T01:02:53.675 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T01:02:53.675 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T01:02:53.675 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T01:02:53.675 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T01:02:53.675 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T01:02:53.675 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T01:02:53.675 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T01:02:53.675 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T01:02:53.676 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T01:02:53.676 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T01:02:53.676 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T01:02:53.676 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T01:02:53.676 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T01:02:53.676 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T01:02:53.676 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T01:02:53.676 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T01:02:53.676 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T01:02:53.676 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T01:02:53.676 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T01:02:53.676 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T01:02:53.676 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T01:02:53.676 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T01:02:53.677 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T01:02:53.677 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T01:02:53.677 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T01:02:53.677 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T01:02:53.677 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T01:02:53.677 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T01:02:53.677 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T01:02:53.677 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T01:02:53.677 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T01:02:53.677 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T01:02:53.677 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T01:02:53.677 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T01:02:53.677 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T01:02:53.677 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T01:02:53.677 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T01:02:53.678 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T01:02:53.678 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T01:02:53.678 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T01:02:53.678 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T01:02:53.678 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T01:02:53.678 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T01:02:53.678 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T01:02:53.678 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T01:02:53.678 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T01:02:53.678 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T01:02:53.678 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T01:02:53.678 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T01:02:53.678 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T01:02:53.679 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T01:02:53.679 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T01:02:53.679 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T01:02:53.679 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T01:02:53.679 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T01:02:53.679 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T01:02:53.679 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T01:02:53.679 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T01:02:53.679 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T01:02:53.679 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T01:02:53.679 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T01:02:53.679 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T01:02:53.679 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T01:02:53.679 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T01:02:53.679 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T01:02:53.680 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T01:02:53.680 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-25T01:02:53.680 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T01:02:53.680 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-25T01:02:53.680 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T01:02:53.680 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T01:02:53.680 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T01:02:53.680 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T01:02:54.162 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:02:54.215 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:02:54.273 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:02:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-25T01:02:55.806 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T01:02:55.895 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T01:02:55.953 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:02:55.953 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T01:02:55.953 INFO:teuthology.orchestra.run.smithi040.stdout:Upgraded: 2024-04-25T01:02:55.953 INFO:teuthology.orchestra.run.smithi040.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.954 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.954 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T01:02:55.954 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T01:02:55.954 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-04-25T01:02:55.954 INFO:teuthology.orchestra.run.smithi040.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T01:02:55.954 INFO:teuthology.orchestra.run.smithi040.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T01:02:55.954 INFO:teuthology.orchestra.run.smithi040.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T01:02:55.955 INFO:teuthology.orchestra.run.smithi040.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T01:02:55.955 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.955 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.955 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.955 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.955 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:02:55.955 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.955 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.956 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.956 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:02:55.956 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:02:55.956 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:02:55.956 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:02:55.956 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:02:55.956 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.956 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.956 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:02:55.957 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.957 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.957 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.957 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:02:55.957 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:02:55.957 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.957 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.958 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:02:55.958 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T01:02:55.958 INFO:teuthology.orchestra.run.smithi040.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T01:02:55.958 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T01:02:55.958 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T01:02:55.958 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T01:02:55.958 INFO:teuthology.orchestra.run.smithi040.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T01:02:55.958 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T01:02:55.959 INFO:teuthology.orchestra.run.smithi040.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T01:02:55.959 INFO:teuthology.orchestra.run.smithi040.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T01:02:55.959 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T01:02:55.960 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T01:02:55.960 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.960 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.960 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.960 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T01:02:55.960 INFO:teuthology.orchestra.run.smithi040.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T01:02:55.960 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T01:02:55.960 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T01:02:55.960 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T01:02:55.960 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.960 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.960 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T01:02:55.960 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.961 INFO:teuthology.orchestra.run.smithi040.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T01:02:55.961 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T01:02:55.961 INFO:teuthology.orchestra.run.smithi040.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T01:02:55.961 INFO:teuthology.orchestra.run.smithi040.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T01:02:55.961 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T01:02:55.961 INFO:teuthology.orchestra.run.smithi040.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T01:02:55.961 INFO:teuthology.orchestra.run.smithi040.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T01:02:55.961 INFO:teuthology.orchestra.run.smithi040.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T01:02:55.961 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T01:02:55.961 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T01:02:55.962 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T01:02:55.962 INFO:teuthology.orchestra.run.smithi040.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T01:02:55.962 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T01:02:55.962 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T01:02:55.962 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T01:02:55.962 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T01:02:55.962 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T01:02:55.963 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T01:02:55.963 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T01:02:55.963 INFO:teuthology.orchestra.run.smithi040.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T01:02:55.963 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T01:02:55.963 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T01:02:55.963 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T01:02:55.963 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.963 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T01:02:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T01:02:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T01:02:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T01:02:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T01:02:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T01:02:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T01:02:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T01:02:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T01:02:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T01:02:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T01:02:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T01:02:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T01:02:55.965 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T01:02:55.965 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T01:02:55.965 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T01:02:55.965 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T01:02:55.965 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T01:02:55.965 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T01:02:55.965 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T01:02:55.965 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T01:02:55.965 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T01:02:55.965 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T01:02:55.965 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T01:02:55.965 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T01:02:55.965 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T01:02:55.965 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T01:02:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T01:02:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T01:02:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T01:02:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T01:02:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T01:02:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T01:02:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T01:02:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T01:02:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T01:02:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T01:02:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T01:02:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T01:02:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T01:02:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T01:02:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T01:02:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T01:02:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T01:02:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T01:02:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T01:02:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T01:02:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T01:02:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T01:02:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T01:02:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T01:02:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T01:02:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T01:02:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T01:02:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T01:02:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T01:02:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T01:02:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T01:02:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:02:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T01:02:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T01:02:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T01:02:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T01:02:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T01:02:55.969 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T01:02:55.969 INFO:teuthology.orchestra.run.smithi040.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T01:02:55.969 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-25T01:02:55.969 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-25T01:02:56.079 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T01:02:56.127 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T01:02:56.127 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T01:02:56.288 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T01:02:56.334 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T01:02:56.334 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T01:02:56.379 DEBUG:teuthology.parallel:result is None 2024-04-25T01:02:56.461 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T01:02:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:02:56.846 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:02:56.846 INFO:teuthology.orchestra.run.smithi031.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-25T01:02:56.846 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:02:57.330 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-25T01:02:57.372 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-25T01:02:57.413 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-25T01:02:57.444 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T01:02:57.487 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T01:02:57.672 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-25T01:02:57.726 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T01:02:57.756 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-25T01:02:58.375 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:02:58.412 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:02:58.431 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:02:59.259 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-25T01:02:59.342 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T01:02:59.383 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T01:02:59.464 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T01:02:59.496 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T01:02:59.496 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T01:02:59.588 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T01:02:59.632 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T01:02:59.632 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T01:02:59.725 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T01:02:59.797 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:03:02.497 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-25T01:03:02.497 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-25T01:03:02.497 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-25T01:03:02.497 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-25T01:03:02.497 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-25T01:03:02.497 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-25T01:03:02.497 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-25T01:03:02.497 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:03:02.498 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:03:02.498 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T01:03:02.498 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-25T01:03:02.498 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-25T01:03:02.499 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:03:02.499 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-25T01:03:02.499 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-25T01:03:02.499 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-25T01:03:02.499 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-25T01:03:02.499 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-25T01:03:02.499 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-25T01:03:02.499 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-25T01:03:02.499 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-25T01:03:02.499 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-25T01:03:02.499 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-25T01:03:02.499 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-25T01:03:02.499 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-25T01:03:02.500 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-25T01:03:02.500 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-25T01:03:02.500 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-25T01:03:02.500 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-25T01:03:02.500 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-25T01:03:02.500 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-25T01:03:02.500 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-25T01:03:02.500 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-25T01:03:02.500 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-25T01:03:02.500 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-25T01:03:02.500 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-25T01:03:02.500 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-25T01:03:02.500 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-25T01:03:02.500 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-25T01:03:02.501 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T01:03:02.501 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T01:03:02.501 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T01:03:02.501 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T01:03:02.501 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T01:03:02.501 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T01:03:02.501 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T01:03:02.501 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T01:03:02.501 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T01:03:02.501 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T01:03:02.501 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T01:03:02.501 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T01:03:02.501 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T01:03:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T01:03:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T01:03:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T01:03:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T01:03:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T01:03:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T01:03:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T01:03:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T01:03:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T01:03:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T01:03:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T01:03:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T01:03:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T01:03:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T01:03:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T01:03:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T01:03:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T01:03:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T01:03:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T01:03:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T01:03:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T01:03:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T01:03:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T01:03:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T01:03:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T01:03:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T01:03:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T01:03:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T01:03:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T01:03:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T01:03:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T01:03:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T01:03:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T01:03:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T01:03:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T01:03:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T01:03:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T01:03:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T01:03:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T01:03:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T01:03:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T01:03:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T01:03:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T01:03:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T01:03:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T01:03:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T01:03:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T01:03:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T01:03:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T01:03:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T01:03:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T01:03:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T01:03:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T01:03:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T01:03:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T01:03:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T01:03:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T01:03:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T01:03:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T01:03:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T01:03:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T01:03:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T01:03:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T01:03:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T01:03:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T01:03:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T01:03:02.507 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T01:03:02.507 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T01:03:02.507 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T01:03:02.507 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T01:03:02.507 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T01:03:02.507 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T01:03:02.507 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T01:03:02.507 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T01:03:02.507 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T01:03:02.507 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T01:03:02.508 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T01:03:02.508 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T01:03:02.508 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T01:03:02.508 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T01:03:02.508 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T01:03:02.508 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T01:03:02.509 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T01:03:02.509 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T01:03:02.509 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T01:03:02.509 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T01:03:02.509 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T01:03:02.509 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T01:03:02.509 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T01:03:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T01:03:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T01:03:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T01:03:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T01:03:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T01:03:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T01:03:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T01:03:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-25T01:03:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T01:03:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-25T01:03:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T01:03:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T01:03:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T01:03:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T01:03:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:03:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:03:04.935 INFO:teuthology.orchestra.run.smithi031.stdout:Upgraded: 2024-04-25T01:03:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T01:03:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T01:03:04.936 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-25T01:03:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T01:03:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T01:03:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T01:03:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T01:03:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:03:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:03:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:03:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:03:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:03:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:03:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:03:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:03:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:03:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:03:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T01:03:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T01:03:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T01:03:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T01:03:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T01:03:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T01:03:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T01:03:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T01:03:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T01:03:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T01:03:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T01:03:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T01:03:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T01:03:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T01:03:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T01:03:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T01:03:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T01:03:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T01:03:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T01:03:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T01:03:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T01:03:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T01:03:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T01:03:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T01:03:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T01:03:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T01:03:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T01:03:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T01:03:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T01:03:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T01:03:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T01:03:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T01:03:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T01:03:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T01:03:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T01:03:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T01:03:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T01:03:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T01:03:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T01:03:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T01:03:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T01:03:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T01:03:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T01:03:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T01:03:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T01:03:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T01:03:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T01:03:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T01:03:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T01:03:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T01:03:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T01:03:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T01:03:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T01:03:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T01:03:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T01:03:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T01:03:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T01:03:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T01:03:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T01:03:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T01:03:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T01:03:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T01:03:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T01:03:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T01:03:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T01:03:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T01:03:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T01:03:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T01:03:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T01:03:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T01:03:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T01:03:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T01:03:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T01:03:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T01:03:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T01:03:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T01:03:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T01:03:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T01:03:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T01:03:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T01:03:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T01:03:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T01:03:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T01:03:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T01:03:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T01:03:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T01:03:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T01:03:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T01:03:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T01:03:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T01:03:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T01:03:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T01:03:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T01:03:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T01:03:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T01:03:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T01:03:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T01:03:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T01:03:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T01:03:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T01:03:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T01:03:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T01:03:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T01:03:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T01:03:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T01:03:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:03:04.951 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-25T01:03:05.484 DEBUG:teuthology.parallel:result is None 2024-04-25T01:03:05.854 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:03:05.854 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-25T01:03:05.854 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-25T01:03:05.854 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-25T01:03:05.854 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-25T01:03:05.854 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-25T01:03:05.854 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-25T01:03:05.854 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-25T01:03:05.854 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:03:05.855 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:03:05.855 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T01:03:05.855 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-25T01:03:05.855 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-25T01:03:05.855 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:03:05.855 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-25T01:03:05.855 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-25T01:03:05.855 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-25T01:03:05.855 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-25T01:03:05.855 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-25T01:03:05.855 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-25T01:03:05.855 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-25T01:03:05.855 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-25T01:03:05.856 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-25T01:03:05.856 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-25T01:03:05.856 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-25T01:03:05.856 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-25T01:03:05.856 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-25T01:03:05.856 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-25T01:03:05.856 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-25T01:03:05.856 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-25T01:03:05.856 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-25T01:03:05.856 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-25T01:03:05.856 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-25T01:03:05.856 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-25T01:03:05.856 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-25T01:03:05.856 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-25T01:03:05.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-25T01:03:05.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-25T01:03:05.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-25T01:03:05.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-25T01:03:05.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T01:03:05.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T01:03:05.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T01:03:05.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T01:03:05.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T01:03:05.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T01:03:05.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T01:03:05.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T01:03:05.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T01:03:05.857 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T01:03:05.858 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T01:03:05.858 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T01:03:05.858 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T01:03:05.858 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T01:03:05.858 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T01:03:05.858 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T01:03:05.858 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T01:03:05.858 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T01:03:05.858 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T01:03:05.858 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T01:03:05.858 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T01:03:05.858 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T01:03:05.858 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T01:03:05.859 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T01:03:05.859 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T01:03:05.859 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T01:03:05.859 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T01:03:05.859 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T01:03:05.859 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T01:03:05.859 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T01:03:05.859 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T01:03:05.859 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T01:03:05.859 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T01:03:05.859 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T01:03:05.859 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T01:03:05.859 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T01:03:05.860 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T01:03:05.860 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T01:03:05.860 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T01:03:05.860 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T01:03:05.860 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T01:03:05.860 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T01:03:05.860 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T01:03:05.860 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T01:03:05.860 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T01:03:05.860 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T01:03:05.860 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T01:03:05.860 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T01:03:05.860 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T01:03:05.860 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T01:03:05.861 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T01:03:05.861 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T01:03:05.861 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T01:03:05.861 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T01:03:05.861 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T01:03:05.861 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T01:03:05.861 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T01:03:05.861 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T01:03:05.861 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T01:03:05.861 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T01:03:05.861 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T01:03:05.861 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T01:03:05.861 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T01:03:05.861 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T01:03:05.862 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T01:03:05.862 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T01:03:05.862 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T01:03:05.862 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T01:03:05.862 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T01:03:05.862 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T01:03:05.862 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T01:03:05.862 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T01:03:05.862 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T01:03:05.862 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T01:03:05.862 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T01:03:05.862 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T01:03:05.862 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T01:03:05.863 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T01:03:05.863 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T01:03:05.863 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T01:03:05.863 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T01:03:05.863 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T01:03:05.863 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T01:03:05.863 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T01:03:05.863 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T01:03:05.863 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T01:03:05.863 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T01:03:05.863 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T01:03:05.863 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T01:03:05.863 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T01:03:05.863 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T01:03:05.864 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T01:03:05.864 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T01:03:05.864 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T01:03:05.864 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T01:03:05.864 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T01:03:05.864 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T01:03:05.864 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T01:03:05.864 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T01:03:05.864 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T01:03:05.864 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T01:03:05.864 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T01:03:05.864 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T01:03:05.864 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T01:03:05.864 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T01:03:05.865 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T01:03:05.865 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T01:03:05.865 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T01:03:05.865 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T01:03:05.865 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-25T01:03:05.865 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T01:03:05.865 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-25T01:03:05.865 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T01:03:05.865 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T01:03:05.865 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T01:03:05.865 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T01:03:07.928 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:03:07.928 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T01:03:07.928 INFO:teuthology.orchestra.run.smithi169.stdout:Upgraded: 2024-04-25T01:03:07.928 INFO:teuthology.orchestra.run.smithi169.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.928 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.928 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T01:03:07.928 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T01:03:07.928 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-04-25T01:03:07.928 INFO:teuthology.orchestra.run.smithi169.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T01:03:07.928 INFO:teuthology.orchestra.run.smithi169.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T01:03:07.929 INFO:teuthology.orchestra.run.smithi169.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T01:03:07.929 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T01:03:07.929 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.929 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.929 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.929 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.929 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:03:07.929 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.929 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.929 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.929 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:03:07.929 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:03:07.929 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:03:07.929 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:03:07.930 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:03:07.930 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.930 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.930 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:03:07.930 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.930 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.930 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.930 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:03:07.930 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:03:07.930 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.930 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.930 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:03:07.930 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T01:03:07.931 INFO:teuthology.orchestra.run.smithi169.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T01:03:07.931 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T01:03:07.931 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T01:03:07.931 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T01:03:07.931 INFO:teuthology.orchestra.run.smithi169.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T01:03:07.931 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T01:03:07.931 INFO:teuthology.orchestra.run.smithi169.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T01:03:07.931 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T01:03:07.931 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T01:03:07.931 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T01:03:07.931 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.931 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.931 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.932 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T01:03:07.932 INFO:teuthology.orchestra.run.smithi169.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T01:03:07.932 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T01:03:07.932 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T01:03:07.932 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T01:03:07.934 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.934 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.934 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T01:03:07.934 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.934 INFO:teuthology.orchestra.run.smithi169.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T01:03:07.934 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T01:03:07.934 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T01:03:07.934 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T01:03:07.934 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T01:03:07.935 INFO:teuthology.orchestra.run.smithi169.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T01:03:07.935 INFO:teuthology.orchestra.run.smithi169.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T01:03:07.935 INFO:teuthology.orchestra.run.smithi169.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T01:03:07.935 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T01:03:07.935 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T01:03:07.935 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T01:03:07.935 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T01:03:07.935 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T01:03:07.935 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T01:03:07.935 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T01:03:07.935 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T01:03:07.935 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T01:03:07.935 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T01:03:07.936 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T01:03:07.936 INFO:teuthology.orchestra.run.smithi169.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T01:03:07.936 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T01:03:07.936 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T01:03:07.936 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T01:03:07.936 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.936 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.936 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.936 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T01:03:07.936 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T01:03:07.936 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T01:03:07.936 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T01:03:07.936 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T01:03:07.936 INFO:teuthology.orchestra.run.smithi169.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T01:03:07.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T01:03:07.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T01:03:07.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T01:03:07.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T01:03:07.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T01:03:07.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T01:03:07.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T01:03:07.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T01:03:07.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T01:03:07.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T01:03:07.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T01:03:07.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T01:03:07.937 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T01:03:07.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T01:03:07.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T01:03:07.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T01:03:07.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T01:03:07.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T01:03:07.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T01:03:07.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T01:03:07.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T01:03:07.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T01:03:07.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T01:03:07.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T01:03:07.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T01:03:07.938 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T01:03:07.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T01:03:07.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T01:03:07.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T01:03:07.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T01:03:07.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T01:03:07.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T01:03:07.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T01:03:07.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T01:03:07.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T01:03:07.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T01:03:07.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T01:03:07.939 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T01:03:07.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T01:03:07.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T01:03:07.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T01:03:07.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T01:03:07.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T01:03:07.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T01:03:07.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T01:03:07.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T01:03:07.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T01:03:07.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T01:03:07.940 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T01:03:07.940 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T01:03:07.940 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T01:03:07.941 INFO:teuthology.orchestra.run.smithi169.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T01:03:07.941 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.941 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.941 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:03:07.941 INFO:teuthology.orchestra.run.smithi169.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T01:03:07.941 INFO:teuthology.orchestra.run.smithi169.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T01:03:07.941 INFO:teuthology.orchestra.run.smithi169.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T01:03:07.941 INFO:teuthology.orchestra.run.smithi169.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T01:03:07.941 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T01:03:07.941 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T01:03:07.942 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T01:03:07.942 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T01:03:07.942 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-04-25T01:03:08.341 DEBUG:teuthology.parallel:result is None 2024-04-25T01:03:08.342 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:03:08.492 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T01:03:08.551 INFO:teuthology.orchestra.run.smithi031.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-25T01:03:08.552 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-25T01:03:08.552 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-25T01:03:08.554 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:03:08.705 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T01:03:08.733 INFO:teuthology.orchestra.run.smithi040.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-25T01:03:08.733 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-25T01:03:08.733 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-25T01:03:08.735 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:03:08.951 DEBUG:teuthology.orchestra.run.smithi169:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T01:03:08.972 INFO:teuthology.orchestra.run.smithi169.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-25T01:03:08.972 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-25T01:03:08.973 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-25T01:03:08.975 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T01:03:08.975 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T01:03:08.975 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T01:03:09.048 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T01:03:09.049 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T01:03:09.081 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T01:03:09.081 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T01:03:09.109 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T01:03:09.109 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T01:03:09.109 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T01:03:09.177 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T01:03:09.293 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T01:03:09.293 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T01:03:09.322 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T01:03:09.396 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T01:03:09.397 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T01:03:09.420 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T01:03:09.486 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T01:03:09.486 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T01:03:09.487 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T01:03:09.556 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T01:03:09.679 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T01:03:09.679 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T01:03:09.705 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T01:03:09.770 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T01:03:09.771 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T01:03:09.794 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T01:03:09.858 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T01:03:09.858 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T01:03:09.858 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T01:03:09.924 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T01:03:10.040 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T01:03:10.040 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T01:03:10.065 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T01:03:10.130 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T01:03:10.130 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T01:03:10.154 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T01:03:10.218 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T01:03:10.331 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd heartbeat grace': 60, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'SLOW_OPS', 'slow request'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'subvols': {'create': 2}} 2024-04-25T01:03:10.332 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:03:10.332 INFO:tasks.cephadm:Cluster fsid is 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:03:10.332 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T01:03:10.333 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.31', 'mon.b': '172.21.15.40', 'mon.c': '172.21.15.169'} 2024-04-25T01:03:10.333 INFO:tasks.cephadm:First mon is mon.a on smithi031 2024-04-25T01:03:10.333 INFO:tasks.cephadm:First mgr is x 2024-04-25T01:03:10.333 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T01:03:10.333 DEBUG:teuthology.orchestra.run.smithi031:> sudo hostname $(hostname -s) 2024-04-25T01:03:10.403 DEBUG:teuthology.orchestra.run.smithi040:> sudo hostname $(hostname -s) 2024-04-25T01:03:10.428 DEBUG:teuthology.orchestra.run.smithi169:> sudo hostname $(hostname -s) 2024-04-25T01:03:10.452 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T01:03:10.453 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:03:10.674 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54987/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2445-gd2c45d97', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2445.gd2c45d97'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 22:37:46.422067', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54981/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2445-gd2c45d97', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2445.gd2c45d97'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 17:00:00.872117', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-25T01:03:10.814 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 d2c45d977142e024a59b68a8f1dce8f0b9840551 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:03:10.816 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm 2024-04-25T01:03:10.816 DEBUG:teuthology.orchestra.run.smithi031:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T01:03:11.240 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 25 01:03 /home/ubuntu/cephtest/cephadm 2024-04-25T01:03:11.241 DEBUG:teuthology.orchestra.run.smithi040:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T01:03:11.625 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 25 01:03 /home/ubuntu/cephtest/cephadm 2024-04-25T01:03:11.625 DEBUG:teuthology.orchestra.run.smithi169:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T01:03:12.011 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 25 01:03 /home/ubuntu/cephtest/cephadm 2024-04-25T01:03:12.012 DEBUG:teuthology.orchestra.run.smithi031:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T01:03:12.058 DEBUG:teuthology.orchestra.run.smithi040:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T01:03:12.078 DEBUG:teuthology.orchestra.run.smithi169:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T01:03:12.116 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 on all hosts... 2024-04-25T01:03:12.116 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-25T01:03:12.121 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-25T01:03:12.124 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-25T01:03:13.781 INFO:teuthology.orchestra.run.smithi169.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T01:03:13.824 INFO:teuthology.orchestra.run.smithi040.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T01:03:15.178 INFO:teuthology.orchestra.run.smithi031.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T01:04:04.155 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T01:04:04.156 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T01:04:04.156 INFO:teuthology.orchestra.run.smithi031.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-25T01:04:04.156 INFO:teuthology.orchestra.run.smithi031.stdout: "repo_digests": [ 2024-04-25T01:04:04.156 INFO:teuthology.orchestra.run.smithi031.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-25T01:04:04.156 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T01:04:04.156 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T01:04:07.460 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T01:04:07.460 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T01:04:07.460 INFO:teuthology.orchestra.run.smithi169.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-25T01:04:07.460 INFO:teuthology.orchestra.run.smithi169.stdout: "repo_digests": [ 2024-04-25T01:04:07.460 INFO:teuthology.orchestra.run.smithi169.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-25T01:04:07.460 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2024-04-25T01:04:07.461 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T01:04:10.405 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-25T01:04:10.405 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T01:04:10.405 INFO:teuthology.orchestra.run.smithi040.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-25T01:04:10.405 INFO:teuthology.orchestra.run.smithi040.stdout: "repo_digests": [ 2024-04-25T01:04:10.405 INFO:teuthology.orchestra.run.smithi040.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-25T01:04:10.405 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-04-25T01:04:10.405 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-25T01:04:10.436 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph 2024-04-25T01:04:10.536 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /etc/ceph 2024-04-25T01:04:10.569 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /etc/ceph 2024-04-25T01:04:10.601 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /etc/ceph 2024-04-25T01:04:10.694 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 777 /etc/ceph 2024-04-25T01:04:10.723 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 777 /etc/ceph 2024-04-25T01:04:10.751 INFO:tasks.cephadm:Writing seed config... 2024-04-25T01:04:10.753 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T01:04:10.753 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T01:04:10.753 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T01:04:10.753 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T01:04:10.753 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T01:04:10.753 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T01:04:10.753 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T01:04:10.753 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T01:04:10.753 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T01:04:10.753 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T01:04:10.753 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T01:04:10.753 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T01:04:10.753 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T01:04:10.754 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T01:04:10.754 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T01:04:10.754 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T01:04:10.754 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T01:04:10.754 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T01:04:10.754 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T01:04:10.754 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T01:04:10.754 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T01:04:10.754 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T01:04:10.754 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T01:04:10.754 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T01:04:10.754 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T01:04:10.754 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T01:04:10.755 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T01:04:10.755 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T01:04:10.755 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T01:04:10.755 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-25T01:04:10.755 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-25T01:04:10.755 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-25T01:04:10.755 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T01:04:10.755 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T01:04:10.755 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-25T01:04:10.755 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-25T01:04:10.755 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T01:04:10.755 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T01:04:10.755 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-25T01:04:10.755 INFO:tasks.cephadm: override: [osd] filestore flush min = 0 2024-04-25T01:04:10.756 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T01:04:10.756 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T01:04:10.756 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T01:04:10.756 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T01:04:10.756 INFO:tasks.cephadm: override: [osd] osd heartbeat grace = 60 2024-04-25T01:04:10.756 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T01:04:10.756 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T01:04:10.756 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T01:04:10.756 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T01:04:10.757 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T01:04:10.757 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T01:04:10.818 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 95849b94-029f-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 filestore flush min = 0 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd heartbeat grace = 60 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T01:04:10.818 DEBUG:teuthology.orchestra.run.smithi031:mon.a> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T01:04:10.866 DEBUG:teuthology.orchestra.run.smithi031:mgr.x> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T01:04:10.912 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T01:04:10.912 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 -v bootstrap --fsid 95849b94-029f-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.31 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T01:04:11.185 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:04:11.185 INFO:teuthology.orchestra.run.smithi031.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551', '-v', 'bootstrap', '--fsid', '95849b94-029f-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.31', '--skip-admin-label'] 2024-04-25T01:04:11.186 INFO:teuthology.orchestra.run.smithi031.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T01:04:11.186 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying podman|docker is present... 2024-04-25T01:04:11.244 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T01:04:11.244 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying lvm2 is present... 2024-04-25T01:04:11.244 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying time synchronization is in place... 2024-04-25T01:04:11.263 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T01:04:11.264 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T01:04:11.284 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T01:04:11.284 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-25T01:04:11.307 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout enabled 2024-04-25T01:04:11.330 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout active 2024-04-25T01:04:11.330 INFO:teuthology.orchestra.run.smithi031.stdout:Unit chronyd.service is enabled and running 2024-04-25T01:04:11.330 INFO:teuthology.orchestra.run.smithi031.stdout:Repeating the final host check... 2024-04-25T01:04:11.394 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T01:04:11.395 INFO:teuthology.orchestra.run.smithi031.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T01:04:11.395 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl is present 2024-04-25T01:04:11.395 INFO:teuthology.orchestra.run.smithi031.stdout:lvcreate is present 2024-04-25T01:04:11.408 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T01:04:11.409 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T01:04:11.422 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T01:04:11.422 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-25T01:04:11.437 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout enabled 2024-04-25T01:04:11.453 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout active 2024-04-25T01:04:11.454 INFO:teuthology.orchestra.run.smithi031.stdout:Unit chronyd.service is enabled and running 2024-04-25T01:04:11.454 INFO:teuthology.orchestra.run.smithi031.stdout:Host looks OK 2024-04-25T01:04:11.454 INFO:teuthology.orchestra.run.smithi031.stdout:Cluster fsid: 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:04:11.454 INFO:teuthology.orchestra.run.smithi031.stdout:Acquiring lock 140350895336992 on /run/cephadm/95849b94-029f-11ef-bc93-c7b262605968.lock 2024-04-25T01:04:11.454 INFO:teuthology.orchestra.run.smithi031.stdout:Lock 140350895336992 acquired on /run/cephadm/95849b94-029f-11ef-bc93-c7b262605968.lock 2024-04-25T01:04:11.454 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying IP 172.21.15.31 port 3300 ... 2024-04-25T01:04:11.455 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying IP 172.21.15.31 port 6789 ... 2024-04-25T01:04:11.455 INFO:teuthology.orchestra.run.smithi031.stdout:Base mon IP(s) is [172.21.15.31:3300, 172.21.15.31:6789], mon addrv is [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2024-04-25T01:04:11.463 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.31 metric 100 2024-04-25T01:04:11.464 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.31 metric 100 2024-04-25T01:04:11.471 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T01:04:11.471 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-25T01:04:11.471 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-25T01:04:11.479 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T01:04:11.479 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T01:04:11.479 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T01:04:11.479 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-25T01:04:11.479 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:caba/64 scope link noprefixroute 2024-04-25T01:04:11.479 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T01:04:11.480 INFO:teuthology.orchestra.run.smithi031.stdout:Mon IP `172.21.15.31` is in CIDR network `172.21.0.0/20` 2024-04-25T01:04:11.480 INFO:teuthology.orchestra.run.smithi031.stdout:Mon IP `172.21.15.31` is in CIDR network `172.21.0.0/20` 2024-04-25T01:04:11.480 INFO:teuthology.orchestra.run.smithi031.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T01:04:11.481 INFO:teuthology.orchestra.run.smithi031.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T01:04:11.481 INFO:teuthology.orchestra.run.smithi031.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T01:04:12.170 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e 2024-04-25T01:04:12.171 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T01:04:12.171 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T01:04:12.171 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying blob sha256:58a99251f41e8a467bafa0ec849428709568906ca732f10972304f37056be6c5 2024-04-25T01:04:12.171 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-25T01:04:12.172 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying config sha256:f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e 2024-04-25T01:04:12.172 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T01:04:13.453 INFO:teuthology.orchestra.run.smithi031.stdout:ceph: stdout ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev) 2024-04-25T01:04:13.453 INFO:teuthology.orchestra.run.smithi031.stdout:Ceph version: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev) 2024-04-25T01:04:13.453 INFO:teuthology.orchestra.run.smithi031.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T01:04:14.601 INFO:teuthology.orchestra.run.smithi031.stdout:stat: stdout 167 167 2024-04-25T01:04:14.601 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial keys... 2024-04-25T01:04:15.819 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQAPrClm3djWCBAAfhj+iS77xJgQjEpsIejU3Q== 2024-04-25T01:04:17.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQAQrClmQ/hLEhAADDw8svT/Te4L+N32ynNszw== 2024-04-25T01:04:18.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQARrClm5QSjKxAAR3iFdPaztz2WH0QHykRAbA== 2024-04-25T01:04:18.435 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial monmap... 2024-04-25T01:04:19.598 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T01:04:19.599 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T01:04:19.599 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:04:19.599 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T01:04:19.600 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool for a [v2:172.21.15.31:3300,v1:172.21.15.31:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T01:04:19.600 INFO:teuthology.orchestra.run.smithi031.stdout:setting min_mon_release = quincy 2024-04-25T01:04:19.600 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: set fsid to 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:04:19.600 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T01:04:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:04:19.600 INFO:teuthology.orchestra.run.smithi031.stdout:Creating mon... 2024-04-25T01:04:20.861 INFO:teuthology.orchestra.run.smithi031.stdout:create mon.a on 2024-04-25T01:04:21.254 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T01:04:21.505 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T01:04:21.812 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-95849b94-029f-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-95849b94-029f-11ef-bc93-c7b262605968.target. 2024-04-25T01:04:21.813 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-95849b94-029f-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-95849b94-029f-11ef-bc93-c7b262605968.target. 2024-04-25T01:04:22.219 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-95849b94-029f-11ef-bc93-c7b262605968@mon.a 2024-04-25T01:04:22.219 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to reset failed state of unit ceph-95849b94-029f-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-95849b94-029f-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T01:04:22.522 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-95849b94-029f-11ef-bc93-c7b262605968.target.wants/ceph-95849b94-029f-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-95849b94-029f-11ef-bc93-c7b262605968@.service. 2024-04-25T01:04:23.219 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:22 smithi031 podman[27655]: 2024-04-25 01:04:22.988898153 +0000 UTC m=+0.183231793 container create 112f70ee557788b75d16b2d9c72aa5fd85aa6798af970c992afe54800d3a03fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T01:04:23.219 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:23 smithi031 podman[27655]: 2024-04-25 01:04:23.218233909 +0000 UTC m=+0.412567593 container init 112f70ee557788b75d16b2d9c72aa5fd85aa6798af970c992afe54800d3a03fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9) 2024-04-25T01:04:23.380 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T01:04:23.380 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-04-25T01:04:23.402 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T01:04:23.403 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-25T01:04:23.403 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-04-25T01:04:23.403 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T01:04:23.404 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mon to start... 2024-04-25T01:04:23.405 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mon... 2024-04-25T01:04:23.517 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:23 smithi031 podman[27655]: 2024-04-25 01:04:23.230478999 +0000 UTC m=+0.424812643 container start 112f70ee557788b75d16b2d9c72aa5fd85aa6798af970c992afe54800d3a03fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T01:04:23.517 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:23 smithi031 ceph-mon[27674]: mkfs 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:04:23.517 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:23 smithi031 ceph-mon[27674]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T01:04:23.517 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:23 smithi031 bash[27655]: 112f70ee557788b75d16b2d9c72aa5fd85aa6798af970c992afe54800d3a03fa 2024-04-25T01:04:23.517 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:23 smithi031 systemd[1]: Started Ceph mon.a for 95849b94-029f-11ef-bc93-c7b262605968. 2024-04-25T01:04:24.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:24 smithi031 ceph-mon[27674]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T01:04:24.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:24 smithi031 ceph-mon[27674]: monmap epoch 1 2024-04-25T01:04:24.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:24 smithi031 ceph-mon[27674]: fsid 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:04:24.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:24 smithi031 ceph-mon[27674]: last_changed 2024-04-25T01:04:18.937945+0000 2024-04-25T01:04:24.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:24 smithi031 ceph-mon[27674]: created 2024-04-25T01:04:18.937945+0000 2024-04-25T01:04:24.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:24 smithi031 ceph-mon[27674]: min_mon_release 19 (squid) 2024-04-25T01:04:24.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:24 smithi031 ceph-mon[27674]: election_strategy: 1 2024-04-25T01:04:24.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:24 smithi031 ceph-mon[27674]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T01:04:24.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:24 smithi031 ceph-mon[27674]: fsmap 2024-04-25T01:04:24.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:24 smithi031 ceph-mon[27674]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T01:04:24.519 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:24 smithi031 ceph-mon[27674]: mgrmap e1: no daemons active 2024-04-25T01:04:24.519 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:24 smithi031 ceph-mon[27674]: from='client.? 172.21.15.31:0/2397540627' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T01:04:25.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T01:04:25.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout id: 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:04:25.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T01:04:25.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T01:04:25.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout services: 2024-04-25T01:04:25.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.932254s) 2024-04-25T01:04:25.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T01:04:25.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T01:04:25.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T01:04:25.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout data: 2024-04-25T01:04:25.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T01:04:25.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T01:04:25.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T01:04:25.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T01:04:25.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T01:04:25.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.148+0000 7f7136c00700 1 Processor -- start 2024-04-25T01:04:25.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.148+0000 7f7136c00700 1 -- start start 2024-04-25T01:04:25.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.148+0000 7f7136c00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7130107ee0 0x7f7130108300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:25.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.148+0000 7f7136c00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7130108840 con 0x7f7130107ee0 2024-04-25T01:04:25.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.149+0000 7f712fe00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7130107ee0 0x7f7130108300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:25.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.149+0000 7f712fe00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7130107ee0 0x7f7130108300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58294/0 (socket says 172.21.15.31:58294) 2024-04-25T01:04:25.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.149+0000 7f712fe00700 1 -- 172.21.15.31:0/3970761832 learned_addr learned my addr 172.21.15.31:0/3970761832 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:04:25.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.150+0000 7f712fe00700 1 -- 172.21.15.31:0/3970761832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7130108fa0 con 0x7f7130107ee0 2024-04-25T01:04:25.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.150+0000 7f712fe00700 1 --2- 172.21.15.31:0/3970761832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7130107ee0 0x7f7130108300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f71200098a0 tx=0x7f7120009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed549784e400a106 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:25.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.151+0000 7f712ea00700 1 -- 172.21.15.31:0/3970761832 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7120004160 con 0x7f7130107ee0 2024-04-25T01:04:25.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.151+0000 7f712ea00700 1 -- 172.21.15.31:0/3970761832 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7120004300 con 0x7f7130107ee0 2024-04-25T01:04:25.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.151+0000 7f712ea00700 1 -- 172.21.15.31:0/3970761832 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71200036a0 con 0x7f7130107ee0 2024-04-25T01:04:25.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.151+0000 7f7136c00700 1 -- 172.21.15.31:0/3970761832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7130107ee0 msgr2=0x7f7130108300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:25.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.151+0000 7f7136c00700 1 --2- 172.21.15.31:0/3970761832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7130107ee0 0x7f7130108300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f71200098a0 tx=0x7f7120009bb0 comp rx=0 tx=0).stop 2024-04-25T01:04:25.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.151+0000 7f7136c00700 1 -- 172.21.15.31:0/3970761832 shutdown_connections 2024-04-25T01:04:25.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.151+0000 7f7136c00700 1 --2- 172.21.15.31:0/3970761832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7130107ee0 0x7f7130108300 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:25.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.151+0000 7f7136c00700 1 -- 172.21.15.31:0/3970761832 >> 172.21.15.31:0/3970761832 conn(0x7f71301034e0 msgr2=0x7f7130105900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:04:25.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.152+0000 7f7136c00700 1 -- 172.21.15.31:0/3970761832 shutdown_connections 2024-04-25T01:04:25.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.152+0000 7f7136c00700 1 -- 172.21.15.31:0/3970761832 wait complete. 2024-04-25T01:04:25.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.153+0000 7f7136c00700 1 Processor -- start 2024-04-25T01:04:25.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.153+0000 7f7136c00700 1 -- start start 2024-04-25T01:04:25.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.153+0000 7f7136c00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7130107ee0 0x7f713007e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:25.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.153+0000 7f7136c00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7130109470 con 0x7f7130107ee0 2024-04-25T01:04:25.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.154+0000 7f712fe00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7130107ee0 0x7f713007e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:25.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.154+0000 7f712fe00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7130107ee0 0x7f713007e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58296/0 (socket says 172.21.15.31:58296) 2024-04-25T01:04:25.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.154+0000 7f712fe00700 1 -- 172.21.15.31:0/2397540627 learned_addr learned my addr 172.21.15.31:0/2397540627 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:04:25.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.154+0000 7f712fe00700 1 -- 172.21.15.31:0/2397540627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713007e940 con 0x7f7130107ee0 2024-04-25T01:04:25.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.154+0000 7f712fe00700 1 --2- 172.21.15.31:0/2397540627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7130107ee0 0x7f713007e400 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7120009fb0 tx=0x7f7120004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:25.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.155+0000 7f712cc00700 1 -- 172.21.15.31:0/2397540627 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7120003de0 con 0x7f7130107ee0 2024-04-25T01:04:25.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.155+0000 7f712cc00700 1 -- 172.21.15.31:0/2397540627 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7120011630 con 0x7f7130107ee0 2024-04-25T01:04:25.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.155+0000 7f712cc00700 1 -- 172.21.15.31:0/2397540627 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7120011930 con 0x7f7130107ee0 2024-04-25T01:04:25.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.155+0000 7f7136c00700 1 -- 172.21.15.31:0/2397540627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f713007cb30 con 0x7f7130107ee0 2024-04-25T01:04:25.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.155+0000 7f7136c00700 1 -- 172.21.15.31:0/2397540627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f713007d070 con 0x7f7130107ee0 2024-04-25T01:04:25.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.155+0000 7f712cc00700 1 -- 172.21.15.31:0/2397540627 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7120024480 con 0x7f7130107ee0 2024-04-25T01:04:25.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.155+0000 7f712cc00700 1 -- 172.21.15.31:0/2397540627 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f712001a7e0 con 0x7f7130107ee0 2024-04-25T01:04:25.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.156+0000 7f7136c00700 1 -- 172.21.15.31:0/2397540627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7130063890 con 0x7f7130107ee0 2024-04-25T01:04:25.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.160+0000 7f712cc00700 1 -- 172.21.15.31:0/2397540627 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f712001aa80 con 0x7f7130107ee0 2024-04-25T01:04:25.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.241+0000 7f7136c00700 1 -- 172.21.15.31:0/2397540627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f7130194cd0 con 0x7f7130107ee0 2024-04-25T01:04:25.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.241+0000 7f712cc00700 1 -- 172.21.15.31:0/2397540627 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f712001ac60 con 0x7f7130107ee0 2024-04-25T01:04:25.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.242+0000 7f7136c00700 1 -- 172.21.15.31:0/2397540627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7130107ee0 msgr2=0x7f713007e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:25.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.242+0000 7f7136c00700 1 --2- 172.21.15.31:0/2397540627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7130107ee0 0x7f713007e400 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7120009fb0 tx=0x7f7120004750 comp rx=0 tx=0).stop 2024-04-25T01:04:25.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.242+0000 7f7136c00700 1 -- 172.21.15.31:0/2397540627 shutdown_connections 2024-04-25T01:04:25.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.242+0000 7f7136c00700 1 --2- 172.21.15.31:0/2397540627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7130107ee0 0x7f713007e400 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:25.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.242+0000 7f7136c00700 1 -- 172.21.15.31:0/2397540627 >> 172.21.15.31:0/2397540627 conn(0x7f71301034e0 msgr2=0x7f71301920f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:04:25.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.243+0000 7f7136c00700 1 -- 172.21.15.31:0/2397540627 shutdown_connections 2024-04-25T01:04:25.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:24.243+0000 7f7136c00700 1 -- 172.21.15.31:0/2397540627 wait complete. 2024-04-25T01:04:25.017 INFO:teuthology.orchestra.run.smithi031.stdout:mon is available 2024-04-25T01:04:25.018 INFO:teuthology.orchestra.run.smithi031.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T01:04:25.951 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:25 smithi031 ceph-mon[27674]: from='client.? 172.21.15.31:0/659600574' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T01:04:25.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:25 smithi031 ceph-mon[27674]: from='client.? 172.21.15.31:0/659600574' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T01:04:26.537 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T01:04:26.538 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T01:04:26.538 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout fsid = 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:04:26.538 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2024-04-25T01:04:26.538 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T01:04:26.538 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T01:04:26.538 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T01:04:26.538 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T01:04:26.538 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T01:04:26.538 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T01:04:26.538 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T01:04:26.538 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T01:04:26.538 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T01:04:26.539 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-04-25T01:04:26.539 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T01:04:26.539 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T01:04:26.539 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T01:04:26.539 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T01:04:26.539 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T01:04:26.539 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.558+0000 7f1b95e00700 1 Processor -- start 2024-04-25T01:04:26.539 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.558+0000 7f1b95e00700 1 -- start start 2024-04-25T01:04:26.539 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.559+0000 7f1b95e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b90107e80 0x7f1b901082a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:26.539 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.559+0000 7f1b95e00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1b901087e0 con 0x7f1b90107e80 2024-04-25T01:04:26.539 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.560+0000 7f1b8ea00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b90107e80 0x7f1b901082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:26.539 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.560+0000 7f1b8ea00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b90107e80 0x7f1b901082a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58306/0 (socket says 172.21.15.31:58306) 2024-04-25T01:04:26.539 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.560+0000 7f1b8ea00700 1 -- 172.21.15.31:0/1725219002 learned_addr learned my addr 172.21.15.31:0/1725219002 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:04:26.539 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.560+0000 7f1b8ea00700 1 -- 172.21.15.31:0/1725219002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b90108960 con 0x7f1b90107e80 2024-04-25T01:04:26.540 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.560+0000 7f1b8ea00700 1 --2- 172.21.15.31:0/1725219002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b90107e80 0x7f1b901082a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1b800098a0 tx=0x7f1b80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2296f2513da22531 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:26.540 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.561+0000 7f1b8d600700 1 -- 172.21.15.31:0/1725219002 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b80004160 con 0x7f1b90107e80 2024-04-25T01:04:26.540 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.561+0000 7f1b8d600700 1 -- 172.21.15.31:0/1725219002 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b80004300 con 0x7f1b90107e80 2024-04-25T01:04:26.540 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.561+0000 7f1b8d600700 1 -- 172.21.15.31:0/1725219002 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b800036a0 con 0x7f1b90107e80 2024-04-25T01:04:26.540 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.562+0000 7f1b95e00700 1 -- 172.21.15.31:0/1725219002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b90107e80 msgr2=0x7f1b901082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:26.540 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.562+0000 7f1b95e00700 1 --2- 172.21.15.31:0/1725219002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b90107e80 0x7f1b901082a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1b800098a0 tx=0x7f1b80009bb0 comp rx=0 tx=0).stop 2024-04-25T01:04:26.540 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.562+0000 7f1b95e00700 1 -- 172.21.15.31:0/1725219002 shutdown_connections 2024-04-25T01:04:26.540 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.562+0000 7f1b95e00700 1 --2- 172.21.15.31:0/1725219002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b90107e80 0x7f1b901082a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:26.540 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.562+0000 7f1b95e00700 1 -- 172.21.15.31:0/1725219002 >> 172.21.15.31:0/1725219002 conn(0x7f1b901034e0 msgr2=0x7f1b90105900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:04:26.540 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.562+0000 7f1b95e00700 1 -- 172.21.15.31:0/1725219002 shutdown_connections 2024-04-25T01:04:26.540 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.562+0000 7f1b95e00700 1 -- 172.21.15.31:0/1725219002 wait complete. 2024-04-25T01:04:26.540 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.563+0000 7f1b95e00700 1 Processor -- start 2024-04-25T01:04:26.540 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.563+0000 7f1b95e00700 1 -- start start 2024-04-25T01:04:26.540 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.564+0000 7f1b95e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b90107e80 0x7f1b9019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:26.540 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.564+0000 7f1b95e00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1b90109330 con 0x7f1b90107e80 2024-04-25T01:04:26.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.564+0000 7f1b8ea00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b90107e80 0x7f1b9019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:26.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.564+0000 7f1b8ea00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b90107e80 0x7f1b9019d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58318/0 (socket says 172.21.15.31:58318) 2024-04-25T01:04:26.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.564+0000 7f1b8ea00700 1 -- 172.21.15.31:0/659600574 learned_addr learned my addr 172.21.15.31:0/659600574 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:04:26.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.564+0000 7f1b8ea00700 1 -- 172.21.15.31:0/659600574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9019dbe0 con 0x7f1b90107e80 2024-04-25T01:04:26.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.565+0000 7f1b8ea00700 1 --2- 172.21.15.31:0/659600574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b90107e80 0x7f1b9019d6a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1b80000c00 tx=0x7f1b80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:26.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.565+0000 7f1b94a00700 1 -- 172.21.15.31:0/659600574 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b80003db0 con 0x7f1b90107e80 2024-04-25T01:04:26.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.565+0000 7f1b94a00700 1 -- 172.21.15.31:0/659600574 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b80011630 con 0x7f1b90107e80 2024-04-25T01:04:26.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.565+0000 7f1b95e00700 1 -- 172.21.15.31:0/659600574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b9019ded0 con 0x7f1b90107e80 2024-04-25T01:04:26.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.565+0000 7f1b94a00700 1 -- 172.21.15.31:0/659600574 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b80011950 con 0x7f1b90107e80 2024-04-25T01:04:26.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.566+0000 7f1b94a00700 1 -- 172.21.15.31:0/659600574 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1b80011af0 con 0x7f1b90107e80 2024-04-25T01:04:26.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.566+0000 7f1b95e00700 1 -- 172.21.15.31:0/659600574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b901a0b60 con 0x7f1b90107e80 2024-04-25T01:04:26.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.566+0000 7f1b94a00700 1 -- 172.21.15.31:0/659600574 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1b8001b790 con 0x7f1b90107e80 2024-04-25T01:04:26.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.567+0000 7f1b95e00700 1 -- 172.21.15.31:0/659600574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b90063890 con 0x7f1b90107e80 2024-04-25T01:04:26.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.570+0000 7f1b94a00700 1 -- 172.21.15.31:0/659600574 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1b80033080 con 0x7f1b90107e80 2024-04-25T01:04:26.542 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.649+0000 7f1b95e00700 1 -- 172.21.15.31:0/659600574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f1b9002de10 con 0x7f1b90107e80 2024-04-25T01:04:26.542 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.655+0000 7f1b94a00700 1 -- 172.21.15.31:0/659600574 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+454 (secure 0 0 0) 0x7f1b80004260 con 0x7f1b90107e80 2024-04-25T01:04:26.542 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.656+0000 7f1b94a00700 1 -- 172.21.15.31:0/659600574 <== mon.0 v2:172.21.15.31:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1b8001b990 con 0x7f1b90107e80 2024-04-25T01:04:26.542 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.657+0000 7f1b95e00700 1 -- 172.21.15.31:0/659600574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b90107e80 msgr2=0x7f1b9019d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:26.542 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.657+0000 7f1b95e00700 1 --2- 172.21.15.31:0/659600574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b90107e80 0x7f1b9019d6a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1b80000c00 tx=0x7f1b80004750 comp rx=0 tx=0).stop 2024-04-25T01:04:26.542 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.657+0000 7f1b95e00700 1 -- 172.21.15.31:0/659600574 shutdown_connections 2024-04-25T01:04:26.542 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.657+0000 7f1b95e00700 1 --2- 172.21.15.31:0/659600574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b90107e80 0x7f1b9019d6a0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:26.542 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.657+0000 7f1b95e00700 1 -- 172.21.15.31:0/659600574 >> 172.21.15.31:0/659600574 conn(0x7f1b901034e0 msgr2=0x7f1b9007e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:04:26.542 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.657+0000 7f1b95e00700 1 -- 172.21.15.31:0/659600574 shutdown_connections 2024-04-25T01:04:26.542 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:25.658+0000 7f1b95e00700 1 -- 172.21.15.31:0/659600574 wait complete. 2024-04-25T01:04:26.542 INFO:teuthology.orchestra.run.smithi031.stdout:Generating new minimal ceph.conf... 2024-04-25T01:04:27.516 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:27 smithi031 ceph-mon[27674]: from='client.? 172.21.15.31:0/882260864' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:04:27.894 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.106+0000 7fb383200700 1 Processor -- start 2024-04-25T01:04:27.899 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.106+0000 7fb383200700 1 -- start start 2024-04-25T01:04:27.899 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.107+0000 7fb383200700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb37c07ee80 0x7fb37c07f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:27.899 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.107+0000 7fb383200700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c07f870 con 0x7fb37c07ee80 2024-04-25T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.107+0000 7fb37be00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb37c07ee80 0x7fb37c07f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.107+0000 7fb37be00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb37c07ee80 0x7fb37c07f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37692/0 (socket says 172.21.15.31:37692) 2024-04-25T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.107+0000 7fb37be00700 1 -- 172.21.15.31:0/369900846 learned_addr learned my addr 172.21.15.31:0/369900846 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.108+0000 7fb37be00700 1 -- 172.21.15.31:0/369900846 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37c080030 con 0x7fb37c07ee80 2024-04-25T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.108+0000 7fb37be00700 1 --2- 172.21.15.31:0/369900846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb37c07ee80 0x7fb37c07f2a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009e80 tx=0x7fb36c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3208ed2b0a9a5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.109+0000 7fb37b400700 1 -- 172.21.15.31:0/369900846 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb36c017070 con 0x7fb37c07ee80 2024-04-25T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.109+0000 7fb37b400700 1 -- 172.21.15.31:0/369900846 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb36c00b8c0 con 0x7fb37c07ee80 2024-04-25T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.109+0000 7fb37b400700 1 -- 172.21.15.31:0/369900846 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb36c0045c0 con 0x7fb37c07ee80 2024-04-25T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.110+0000 7fb383200700 1 -- 172.21.15.31:0/369900846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb37c07ee80 msgr2=0x7fb37c07f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.110+0000 7fb383200700 1 --2- 172.21.15.31:0/369900846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb37c07ee80 0x7fb37c07f2a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009e80 tx=0x7fb36c00b330 comp rx=0 tx=0).stop 2024-04-25T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.110+0000 7fb383200700 1 -- 172.21.15.31:0/369900846 shutdown_connections 2024-04-25T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.110+0000 7fb383200700 1 --2- 172.21.15.31:0/369900846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb37c07ee80 0x7fb37c07f2a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.110+0000 7fb383200700 1 -- 172.21.15.31:0/369900846 >> 172.21.15.31:0/369900846 conn(0x7fb37c07d520 msgr2=0x7fb37c07d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.110+0000 7fb383200700 1 -- 172.21.15.31:0/369900846 shutdown_connections 2024-04-25T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.110+0000 7fb383200700 1 -- 172.21.15.31:0/369900846 wait complete. 2024-04-25T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.111+0000 7fb383200700 1 Processor -- start 2024-04-25T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.112+0000 7fb383200700 1 -- start start 2024-04-25T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.112+0000 7fb383200700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb37c07ee80 0x7fb37c1a2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.112+0000 7fb383200700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c0813f0 con 0x7fb37c07ee80 2024-04-25T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.112+0000 7fb37be00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb37c07ee80 0x7fb37c1a2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.112+0000 7fb37be00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb37c07ee80 0x7fb37c1a2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37700/0 (socket says 172.21.15.31:37700) 2024-04-25T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.113+0000 7fb37be00700 1 -- 172.21.15.31:0/882260864 learned_addr learned my addr 172.21.15.31:0/882260864 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.113+0000 7fb37be00700 1 -- 172.21.15.31:0/882260864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37c1a2880 con 0x7fb37c07ee80 2024-04-25T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.113+0000 7fb37be00700 1 --2- 172.21.15.31:0/882260864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb37c07ee80 0x7fb37c1a2340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009530 tx=0x7fb36c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.114+0000 7fb379600700 1 -- 172.21.15.31:0/882260864 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb36c020020 con 0x7fb37c07ee80 2024-04-25T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.114+0000 7fb383200700 1 -- 172.21.15.31:0/882260864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37c1a2b10 con 0x7fb37c07ee80 2024-04-25T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.114+0000 7fb379600700 1 -- 172.21.15.31:0/882260864 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb36c003b50 con 0x7fb37c07ee80 2024-04-25T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.114+0000 7fb379600700 1 -- 172.21.15.31:0/882260864 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb36c017040 con 0x7fb37c07ee80 2024-04-25T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.114+0000 7fb383200700 1 -- 172.21.15.31:0/882260864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb37c1a5800 con 0x7fb37c07ee80 2024-04-25T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.114+0000 7fb379600700 1 -- 172.21.15.31:0/882260864 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb36c023020 con 0x7fb37c07ee80 2024-04-25T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.114+0000 7fb379600700 1 -- 172.21.15.31:0/882260864 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb36c00f050 con 0x7fb37c07ee80 2024-04-25T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.115+0000 7fb383200700 1 -- 172.21.15.31:0/882260864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb37c063890 con 0x7fb37c07ee80 2024-04-25T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.118+0000 7fb379600700 1 -- 172.21.15.31:0/882260864 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb36c02a020 con 0x7fb37c07ee80 2024-04-25T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.195+0000 7fb383200700 1 -- 172.21.15.31:0/882260864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fb37c02dcf0 con 0x7fb37c07ee80 2024-04-25T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.196+0000 7fb379600700 1 -- 172.21.15.31:0/882260864 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fb36c00b860 con 0x7fb37c07ee80 2024-04-25T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.197+0000 7fb383200700 1 -- 172.21.15.31:0/882260864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb37c07ee80 msgr2=0x7fb37c1a2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.197+0000 7fb383200700 1 --2- 172.21.15.31:0/882260864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb37c07ee80 0x7fb37c1a2340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009530 tx=0x7fb36c00bd50 comp rx=0 tx=0).stop 2024-04-25T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.198+0000 7fb383200700 1 -- 172.21.15.31:0/882260864 shutdown_connections 2024-04-25T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.198+0000 7fb383200700 1 --2- 172.21.15.31:0/882260864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb37c07ee80 0x7fb37c1a2340 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.198+0000 7fb383200700 1 -- 172.21.15.31:0/882260864 >> 172.21.15.31:0/882260864 conn(0x7fb37c07d520 msgr2=0x7fb37c108df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:04:27.903 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.198+0000 7fb383200700 1 -- 172.21.15.31:0/882260864 shutdown_connections 2024-04-25T01:04:27.903 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:27.198+0000 7fb383200700 1 -- 172.21.15.31:0/882260864 wait complete. 2024-04-25T01:04:27.903 INFO:teuthology.orchestra.run.smithi031.stdout:Restarting the monitor... 2024-04-25T01:04:28.266 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:27 smithi031 systemd[1]: Stopping Ceph mon.a for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:04:28.719 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:28 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-a[27670]: 2024-04-25T01:04:28.414+0000 7feafb400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:04:28.720 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:28 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-a[27670]: 2024-04-25T01:04:28.414+0000 7feafb400700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T01:04:28.722 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:28 smithi031 podman[27929]: 2024-04-25 01:04:28.466124511 +0000 UTC m=+0.196327926 container died 112f70ee557788b75d16b2d9c72aa5fd85aa6798af970c992afe54800d3a03fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T01:04:29.017 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:28 smithi031 podman[27929]: 2024-04-25 01:04:28.86476642 +0000 UTC m=+0.594969862 container cleanup 112f70ee557788b75d16b2d9c72aa5fd85aa6798af970c992afe54800d3a03fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:04:29.017 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:28 smithi031 bash[27929]: ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-a 2024-04-25T01:04:29.516 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:29 smithi031 podman[27943]: 2024-04-25 01:04:29.074032868 +0000 UTC m=+0.594418957 container remove 112f70ee557788b75d16b2d9c72aa5fd85aa6798af970c992afe54800d3a03fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T01:04:29.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:29 smithi031 systemd[1]: ceph-95849b94-029f-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T01:04:29.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:29 smithi031 systemd[1]: Stopped Ceph mon.a for 95849b94-029f-11ef-bc93-c7b262605968. 2024-04-25T01:04:29.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:29 smithi031 systemd[1]: ceph-95849b94-029f-11ef-bc93-c7b262605968@mon.a.service: Consumed 1.996s CPU time. 2024-04-25T01:04:29.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:29 smithi031 systemd[1]: Starting Ceph mon.a for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:04:30.244 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:29 smithi031 podman[28035]: 2024-04-25 01:04:29.86404163 +0000 UTC m=+0.038366299 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:04:30.244 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:29 smithi031 podman[28035]: 2024-04-25 01:04:29.960452885 +0000 UTC m=+0.134777579 container create b4a98098cf56696d4aed04fea09585ba196427b8ceae955658abde5b734336fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T01:04:30.245 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 podman[28035]: 2024-04-25 01:04:30.247157533 +0000 UTC m=+0.421482234 container init b4a98098cf56696d4aed04fea09585ba196427b8ceae955658abde5b734336fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T01:04:30.336 INFO:teuthology.orchestra.run.smithi031.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T01:04:30.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 podman[28035]: 2024-04-25 01:04:30.258810459 +0000 UTC m=+0.433135163 container start b4a98098cf56696d4aed04fea09585ba196427b8ceae955658abde5b734336fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9) 2024-04-25T01:04:30.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T01:04:30.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-25T01:04:30.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: pidfile_write: ignore empty --pid-file 2024-04-25T01:04:30.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: load: jerasure load: lrc 2024-04-25T01:04:30.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: RocksDB version: 7.9.2 2024-04-25T01:04:30.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Git sha 0 2024-04-25T01:04:30.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-25T01:04:30.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: DB SUMMARY 2024-04-25T01:04:30.519 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: DB Session ID: Q54EANIAOEHOH6SQ448Z 2024-04-25T01:04:30.519 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: CURRENT file: CURRENT 2024-04-25T01:04:30.519 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T01:04:30.519 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T01:04:30.519 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T01:04:30.519 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105548 ; 2024-04-25T01:04:30.519 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.error_if_exists: 0 2024-04-25T01:04:30.519 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.create_if_missing: 0 2024-04-25T01:04:30.519 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.paranoid_checks: 1 2024-04-25T01:04:30.519 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T01:04:30.519 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T01:04:30.519 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T01:04:30.519 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.env: 0x559c148b4c60 2024-04-25T01:04:30.519 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T01:04:30.520 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.info_log: 0x559c27e3bae0 2024-04-25T01:04:30.520 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T01:04:30.520 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.statistics: (nil) 2024-04-25T01:04:30.520 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.use_fsync: 0 2024-04-25T01:04:30.520 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_log_file_size: 0 2024-04-25T01:04:30.520 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T01:04:30.520 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T01:04:30.520 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T01:04:30.520 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T01:04:30.520 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.allow_fallocate: 1 2024-04-25T01:04:30.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T01:04:30.524 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T01:04:30.524 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.use_direct_reads: 0 2024-04-25T01:04:30.524 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T01:04:30.524 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T01:04:30.524 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.db_log_dir: 2024-04-25T01:04:30.524 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.wal_dir: 2024-04-25T01:04:30.524 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T01:04:30.524 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T01:04:30.524 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T01:04:30.524 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T01:04:30.524 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T01:04:30.524 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T01:04:30.525 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T01:04:30.525 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T01:04:30.525 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.write_buffer_manager: 0x559c27e0e1e0 2024-04-25T01:04:30.525 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T01:04:30.525 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T01:04:30.525 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T01:04:30.525 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T01:04:30.525 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T01:04:30.525 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T01:04:30.525 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T01:04:30.525 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T01:04:30.525 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.unordered_write: 0 2024-04-25T01:04:30.525 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T01:04:30.526 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T01:04:30.526 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T01:04:30.527 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T01:04:30.527 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.row_cache: None 2024-04-25T01:04:30.527 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.wal_filter: None 2024-04-25T01:04:30.527 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T01:04:30.527 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T01:04:30.528 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.two_write_queues: 0 2024-04-25T01:04:30.528 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T01:04:30.528 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.wal_compression: 0 2024-04-25T01:04:30.528 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.atomic_flush: 0 2024-04-25T01:04:30.528 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T01:04:30.528 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T01:04:30.528 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T01:04:30.528 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.log_readahead_size: 0 2024-04-25T01:04:30.528 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T01:04:30.528 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T01:04:30.529 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T01:04:30.529 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T01:04:30.529 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T01:04:30.529 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T01:04:30.529 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T01:04:30.529 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_background_jobs: 2 2024-04-25T01:04:30.529 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_background_compactions: -1 2024-04-25T01:04:30.529 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_subcompactions: 1 2024-04-25T01:04:30.529 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T01:04:30.529 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T01:04:30.529 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T01:04:30.529 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T01:04:30.530 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T01:04:30.530 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T01:04:30.530 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T01:04:30.530 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T01:04:30.530 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_open_files: -1 2024-04-25T01:04:30.530 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T01:04:30.530 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T01:04:30.530 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T01:04:30.530 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T01:04:30.530 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_background_flushes: -1 2024-04-25T01:04:30.530 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Compression algorithms supported: 2024-04-25T01:04:30.531 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T01:04:30.531 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: kZSTD supported: 0 2024-04-25T01:04:30.531 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: kXpressCompression supported: 0 2024-04-25T01:04:30.531 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T01:04:30.531 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: kZlibCompression supported: 1 2024-04-25T01:04:30.531 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: kSnappyCompression supported: 1 2024-04-25T01:04:30.531 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: kLZ4Compression supported: 1 2024-04-25T01:04:30.531 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: kBZip2Compression supported: 0 2024-04-25T01:04:30.531 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T01:04:30.531 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T01:04:30.531 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T01:04:30.532 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T01:04:30.532 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T01:04:30.532 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.merge_operator: 2024-04-25T01:04:30.532 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.compaction_filter: None 2024-04-25T01:04:30.532 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T01:04:30.532 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T01:04:30.532 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T01:04:30.533 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T01:04:30.533 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559c27e3bbe0) 2024-04-25T01:04:30.533 INFO:journalctl@ceph.mon.a.smithi031.stdout: cache_index_and_filter_blocks: 1 2024-04-25T01:04:30.533 INFO:journalctl@ceph.mon.a.smithi031.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T01:04:30.533 INFO:journalctl@ceph.mon.a.smithi031.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T01:04:30.533 INFO:journalctl@ceph.mon.a.smithi031.stdout: pin_top_level_index_and_filter: 1 2024-04-25T01:04:30.533 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_type: 0 2024-04-25T01:04:30.533 INFO:journalctl@ceph.mon.a.smithi031.stdout: data_block_index_type: 0 2024-04-25T01:04:30.533 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_shortening: 1 2024-04-25T01:04:30.534 INFO:journalctl@ceph.mon.a.smithi031.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T01:04:30.534 INFO:journalctl@ceph.mon.a.smithi031.stdout: checksum: 4 2024-04-25T01:04:30.534 INFO:journalctl@ceph.mon.a.smithi031.stdout: no_block_cache: 0 2024-04-25T01:04:30.534 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache: 0x559c27e3f090 2024-04-25T01:04:30.534 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_name: BinnedLRUCache 2024-04-25T01:04:30.534 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_options: 2024-04-25T01:04:30.534 INFO:journalctl@ceph.mon.a.smithi031.stdout: capacity : 536870912 2024-04-25T01:04:30.534 INFO:journalctl@ceph.mon.a.smithi031.stdout: num_shard_bits : 4 2024-04-25T01:04:30.534 INFO:journalctl@ceph.mon.a.smithi031.stdout: strict_capacity_limit : 0 2024-04-25T01:04:30.534 INFO:journalctl@ceph.mon.a.smithi031.stdout: high_pri_pool_ratio: 0.000 2024-04-25T01:04:30.534 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_compressed: (nil) 2024-04-25T01:04:30.534 INFO:journalctl@ceph.mon.a.smithi031.stdout: persistent_cache: (nil) 2024-04-25T01:04:30.535 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_size: 4096 2024-04-25T01:04:30.535 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_size_deviation: 10 2024-04-25T01:04:30.535 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_restart_interval: 16 2024-04-25T01:04:30.535 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_block_restart_interval: 1 2024-04-25T01:04:30.535 INFO:journalctl@ceph.mon.a.smithi031.stdout: metadata_block_size: 4096 2024-04-25T01:04:30.535 INFO:journalctl@ceph.mon.a.smithi031.stdout: partition_filters: 0 2024-04-25T01:04:30.535 INFO:journalctl@ceph.mon.a.smithi031.stdout: use_delta_encoding: 1 2024-04-25T01:04:30.535 INFO:journalctl@ceph.mon.a.smithi031.stdout: filter_policy: bloomfilter 2024-04-25T01:04:30.535 INFO:journalctl@ceph.mon.a.smithi031.stdout: whole_key_filtering: 1 2024-04-25T01:04:30.535 INFO:journalctl@ceph.mon.a.smithi031.stdout: verify_compression: 0 2024-04-25T01:04:30.535 INFO:journalctl@ceph.mon.a.smithi031.stdout: read_amp_bytes_per_bit: 0 2024-04-25T01:04:30.535 INFO:journalctl@ceph.mon.a.smithi031.stdout: format_version: 5 2024-04-25T01:04:30.536 INFO:journalctl@ceph.mon.a.smithi031.stdout: enable_index_compression: 1 2024-04-25T01:04:30.536 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_align: 0 2024-04-25T01:04:30.536 INFO:journalctl@ceph.mon.a.smithi031.stdout: max_auto_readahead_size: 262144 2024-04-25T01:04:30.536 INFO:journalctl@ceph.mon.a.smithi031.stdout: prepopulate_block_cache: 0 2024-04-25T01:04:30.536 INFO:journalctl@ceph.mon.a.smithi031.stdout: initial_auto_readahead_size: 8192 2024-04-25T01:04:30.536 INFO:journalctl@ceph.mon.a.smithi031.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T01:04:30.536 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T01:04:30.536 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T01:04:30.536 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.compression: NoCompression 2024-04-25T01:04:30.536 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T01:04:30.536 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T01:04:30.537 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T01:04:30.537 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.num_levels: 7 2024-04-25T01:04:30.537 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T01:04:30.537 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T01:04:30.537 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T01:04:30.537 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T01:04:30.538 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T01:04:30.538 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T01:04:30.538 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T01:04:30.538 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T01:04:30.538 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T01:04:30.538 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T01:04:30.538 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T01:04:30.538 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T01:04:30.538 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T01:04:30.538 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T01:04:30.538 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T01:04:30.538 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T01:04:30.539 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T01:04:30.539 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T01:04:30.539 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T01:04:30.539 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T01:04:30.539 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T01:04:30.539 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T01:04:30.539 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T01:04:30.539 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T01:04:30.539 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T01:04:30.539 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T01:04:30.539 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T01:04:30.539 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T01:04:30.539 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T01:04:30.540 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T01:04:30.540 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T01:04:30.540 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T01:04:30.540 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T01:04:30.540 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T01:04:30.540 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T01:04:30.540 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T01:04:30.540 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T01:04:30.540 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T01:04:30.541 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T01:04:30.541 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T01:04:30.541 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T01:04:30.541 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T01:04:30.541 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T01:04:30.541 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T01:04:30.541 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T01:04:30.541 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T01:04:30.541 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T01:04:30.541 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T01:04:30.541 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T01:04:30.542 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T01:04:30.542 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T01:04:30.542 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T01:04:30.542 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T01:04:30.542 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.table_properties_collectors: 2024-04-25T01:04:30.542 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.inplace_update_support: 0 2024-04-25T01:04:30.542 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T01:04:30.542 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T01:04:30.542 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T01:04:30.542 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T01:04:30.542 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.bloom_locality: 0 2024-04-25T01:04:30.543 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.max_successive_merges: 0 2024-04-25T01:04:30.543 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T01:04:30.543 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T01:04:30.543 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T01:04:30.543 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T01:04:30.543 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.ttl: 2592000 2024-04-25T01:04:30.543 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T01:04:30.543 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T01:04:30.543 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T01:04:30.544 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.enable_blob_files: false 2024-04-25T01:04:30.544 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.min_blob_size: 0 2024-04-25T01:04:30.544 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T01:04:30.544 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T01:04:30.544 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T01:04:30.544 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T01:04:30.544 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T01:04:30.544 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T01:04:30.545 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T01:04:30.545 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T01:04:30.545 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T01:04:30.545 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T01:04:30.545 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: fe25e2d9-69e0-4d35-9aa5-067a9c009397 2024-04-25T01:04:30.545 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714007070319590, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T01:04:30.545 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T01:04:30.546 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714007070321305, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100559, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98469, "index_size": 308, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14389, "raw_average_key_size": 49, "raw_value_size": 90684, "raw_average_value_size": 313, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714007070, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "fe25e2d9-69e0-4d35-9aa5-067a9c009397", "db_session_id": "Q54EANIAOEHOH6SQ448Z", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T01:04:30.546 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714007070321485, "job": 1, "event": "recovery_finished"} 2024-04-25T01:04:30.546 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T01:04:30.546 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T01:04:30.546 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559c27f30000 2024-04-25T01:04:30.546 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: DB pointer 0x559c27f1a000 2024-04-25T01:04:30.546 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T01:04:30.546 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T01:04:30.546 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** DB Stats ** 2024-04-25T01:04:30.546 INFO:journalctl@ceph.mon.a.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T01:04:30.546 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T01:04:30.546 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T01:04:30.547 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T01:04:30.547 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T01:04:30.547 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T01:04:30.547 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T01:04:30.547 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-04-25T01:04:30.547 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** Compaction Stats [default] ** 2024-04-25T01:04:30.547 INFO:journalctl@ceph.mon.a.smithi031.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T01:04:30.547 INFO:journalctl@ceph.mon.a.smithi031.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T01:04:30.548 INFO:journalctl@ceph.mon.a.smithi031.stdout: L0 2/0 100.04 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 85.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T01:04:30.548 INFO:journalctl@ceph.mon.a.smithi031.stdout: Sum 2/0 100.04 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 85.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T01:04:30.548 INFO:journalctl@ceph.mon.a.smithi031.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 85.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T01:04:30.548 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-04-25T01:04:30.548 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** Compaction Stats [default] ** 2024-04-25T01:04:30.548 INFO:journalctl@ceph.mon.a.smithi031.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T01:04:30.548 INFO:journalctl@ceph.mon.a.smithi031.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T01:04:30.548 INFO:journalctl@ceph.mon.a.smithi031.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 85.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T01:04:30.548 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-04-25T01:04:30.548 INFO:journalctl@ceph.mon.a.smithi031.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T01:04:30.548 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-04-25T01:04:30.548 INFO:journalctl@ceph.mon.a.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T01:04:30.549 INFO:journalctl@ceph.mon.a.smithi031.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T01:04:30.549 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T01:04:30.549 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T01:04:30.549 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T01:04:30.549 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T01:04:30.549 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative compaction: 0.00 GB write, 13.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T01:04:30.549 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval compaction: 0.00 GB write, 13.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T01:04:30.549 INFO:journalctl@ceph.mon.a.smithi031.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T01:04:30.549 INFO:journalctl@ceph.mon.a.smithi031.stdout: Block cache BinnedLRUCache@0x559c27e3f090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 2.7e-05 secs_since: 0 2024-04-25T01:04:30.550 INFO:journalctl@ceph.mon.a.smithi031.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T01:04:30.550 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-04-25T01:04:30.550 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T01:04:30.550 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: starting mon.a rank 0 at public addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] at bind addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:04:30.550 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: mon.a@-1(???) e1 preinit fsid 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:04:30.550 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: mon.a@-1(???).mds e1 new map 2024-04-25T01:04:30.550 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: mon.a@-1(???).mds e1 print_map 2024-04-25T01:04:30.550 INFO:journalctl@ceph.mon.a.smithi031.stdout: e1 2024-04-25T01:04:30.550 INFO:journalctl@ceph.mon.a.smithi031.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:04:30.550 INFO:journalctl@ceph.mon.a.smithi031.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T01:04:30.550 INFO:journalctl@ceph.mon.a.smithi031.stdout: legacy client fscid: -1 2024-04-25T01:04:30.551 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-04-25T01:04:30.551 INFO:journalctl@ceph.mon.a.smithi031.stdout: No filesystems configured 2024-04-25T01:04:30.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T01:04:30.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T01:04:30.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T01:04:30.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T01:04:30.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T01:04:30.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 bash[28035]: b4a98098cf56696d4aed04fea09585ba196427b8ceae955658abde5b734336fe 2024-04-25T01:04:30.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 systemd[1]: Started Ceph mon.a for 95849b94-029f-11ef-bc93-c7b262605968. 2024-04-25T01:04:30.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T01:04:30.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: monmap epoch 1 2024-04-25T01:04:30.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: fsid 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:04:30.552 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: last_changed 2024-04-25T01:04:18.937945+0000 2024-04-25T01:04:30.552 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: created 2024-04-25T01:04:18.937945+0000 2024-04-25T01:04:30.552 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: min_mon_release 19 (squid) 2024-04-25T01:04:30.552 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: election_strategy: 1 2024-04-25T01:04:30.552 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T01:04:30.552 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: fsmap 2024-04-25T01:04:30.552 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T01:04:30.552 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:30 smithi031 ceph-mon[28053]: mgrmap e1: no daemons active 2024-04-25T01:04:31.784 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.047+0000 7fe9ba800700 1 Processor -- start 2024-04-25T01:04:31.790 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.047+0000 7fe9ba800700 1 -- start start 2024-04-25T01:04:31.790 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.047+0000 7fe9ba800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b410a150 0x7fe9b410a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:31.790 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.047+0000 7fe9ba800700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b410ab40 con 0x7fe9b410a150 2024-04-25T01:04:31.790 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.048+0000 7fe9b3400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b410a150 0x7fe9b410a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:31.790 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.048+0000 7fe9b3400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b410a150 0x7fe9b410a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37702/0 (socket says 172.21.15.31:37702) 2024-04-25T01:04:31.790 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.048+0000 7fe9b3400700 1 -- 172.21.15.31:0/4023538844 learned_addr learned my addr 172.21.15.31:0/4023538844 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:04:31.790 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.049+0000 7fe9b3400700 1 -- 172.21.15.31:0/4023538844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b410b270 con 0x7fe9b410a150 2024-04-25T01:04:31.790 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.049+0000 7fe9b3400700 1 --2- 172.21.15.31:0/4023538844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b410a150 0x7fe9b410a570 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe9a40098a0 tx=0x7fe9a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58ad075c2d00fa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:31.790 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.050+0000 7fe9b2000700 1 -- 172.21.15.31:0/4023538844 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9a4004160 con 0x7fe9b410a150 2024-04-25T01:04:31.790 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.050+0000 7fe9b2000700 1 -- 172.21.15.31:0/4023538844 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe9a4004300 con 0x7fe9b410a150 2024-04-25T01:04:31.790 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.050+0000 7fe9b2000700 1 -- 172.21.15.31:0/4023538844 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9a4011b50 con 0x7fe9b410a150 2024-04-25T01:04:31.791 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.050+0000 7fe9ba800700 1 -- 172.21.15.31:0/4023538844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b410a150 msgr2=0x7fe9b410a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:31.791 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.050+0000 7fe9ba800700 1 --2- 172.21.15.31:0/4023538844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b410a150 0x7fe9b410a570 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe9a40098a0 tx=0x7fe9a4009bb0 comp rx=0 tx=0).stop 2024-04-25T01:04:31.791 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.050+0000 7fe9ba800700 1 -- 172.21.15.31:0/4023538844 shutdown_connections 2024-04-25T01:04:31.791 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.050+0000 7fe9ba800700 1 --2- 172.21.15.31:0/4023538844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b410a150 0x7fe9b410a570 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:31.791 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.050+0000 7fe9ba800700 1 -- 172.21.15.31:0/4023538844 >> 172.21.15.31:0/4023538844 conn(0x7fe9b407d8e0 msgr2=0x7fe9b407dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:04:31.791 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.051+0000 7fe9ba800700 1 -- 172.21.15.31:0/4023538844 shutdown_connections 2024-04-25T01:04:31.791 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.051+0000 7fe9ba800700 1 -- 172.21.15.31:0/4023538844 wait complete. 2024-04-25T01:04:31.791 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.052+0000 7fe9ba800700 1 Processor -- start 2024-04-25T01:04:31.791 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.052+0000 7fe9ba800700 1 -- start start 2024-04-25T01:04:31.791 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.052+0000 7fe9ba800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b410a150 0x7fe9b419e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:31.791 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.052+0000 7fe9ba800700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b410e620 con 0x7fe9b410a150 2024-04-25T01:04:31.791 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.053+0000 7fe9b3400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b410a150 0x7fe9b419e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:31.791 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.053+0000 7fe9b3400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b410a150 0x7fe9b419e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37706/0 (socket says 172.21.15.31:37706) 2024-04-25T01:04:31.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.053+0000 7fe9b3400700 1 -- 172.21.15.31:0/1159660416 learned_addr learned my addr 172.21.15.31:0/1159660416 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:04:31.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.053+0000 7fe9b3400700 1 -- 172.21.15.31:0/1159660416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b419e540 con 0x7fe9b410a150 2024-04-25T01:04:31.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.053+0000 7fe9b3400700 1 --2- 172.21.15.31:0/1159660416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b410a150 0x7fe9b419e000 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4011850 tx=0x7fe9a4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:31.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.054+0000 7fe9abe00700 1 -- 172.21.15.31:0/1159660416 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9a4003af0 con 0x7fe9b410a150 2024-04-25T01:04:31.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.054+0000 7fe9abe00700 1 -- 172.21.15.31:0/1159660416 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe9a4020070 con 0x7fe9b410a150 2024-04-25T01:04:31.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.054+0000 7fe9ba800700 1 -- 172.21.15.31:0/1159660416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b419e7d0 con 0x7fe9b410a150 2024-04-25T01:04:31.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.054+0000 7fe9abe00700 1 -- 172.21.15.31:0/1159660416 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9a40195a0 con 0x7fe9b410a150 2024-04-25T01:04:31.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.054+0000 7fe9ba800700 1 -- 172.21.15.31:0/1159660416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b41a14c0 con 0x7fe9b410a150 2024-04-25T01:04:31.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.054+0000 7fe9abe00700 1 -- 172.21.15.31:0/1159660416 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe9a4019c30 con 0x7fe9b410a150 2024-04-25T01:04:31.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.054+0000 7fe9abe00700 1 -- 172.21.15.31:0/1159660416 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe9a4011e00 con 0x7fe9b410a150 2024-04-25T01:04:31.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.055+0000 7fe9ba800700 1 -- 172.21.15.31:0/1159660416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9b4063890 con 0x7fe9b410a150 2024-04-25T01:04:31.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.059+0000 7fe9abe00700 1 -- 172.21.15.31:0/1159660416 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe9a4003c90 con 0x7fe9b410a150 2024-04-25T01:04:31.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.136+0000 7fe9ba800700 1 -- 172.21.15.31:0/1159660416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fe9b41a1690 con 0x7fe9b410a150 2024-04-25T01:04:31.793 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.139+0000 7fe9abe00700 1 -- 172.21.15.31:0/1159660416 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fe9a401a430 con 0x7fe9b410a150 2024-04-25T01:04:31.793 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.140+0000 7fe9ba800700 1 -- 172.21.15.31:0/1159660416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b410a150 msgr2=0x7fe9b419e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:31.793 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.140+0000 7fe9ba800700 1 --2- 172.21.15.31:0/1159660416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b410a150 0x7fe9b419e000 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4011850 tx=0x7fe9a4011c10 comp rx=0 tx=0).stop 2024-04-25T01:04:31.793 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.140+0000 7fe9ba800700 1 -- 172.21.15.31:0/1159660416 shutdown_connections 2024-04-25T01:04:31.793 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.140+0000 7fe9ba800700 1 --2- 172.21.15.31:0/1159660416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b410a150 0x7fe9b419e000 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:31.793 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.140+0000 7fe9ba800700 1 -- 172.21.15.31:0/1159660416 >> 172.21.15.31:0/1159660416 conn(0x7fe9b407d8e0 msgr2=0x7fe9b4107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:04:31.793 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.140+0000 7fe9ba800700 1 -- 172.21.15.31:0/1159660416 shutdown_connections 2024-04-25T01:04:31.793 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:31.141+0000 7fe9ba800700 1 -- 172.21.15.31:0/1159660416 wait complete. 2024-04-25T01:04:31.793 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T01:04:31.823 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T01:04:31.824 INFO:teuthology.orchestra.run.smithi031.stdout:Creating mgr... 2024-04-25T01:04:31.824 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T01:04:31.825 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T01:04:32.244 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-95849b94-029f-11ef-bc93-c7b262605968@mgr.x 2024-04-25T01:04:32.244 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to reset failed state of unit ceph-95849b94-029f-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-95849b94-029f-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T01:04:32.516 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:32 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1159660416' entity='client.admin' 2024-04-25T01:04:32.540 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-95849b94-029f-11ef-bc93-c7b262605968.target.wants/ceph-95849b94-029f-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-95849b94-029f-11ef-bc93-c7b262605968@.service. 2024-04-25T01:04:32.969 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:32 smithi031 systemd[1]: Starting Ceph mgr.x for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:04:32.969 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:32 smithi031 podman[28264]: 2024-04-25 01:04:32.875917728 +0000 UTC m=+0.037828021 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:04:32.969 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:32 smithi031 podman[28264]: 2024-04-25 01:04:32.97110188 +0000 UTC m=+0.133012226 container create f54adae2952fdce4d53f5a18ad70c7cfb5ff481d3813ae338b2f20794c921f4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T01:04:33.338 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T01:04:35.985 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-04-25T01:04:35.987 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T01:04:35.987 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-25T01:04:35.987 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-04-25T01:04:35.987 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T01:04:35.987 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T01:04:35.987 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-04-25T01:04:35.987 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T01:04:35.987 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-25T01:04:35.987 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-04-25T01:04:35.987 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T01:04:35.987 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr to start... 2024-04-25T01:04:35.988 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr... 2024-04-25T01:04:35.988 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:33 smithi031 podman[28264]: 2024-04-25 01:04:33.224285946 +0000 UTC m=+0.386196282 container init f54adae2952fdce4d53f5a18ad70c7cfb5ff481d3813ae338b2f20794c921f4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T01:04:35.988 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:33 smithi031 podman[28264]: 2024-04-25 01:04:33.234574434 +0000 UTC m=+0.396484737 container start f54adae2952fdce4d53f5a18ad70c7cfb5ff481d3813ae338b2f20794c921f4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T01:04:35.988 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:33 smithi031 bash[28264]: f54adae2952fdce4d53f5a18ad70c7cfb5ff481d3813ae338b2f20794c921f4d 2024-04-25T01:04:35.988 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:33 smithi031 systemd[1]: Started Ceph mgr.x for 95849b94-029f-11ef-bc93-c7b262605968. 2024-04-25T01:04:35.988 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:33 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:33.496+0000 7f39f5a42200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T01:04:35.988 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:33 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:33.765+0000 7f39f5a42200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:04:35.989 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:33 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:33.912+0000 7f39f5a42200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T01:04:35.989 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:34 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2190123473' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T01:04:35.989 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:34 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:34.350+0000 7f39f5a42200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T01:04:35.989 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:34 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:34.466+0000 7f39f5a42200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T01:04:36.000 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T01:04:36.000 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-25T01:04:36.000 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "95849b94-029f-11ef-bc93-c7b262605968", 2024-04-25T01:04:36.000 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T01:04:36.000 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T01:04:36.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T01:04:36.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T01:04:36.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:36.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T01:04:36.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T01:04:36.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-04-25T01:04:36.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:04:36.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T01:04:36.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T01:04:36.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:04:36.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-25T01:04:36.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T01:04:36.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:04:36.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T01:04:36.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T01:04:36.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:36.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T01:04:36.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:04:36.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T01:04:36.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T01:04:36.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T01:04:36.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T01:04:36.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T01:04:36.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T01:04:36.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:36.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T01:04:36.003 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T01:04:36.003 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T01:04:36.003 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T01:04:36.003 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T01:04:36.003 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T01:04:36.003 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T01:04:36.003 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T01:04:36.003 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T01:04:36.003 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:36.003 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T01:04:36.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:04:36.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T01:04:36.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T01:04:36.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:36.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T01:04:36.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T01:04:36.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T01:04:36.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T01:04:36.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T01:04:36.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T01:04:36.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T01:04:36.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:04:36.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T01:04:36.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:36.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T01:04:36.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:04:36.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T01:04:23.311466+0000", 2024-04-25T01:04:36.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T01:04:36.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:36.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T01:04:36.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-25T01:04:36.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.017+0000 7f0a27e00700 1 Processor -- start 2024-04-25T01:04:36.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.018+0000 7f0a27e00700 1 -- start start 2024-04-25T01:04:36.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.018+0000 7f0a27e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a28109f20 0x7f0a2810a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:36.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.018+0000 7f0a27e00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2810a880 con 0x7f0a28109f20 2024-04-25T01:04:36.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.019+0000 7f0a26a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a28109f20 0x7f0a2810a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:36.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.019+0000 7f0a26a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a28109f20 0x7f0a2810a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37736/0 (socket says 172.21.15.31:37736) 2024-04-25T01:04:36.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.019+0000 7f0a26a00700 1 -- 172.21.15.31:0/1628535792 learned_addr learned my addr 172.21.15.31:0/1628535792 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:04:36.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.019+0000 7f0a26a00700 1 -- 172.21.15.31:0/1628535792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2810afe0 con 0x7f0a28109f20 2024-04-25T01:04:36.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.020+0000 7f0a26a00700 1 --2- 172.21.15.31:0/1628535792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a28109f20 0x7f0a2810a340 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0a180098a0 tx=0x7f0a18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a677a8ced9fb2f22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:36.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.020+0000 7f0a26000700 1 -- 172.21.15.31:0/1628535792 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a18004160 con 0x7f0a28109f20 2024-04-25T01:04:36.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.020+0000 7f0a26000700 1 -- 172.21.15.31:0/1628535792 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0a18004300 con 0x7f0a28109f20 2024-04-25T01:04:36.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.020+0000 7f0a26000700 1 -- 172.21.15.31:0/1628535792 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a18011b50 con 0x7f0a28109f20 2024-04-25T01:04:36.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.021+0000 7f0a27e00700 1 -- 172.21.15.31:0/1628535792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a28109f20 msgr2=0x7f0a2810a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:36.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.021+0000 7f0a27e00700 1 --2- 172.21.15.31:0/1628535792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a28109f20 0x7f0a2810a340 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0a180098a0 tx=0x7f0a18009bb0 comp rx=0 tx=0).stop 2024-04-25T01:04:36.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.021+0000 7f0a27e00700 1 -- 172.21.15.31:0/1628535792 shutdown_connections 2024-04-25T01:04:36.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.021+0000 7f0a27e00700 1 --2- 172.21.15.31:0/1628535792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a28109f20 0x7f0a2810a340 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:36.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.021+0000 7f0a27e00700 1 -- 172.21.15.31:0/1628535792 >> 172.21.15.31:0/1628535792 conn(0x7f0a2807d330 msgr2=0x7f0a2807d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:04:36.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.022+0000 7f0a27e00700 1 -- 172.21.15.31:0/1628535792 shutdown_connections 2024-04-25T01:04:36.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.022+0000 7f0a27e00700 1 -- 172.21.15.31:0/1628535792 wait complete. 2024-04-25T01:04:36.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.023+0000 7f0a27e00700 1 Processor -- start 2024-04-25T01:04:36.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.023+0000 7f0a27e00700 1 -- start start 2024-04-25T01:04:36.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.023+0000 7f0a27e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a28109f20 0x7f0a2819de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:36.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.023+0000 7f0a27e00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2810e2a0 con 0x7f0a28109f20 2024-04-25T01:04:36.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.024+0000 7f0a26a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a28109f20 0x7f0a2819de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:36.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.024+0000 7f0a26a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a28109f20 0x7f0a2819de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37746/0 (socket says 172.21.15.31:37746) 2024-04-25T01:04:36.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.024+0000 7f0a26a00700 1 -- 172.21.15.31:0/2190123473 learned_addr learned my addr 172.21.15.31:0/2190123473 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:04:36.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.024+0000 7f0a26a00700 1 -- 172.21.15.31:0/2190123473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2819e360 con 0x7f0a28109f20 2024-04-25T01:04:36.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.024+0000 7f0a26a00700 1 --2- 172.21.15.31:0/2190123473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a28109f20 0x7f0a2819de20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0a18009fb0 tx=0x7f0a18011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:36.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.025+0000 7f0a1fe00700 1 -- 172.21.15.31:0/2190123473 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a180038b0 con 0x7f0a28109f20 2024-04-25T01:04:36.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.025+0000 7f0a1fe00700 1 -- 172.21.15.31:0/2190123473 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0a18011e50 con 0x7f0a28109f20 2024-04-25T01:04:36.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.025+0000 7f0a27e00700 1 -- 172.21.15.31:0/2190123473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a2819e5f0 con 0x7f0a28109f20 2024-04-25T01:04:36.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.025+0000 7f0a1fe00700 1 -- 172.21.15.31:0/2190123473 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a18019420 con 0x7f0a28109f20 2024-04-25T01:04:36.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.025+0000 7f0a27e00700 1 -- 172.21.15.31:0/2190123473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a281a12e0 con 0x7f0a28109f20 2024-04-25T01:04:36.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.025+0000 7f0a1fe00700 1 -- 172.21.15.31:0/2190123473 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0a18019a60 con 0x7f0a28109f20 2024-04-25T01:04:36.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.026+0000 7f0a1fe00700 1 -- 172.21.15.31:0/2190123473 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0a18024730 con 0x7f0a28109f20 2024-04-25T01:04:36.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.026+0000 7f0a27e00700 1 -- 172.21.15.31:0/2190123473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a2810e420 con 0x7f0a28109f20 2024-04-25T01:04:36.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.029+0000 7f0a1fe00700 1 -- 172.21.15.31:0/2190123473 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0a18043dd0 con 0x7f0a28109f20 2024-04-25T01:04:36.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.110+0000 7f0a27e00700 1 -- 172.21.15.31:0/2190123473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0a28063890 con 0x7f0a28109f20 2024-04-25T01:04:36.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.110+0000 7f0a1fe00700 1 -- 172.21.15.31:0/2190123473 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f0a18032030 con 0x7f0a28109f20 2024-04-25T01:04:36.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.112+0000 7f0a27e00700 1 -- 172.21.15.31:0/2190123473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a28109f20 msgr2=0x7f0a2819de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:36.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.112+0000 7f0a27e00700 1 --2- 172.21.15.31:0/2190123473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a28109f20 0x7f0a2819de20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0a18009fb0 tx=0x7f0a18011bf0 comp rx=0 tx=0).stop 2024-04-25T01:04:36.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.112+0000 7f0a27e00700 1 -- 172.21.15.31:0/2190123473 shutdown_connections 2024-04-25T01:04:36.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.112+0000 7f0a27e00700 1 --2- 172.21.15.31:0/2190123473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a28109f20 0x7f0a2819de20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:36.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.112+0000 7f0a27e00700 1 -- 172.21.15.31:0/2190123473 >> 172.21.15.31:0/2190123473 conn(0x7f0a2807d330 msgr2=0x7f0a28107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:04:36.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.112+0000 7f0a27e00700 1 -- 172.21.15.31:0/2190123473 shutdown_connections 2024-04-25T01:04:36.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:34.112+0000 7f0a27e00700 1 -- 172.21.15.31:0/2190123473 wait complete. 2024-04-25T01:04:36.009 INFO:teuthology.orchestra.run.smithi031.stdout:mgr not available, waiting (1/15)... 2024-04-25T01:04:36.363 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:36 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:36.125+0000 7f39f5a42200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T01:04:36.364 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:36 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:36.366+0000 7f39f5a42200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T01:04:36.766 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:36 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:36.583+0000 7f39f5a42200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T01:04:37.189 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:36 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:36.995+0000 7f39f5a42200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T01:04:37.189 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:37 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:37.124+0000 7f39f5a42200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T01:04:37.514 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:37 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:37.388+0000 7f39f5a42200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T01:04:37.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:37 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2360894725' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T01:04:37.766 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:37 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:37.655+0000 7f39f5a42200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:04:38.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T01:04:38.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-25T01:04:38.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "95849b94-029f-11ef-bc93-c7b262605968", 2024-04-25T01:04:38.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T01:04:38.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T01:04:38.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T01:04:38.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T01:04:38.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:38.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T01:04:38.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T01:04:38.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-04-25T01:04:38.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:04:38.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T01:04:38.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T01:04:38.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:04:38.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-25T01:04:38.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T01:04:38.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:04:38.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T01:04:38.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T01:04:38.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:38.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T01:04:38.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:04:38.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T01:04:38.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T01:04:38.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T01:04:38.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T01:04:38.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T01:04:38.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T01:04:38.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:38.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T01:04:38.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T01:04:38.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T01:04:38.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T01:04:38.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T01:04:38.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T01:04:38.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T01:04:38.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T01:04:38.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T01:04:38.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:38.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T01:04:38.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:04:38.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T01:04:38.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T01:04:38.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:38.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T01:04:38.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T01:04:38.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T01:04:38.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T01:04:38.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T01:04:38.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T01:04:38.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T01:04:38.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:04:38.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T01:04:38.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:38.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T01:04:38.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:04:38.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T01:04:23.311466+0000", 2024-04-25T01:04:38.199 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T01:04:38.199 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:38.199 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T01:04:38.199 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-25T01:04:38.199 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.426+0000 7f241b200700 1 Processor -- start 2024-04-25T01:04:38.199 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.427+0000 7f241b200700 1 -- start start 2024-04-25T01:04:38.199 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.427+0000 7f241b200700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f241c1087b0 0x7f241c10aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:38.199 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.427+0000 7f241b200700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f241c07be30 con 0x7f241c1087b0 2024-04-25T01:04:38.199 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.427+0000 7f2419e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f241c1087b0 0x7f241c10aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:38.199 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.428+0000 7f2419e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f241c1087b0 0x7f241c10aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48202/0 (socket says 172.21.15.31:48202) 2024-04-25T01:04:38.199 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.428+0000 7f2419e00700 1 -- 172.21.15.31:0/82265946 learned_addr learned my addr 172.21.15.31:0/82265946 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:04:38.199 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.428+0000 7f2419e00700 1 -- 172.21.15.31:0/82265946 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241c10b0e0 con 0x7f241c1087b0 2024-04-25T01:04:38.199 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.428+0000 7f2419e00700 1 --2- 172.21.15.31:0/82265946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f241c1087b0 0x7f241c10aba0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f240c0098a0 tx=0x7f240c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4827e8e3c7b70cdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:38.199 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.428+0000 7f2419400700 1 -- 172.21.15.31:0/82265946 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f240c004160 con 0x7f241c1087b0 2024-04-25T01:04:38.200 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.428+0000 7f2419400700 1 -- 172.21.15.31:0/82265946 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f240c004300 con 0x7f241c1087b0 2024-04-25T01:04:38.200 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.429+0000 7f2419400700 1 -- 172.21.15.31:0/82265946 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f240c011b50 con 0x7f241c1087b0 2024-04-25T01:04:38.200 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.429+0000 7f241b200700 1 -- 172.21.15.31:0/82265946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f241c1087b0 msgr2=0x7f241c10aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:38.200 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.429+0000 7f241b200700 1 --2- 172.21.15.31:0/82265946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f241c1087b0 0x7f241c10aba0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f240c0098a0 tx=0x7f240c009bb0 comp rx=0 tx=0).stop 2024-04-25T01:04:38.200 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.429+0000 7f241b200700 1 -- 172.21.15.31:0/82265946 shutdown_connections 2024-04-25T01:04:38.200 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.429+0000 7f241b200700 1 --2- 172.21.15.31:0/82265946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f241c1087b0 0x7f241c10aba0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:38.200 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.429+0000 7f241b200700 1 -- 172.21.15.31:0/82265946 >> 172.21.15.31:0/82265946 conn(0x7f241c102160 msgr2=0x7f241c1045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:04:38.200 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.429+0000 7f241b200700 1 -- 172.21.15.31:0/82265946 shutdown_connections 2024-04-25T01:04:38.200 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.430+0000 7f241b200700 1 -- 172.21.15.31:0/82265946 wait complete. 2024-04-25T01:04:38.200 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.430+0000 7f241b200700 1 Processor -- start 2024-04-25T01:04:38.200 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.431+0000 7f241b200700 1 -- start start 2024-04-25T01:04:38.200 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.431+0000 7f241b200700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f241c1087b0 0x7f241c19bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:38.200 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.431+0000 7f241b200700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f241c10c030 con 0x7f241c1087b0 2024-04-25T01:04:38.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.431+0000 7f2419e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f241c1087b0 0x7f241c19bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:38.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.431+0000 7f2419e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f241c1087b0 0x7f241c19bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48208/0 (socket says 172.21.15.31:48208) 2024-04-25T01:04:38.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.431+0000 7f2419e00700 1 -- 172.21.15.31:0/2360894725 learned_addr learned my addr 172.21.15.31:0/2360894725 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:04:38.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.432+0000 7f2419e00700 1 -- 172.21.15.31:0/2360894725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241c19c1a0 con 0x7f241c1087b0 2024-04-25T01:04:38.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.432+0000 7f2419e00700 1 --2- 172.21.15.31:0/2360894725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f241c1087b0 0x7f241c19bc60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f240c009fb0 tx=0x7f240c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:38.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.432+0000 7f2413400700 1 -- 172.21.15.31:0/2360894725 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f240c0038b0 con 0x7f241c1087b0 2024-04-25T01:04:38.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.432+0000 7f2413400700 1 -- 172.21.15.31:0/2360894725 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f240c011e50 con 0x7f241c1087b0 2024-04-25T01:04:38.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.432+0000 7f241b200700 1 -- 172.21.15.31:0/2360894725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241c19c430 con 0x7f241c1087b0 2024-04-25T01:04:38.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.432+0000 7f2413400700 1 -- 172.21.15.31:0/2360894725 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f240c019420 con 0x7f241c1087b0 2024-04-25T01:04:38.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.432+0000 7f241b200700 1 -- 172.21.15.31:0/2360894725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f241c19f120 con 0x7f241c1087b0 2024-04-25T01:04:38.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.433+0000 7f2413400700 1 -- 172.21.15.31:0/2360894725 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f240c01a430 con 0x7f241c1087b0 2024-04-25T01:04:38.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.433+0000 7f2413400700 1 -- 172.21.15.31:0/2360894725 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f240c02dc70 con 0x7f241c1087b0 2024-04-25T01:04:38.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.434+0000 7f241b200700 1 -- 172.21.15.31:0/2360894725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f241c063890 con 0x7f241c1087b0 2024-04-25T01:04:38.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.436+0000 7f2413400700 1 -- 172.21.15.31:0/2360894725 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f240c043dd0 con 0x7f241c1087b0 2024-04-25T01:04:38.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.517+0000 7f241b200700 1 -- 172.21.15.31:0/2360894725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f241c19f2f0 con 0x7f241c1087b0 2024-04-25T01:04:38.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.517+0000 7f2413400700 1 -- 172.21.15.31:0/2360894725 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f240c032020 con 0x7f241c1087b0 2024-04-25T01:04:38.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.519+0000 7f241b200700 1 -- 172.21.15.31:0/2360894725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f241c1087b0 msgr2=0x7f241c19bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:38.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.519+0000 7f241b200700 1 --2- 172.21.15.31:0/2360894725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f241c1087b0 0x7f241c19bc60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f240c009fb0 tx=0x7f240c011bf0 comp rx=0 tx=0).stop 2024-04-25T01:04:38.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.519+0000 7f241b200700 1 -- 172.21.15.31:0/2360894725 shutdown_connections 2024-04-25T01:04:38.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.519+0000 7f241b200700 1 --2- 172.21.15.31:0/2360894725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f241c1087b0 0x7f241c19bc60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:38.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.519+0000 7f241b200700 1 -- 172.21.15.31:0/2360894725 >> 172.21.15.31:0/2360894725 conn(0x7f241c102160 msgr2=0x7f241c102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:04:38.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.520+0000 7f241b200700 1 -- 172.21.15.31:0/2360894725 shutdown_connections 2024-04-25T01:04:38.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:37.520+0000 7f241b200700 1 -- 172.21.15.31:0/2360894725 wait complete. 2024-04-25T01:04:38.202 INFO:teuthology.orchestra.run.smithi031.stdout:mgr not available, waiting (2/15)... 2024-04-25T01:04:38.766 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:38 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:38.484+0000 7f39f5a42200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T01:04:39.456 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:39 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:39.056+0000 7f39f5a42200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T01:04:39.456 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:39 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:39.187+0000 7f39f5a42200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T01:04:39.456 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:39 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:39.459+0000 7f39f5a42200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T01:04:40.374 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:40 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:40.125+0000 7f39f5a42200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T01:04:40.374 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:40 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:40.265+0000 7f39f5a42200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T01:04:40.668 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:40 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:40.620+0000 7f39f5a42200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T01:04:40.918 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:40 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:40.808+0000 7f39f5a42200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T01:04:41.231 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:40 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3880945416' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T01:04:41.232 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:40 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:40.943+0000 7f39f5a42200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T01:04:41.232 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:41 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:41.109+0000 7f39f5a42200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T01:04:41.516 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:41 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:41.235+0000 7f39f5a42200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T01:04:41.864 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T01:04:41.864 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-25T01:04:41.864 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "95849b94-029f-11ef-bc93-c7b262605968", 2024-04-25T01:04:41.864 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T01:04:41.864 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T01:04:41.864 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T01:04:41.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T01:04:41.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:41.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T01:04:41.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T01:04:41.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-04-25T01:04:41.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:04:41.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T01:04:41.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T01:04:41.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:04:41.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-25T01:04:41.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T01:04:41.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:04:41.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T01:04:41.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T01:04:41.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:41.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T01:04:41.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:04:41.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T01:04:41.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T01:04:41.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T01:04:41.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T01:04:41.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T01:04:41.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T01:04:41.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:41.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T01:04:41.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T01:04:41.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T01:04:41.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T01:04:41.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T01:04:41.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T01:04:41.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T01:04:41.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T01:04:41.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T01:04:41.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:41.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T01:04:41.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:04:41.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T01:04:41.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T01:04:41.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:41.867 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T01:04:41.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T01:04:41.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T01:04:41.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T01:04:41.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T01:04:41.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T01:04:41.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T01:04:41.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:04:41.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T01:04:41.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:41.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T01:04:41.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:04:41.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T01:04:23.311466+0000", 2024-04-25T01:04:41.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T01:04:41.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:41.868 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T01:04:41.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-25T01:04:41.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.829+0000 7fb484e00700 1 Processor -- start 2024-04-25T01:04:41.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.830+0000 7fb484e00700 1 -- start start 2024-04-25T01:04:41.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.830+0000 7fb484e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4801065a0 0x7fb4801069c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:41.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.830+0000 7fb484e00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb48007be30 con 0x7fb4801065a0 2024-04-25T01:04:41.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.830+0000 7fb47f400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4801065a0 0x7fb4801069c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:41.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.830+0000 7fb47f400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4801065a0 0x7fb4801069c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48216/0 (socket says 172.21.15.31:48216) 2024-04-25T01:04:41.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.830+0000 7fb47f400700 1 -- 172.21.15.31:0/3213410421 learned_addr learned my addr 172.21.15.31:0/3213410421 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:04:41.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.831+0000 7fb47f400700 1 -- 172.21.15.31:0/3213410421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb480106f00 con 0x7fb4801065a0 2024-04-25T01:04:41.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.831+0000 7fb47f400700 1 --2- 172.21.15.31:0/3213410421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4801065a0 0x7fb4801069c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb4700098a0 tx=0x7fb470009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ed0fe611abadc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:41.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.831+0000 7fb47e000700 1 -- 172.21.15.31:0/3213410421 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb470004160 con 0x7fb4801065a0 2024-04-25T01:04:41.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.831+0000 7fb47e000700 1 -- 172.21.15.31:0/3213410421 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb470004300 con 0x7fb4801065a0 2024-04-25T01:04:41.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.831+0000 7fb47e000700 1 -- 172.21.15.31:0/3213410421 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb470011b50 con 0x7fb4801065a0 2024-04-25T01:04:41.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.832+0000 7fb484e00700 1 -- 172.21.15.31:0/3213410421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4801065a0 msgr2=0x7fb4801069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:41.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.832+0000 7fb484e00700 1 --2- 172.21.15.31:0/3213410421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4801065a0 0x7fb4801069c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb4700098a0 tx=0x7fb470009bb0 comp rx=0 tx=0).stop 2024-04-25T01:04:41.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.832+0000 7fb484e00700 1 -- 172.21.15.31:0/3213410421 shutdown_connections 2024-04-25T01:04:41.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.832+0000 7fb484e00700 1 --2- 172.21.15.31:0/3213410421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4801065a0 0x7fb4801069c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:41.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.832+0000 7fb484e00700 1 -- 172.21.15.31:0/3213410421 >> 172.21.15.31:0/3213410421 conn(0x7fb480102160 msgr2=0x7fb4801045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:04:41.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.832+0000 7fb484e00700 1 -- 172.21.15.31:0/3213410421 shutdown_connections 2024-04-25T01:04:41.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.832+0000 7fb484e00700 1 -- 172.21.15.31:0/3213410421 wait complete. 2024-04-25T01:04:41.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.833+0000 7fb484e00700 1 Processor -- start 2024-04-25T01:04:41.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.833+0000 7fb484e00700 1 -- start start 2024-04-25T01:04:41.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.834+0000 7fb484e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4801065a0 0x7fb48019de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:41.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.834+0000 7fb484e00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb480107f20 con 0x7fb4801065a0 2024-04-25T01:04:41.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.834+0000 7fb47f400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4801065a0 0x7fb48019de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:41.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.834+0000 7fb47f400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4801065a0 0x7fb48019de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48230/0 (socket says 172.21.15.31:48230) 2024-04-25T01:04:41.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.834+0000 7fb47f400700 1 -- 172.21.15.31:0/3880945416 learned_addr learned my addr 172.21.15.31:0/3880945416 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:04:41.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.834+0000 7fb47f400700 1 -- 172.21.15.31:0/3880945416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48019e3d0 con 0x7fb4801065a0 2024-04-25T01:04:41.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.835+0000 7fb47f400700 1 --2- 172.21.15.31:0/3880945416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4801065a0 0x7fb48019de90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb470011b60 tx=0x7fb470011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:41.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.835+0000 7fb477e00700 1 -- 172.21.15.31:0/3880945416 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb470003970 con 0x7fb4801065a0 2024-04-25T01:04:41.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.835+0000 7fb484e00700 1 -- 172.21.15.31:0/3880945416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb48019e660 con 0x7fb4801065a0 2024-04-25T01:04:41.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.835+0000 7fb477e00700 1 -- 172.21.15.31:0/3880945416 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb470003b10 con 0x7fb4801065a0 2024-04-25T01:04:41.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.835+0000 7fb477e00700 1 -- 172.21.15.31:0/3880945416 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb470024610 con 0x7fb4801065a0 2024-04-25T01:04:41.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.835+0000 7fb484e00700 1 -- 172.21.15.31:0/3880945416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4801a1350 con 0x7fb4801065a0 2024-04-25T01:04:41.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.835+0000 7fb477e00700 1 -- 172.21.15.31:0/3880945416 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb470024c50 con 0x7fb4801065a0 2024-04-25T01:04:41.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.836+0000 7fb477e00700 1 -- 172.21.15.31:0/3880945416 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb470011e00 con 0x7fb4801065a0 2024-04-25T01:04:41.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.836+0000 7fb484e00700 1 -- 172.21.15.31:0/3880945416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb480063890 con 0x7fb4801065a0 2024-04-25T01:04:41.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.839+0000 7fb477e00700 1 -- 172.21.15.31:0/3880945416 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb47001a9f0 con 0x7fb4801065a0 2024-04-25T01:04:41.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.919+0000 7fb484e00700 1 -- 172.21.15.31:0/3880945416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb4800008d0 con 0x7fb4801065a0 2024-04-25T01:04:41.872 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.920+0000 7fb477e00700 1 -- 172.21.15.31:0/3880945416 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fb47001a430 con 0x7fb4801065a0 2024-04-25T01:04:41.872 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.921+0000 7fb484e00700 1 -- 172.21.15.31:0/3880945416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4801065a0 msgr2=0x7fb48019de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:41.872 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.921+0000 7fb484e00700 1 --2- 172.21.15.31:0/3880945416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4801065a0 0x7fb48019de90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb470011b60 tx=0x7fb470011c40 comp rx=0 tx=0).stop 2024-04-25T01:04:41.872 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.921+0000 7fb484e00700 1 -- 172.21.15.31:0/3880945416 shutdown_connections 2024-04-25T01:04:41.872 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.921+0000 7fb484e00700 1 --2- 172.21.15.31:0/3880945416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4801065a0 0x7fb48019de90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:41.872 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.921+0000 7fb484e00700 1 -- 172.21.15.31:0/3880945416 >> 172.21.15.31:0/3880945416 conn(0x7fb480102160 msgr2=0x7fb480194b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:04:41.872 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.921+0000 7fb484e00700 1 -- 172.21.15.31:0/3880945416 shutdown_connections 2024-04-25T01:04:41.872 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:40.921+0000 7fb484e00700 1 -- 172.21.15.31:0/3880945416 wait complete. 2024-04-25T01:04:41.872 INFO:teuthology.orchestra.run.smithi031.stdout:mgr not available, waiting (3/15)... 2024-04-25T01:04:42.563 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:42 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:42.277+0000 7f39f5a42200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T01:04:42.563 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:42 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:42.565+0000 7f39f5a42200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T01:04:43.017 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:42 smithi031 ceph-mon[28053]: Activating manager daemon x 2024-04-25T01:04:43.017 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:42 smithi031 ceph-mon[28053]: mgrmap e2: x(active, starting, since 0.00989702s) 2024-04-25T01:04:43.017 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:42 smithi031 ceph-mon[28053]: from='mgr.14100 172.21.15.31:0/1769539995' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T01:04:43.017 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:42 smithi031 ceph-mon[28053]: from='mgr.14100 172.21.15.31:0/1769539995' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T01:04:43.017 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:42 smithi031 ceph-mon[28053]: from='mgr.14100 172.21.15.31:0/1769539995' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T01:04:43.017 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:42 smithi031 ceph-mon[28053]: from='mgr.14100 172.21.15.31:0/1769539995' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:04:43.017 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:42 smithi031 ceph-mon[28053]: from='mgr.14100 172.21.15.31:0/1769539995' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T01:04:43.017 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:42 smithi031 ceph-mon[28053]: Manager daemon x is now available 2024-04-25T01:04:43.017 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:42 smithi031 ceph-mon[28053]: from='mgr.14100 172.21.15.31:0/1769539995' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T01:04:43.017 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:42 smithi031 ceph-mon[28053]: from='mgr.14100 172.21.15.31:0/1769539995' entity='mgr.x' 2024-04-25T01:04:43.017 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:42 smithi031 ceph-mon[28053]: from='mgr.14100 172.21.15.31:0/1769539995' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T01:04:43.017 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:42 smithi031 ceph-mon[28053]: from='mgr.14100 172.21.15.31:0/1769539995' entity='mgr.x' 2024-04-25T01:04:44.016 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:43 smithi031 ceph-mon[28053]: from='mgr.14100 172.21.15.31:0/1769539995' entity='mgr.x' 2024-04-25T01:04:44.016 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:43 smithi031 ceph-mon[28053]: mgrmap e3: x(active, since 1.02142s) 2024-04-25T01:04:45.467 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T01:04:45.467 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-25T01:04:45.467 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "95849b94-029f-11ef-bc93-c7b262605968", 2024-04-25T01:04:45.467 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T01:04:45.467 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T01:04:45.468 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T01:04:45.468 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T01:04:45.468 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:45.468 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T01:04:45.468 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T01:04:45.468 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-04-25T01:04:45.468 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:04:45.468 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T01:04:45.468 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T01:04:45.468 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:04:45.468 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-25T01:04:45.468 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T01:04:45.468 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:04:45.468 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T01:04:45.469 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T01:04:45.469 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:45.469 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T01:04:45.469 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:04:45.469 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T01:04:45.469 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T01:04:45.469 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T01:04:45.478 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T01:04:45.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T01:04:45.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T01:04:45.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:45.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T01:04:45.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T01:04:45.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T01:04:45.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T01:04:45.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T01:04:45.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T01:04:45.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T01:04:45.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T01:04:45.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T01:04:45.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:45.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T01:04:45.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:04:45.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T01:04:45.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T01:04:45.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:45.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T01:04:45.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T01:04:45.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T01:04:45.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T01:04:45.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T01:04:45.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T01:04:45.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T01:04:45.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:04:45.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T01:04:45.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:45.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T01:04:45.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:04:45.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T01:04:23.311466+0000", 2024-04-25T01:04:45.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T01:04:45.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:04:45.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T01:04:45.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-25T01:04:45.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.453+0000 7fe7b4600700 1 Processor -- start 2024-04-25T01:04:45.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.453+0000 7fe7b4600700 1 -- start start 2024-04-25T01:04:45.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.454+0000 7fe7b4600700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7ac07de30 0x7fe7ac07c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:45.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.454+0000 7fe7b4600700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac07c820 con 0x7fe7ac07de30 2024-04-25T01:04:45.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.454+0000 7fe7b1c00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7ac07de30 0x7fe7ac07c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.454+0000 7fe7b1c00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7ac07de30 0x7fe7ac07c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48292/0 (socket says 172.21.15.31:48292) 2024-04-25T01:04:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.454+0000 7fe7b1c00700 1 -- 172.21.15.31:0/2673402128 learned_addr learned my addr 172.21.15.31:0/2673402128 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:04:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.454+0000 7fe7b1c00700 1 -- 172.21.15.31:0/2673402128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ac07c9a0 con 0x7fe7ac07de30 2024-04-25T01:04:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.455+0000 7fe7b1c00700 1 --2- 172.21.15.31:0/2673402128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7ac07de30 0x7fe7ac07c2e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00098a0 tx=0x7fe7a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5cd53a5ff2ad73f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.455+0000 7fe7abe00700 1 -- 172.21.15.31:0/2673402128 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7a0004160 con 0x7fe7ac07de30 2024-04-25T01:04:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.455+0000 7fe7abe00700 1 -- 172.21.15.31:0/2673402128 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe7a0004300 con 0x7fe7ac07de30 2024-04-25T01:04:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.455+0000 7fe7abe00700 1 -- 172.21.15.31:0/2673402128 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7a0011b50 con 0x7fe7ac07de30 2024-04-25T01:04:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.456+0000 7fe7b4600700 1 -- 172.21.15.31:0/2673402128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7ac07de30 msgr2=0x7fe7ac07c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.456+0000 7fe7b4600700 1 --2- 172.21.15.31:0/2673402128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7ac07de30 0x7fe7ac07c2e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00098a0 tx=0x7fe7a0009bb0 comp rx=0 tx=0).stop 2024-04-25T01:04:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.456+0000 7fe7b4600700 1 -- 172.21.15.31:0/2673402128 shutdown_connections 2024-04-25T01:04:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.456+0000 7fe7b4600700 1 --2- 172.21.15.31:0/2673402128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7ac07de30 0x7fe7ac07c2e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.456+0000 7fe7b4600700 1 -- 172.21.15.31:0/2673402128 >> 172.21.15.31:0/2673402128 conn(0x7fe7ac103420 msgr2=0x7fe7ac105840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:04:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.456+0000 7fe7b4600700 1 -- 172.21.15.31:0/2673402128 shutdown_connections 2024-04-25T01:04:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.456+0000 7fe7b4600700 1 -- 172.21.15.31:0/2673402128 wait complete. 2024-04-25T01:04:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.457+0000 7fe7b4600700 1 Processor -- start 2024-04-25T01:04:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.457+0000 7fe7b4600700 1 -- start start 2024-04-25T01:04:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.458+0000 7fe7b4600700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7ac07de30 0x7fe7ac19df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.458+0000 7fe7b4600700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac10e370 con 0x7fe7ac07de30 2024-04-25T01:04:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.458+0000 7fe7b1c00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7ac07de30 0x7fe7ac19df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.458+0000 7fe7b1c00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7ac07de30 0x7fe7ac19df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48304/0 (socket says 172.21.15.31:48304) 2024-04-25T01:04:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.458+0000 7fe7b1c00700 1 -- 172.21.15.31:0/1515807681 learned_addr learned my addr 172.21.15.31:0/1515807681 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:04:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.458+0000 7fe7b1c00700 1 -- 172.21.15.31:0/1515807681 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ac19e4b0 con 0x7fe7ac07de30 2024-04-25T01:04:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.459+0000 7fe7b1c00700 1 --2- 172.21.15.31:0/1515807681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7ac07de30 0x7fe7ac19df70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0011670 tx=0x7fe7a0011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.459+0000 7fe7aa000700 1 -- 172.21.15.31:0/1515807681 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7a00039c0 con 0x7fe7ac07de30 2024-04-25T01:04:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.459+0000 7fe7aa000700 1 -- 172.21.15.31:0/1515807681 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe7a0011e50 con 0x7fe7ac07de30 2024-04-25T01:04:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.459+0000 7fe7b4600700 1 -- 172.21.15.31:0/1515807681 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ac19e740 con 0x7fe7ac07de30 2024-04-25T01:04:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.459+0000 7fe7aa000700 1 -- 172.21.15.31:0/1515807681 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7a001a5c0 con 0x7fe7ac07de30 2024-04-25T01:04:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.459+0000 7fe7b4600700 1 -- 172.21.15.31:0/1515807681 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ac1a1430 con 0x7fe7ac07de30 2024-04-25T01:04:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.460+0000 7fe7aa000700 1 -- 172.21.15.31:0/1515807681 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7fe7a0024430 con 0x7fe7ac07de30 2024-04-25T01:04:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.460+0000 7fe7b4600700 1 -- 172.21.15.31:0/1515807681 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7ac063890 con 0x7fe7ac07de30 2024-04-25T01:04:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.461+0000 7fe7aa000700 1 --2- 172.21.15.31:0/1515807681 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7fe794038ef0 0x7fe79403b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.461+0000 7fe7aa000700 1 -- 172.21.15.31:0/1515807681 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe7a004d230 con 0x7fe7ac07de30 2024-04-25T01:04:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.461+0000 7fe7b1200700 1 --2- 172.21.15.31:0/1515807681 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7fe794038ef0 0x7fe79403b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.462+0000 7fe7b1200700 1 --2- 172.21.15.31:0/1515807681 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7fe794038ef0 0x7fe79403b3b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe798006fd0 tx=0x7fe798006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.467+0000 7fe7aa000700 1 -- 172.21.15.31:0/1515807681 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe7a002d3f0 con 0x7fe7ac07de30 2024-04-25T01:04:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.631+0000 7fe7aa000700 1 -- 172.21.15.31:0/1515807681 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fe7a0024720 con 0x7fe7ac07de30 2024-04-25T01:04:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.791+0000 7fe7b4600700 1 -- 172.21.15.31:0/1515807681 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe7ac1a1600 con 0x7fe7ac07de30 2024-04-25T01:04:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.792+0000 7fe7aa000700 1 -- 172.21.15.31:0/1515807681 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fe7ac1a1600 con 0x7fe7ac07de30 2024-04-25T01:04:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.795+0000 7fe7b4600700 1 -- 172.21.15.31:0/1515807681 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7fe794038ef0 msgr2=0x7fe79403b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:45.485 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.795+0000 7fe7b4600700 1 --2- 172.21.15.31:0/1515807681 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7fe794038ef0 0x7fe79403b3b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe798006fd0 tx=0x7fe798006e40 comp rx=0 tx=0).stop 2024-04-25T01:04:45.485 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.795+0000 7fe7b4600700 1 -- 172.21.15.31:0/1515807681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7ac07de30 msgr2=0x7fe7ac19df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:45.485 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.795+0000 7fe7b4600700 1 --2- 172.21.15.31:0/1515807681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7ac07de30 0x7fe7ac19df70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0011670 tx=0x7fe7a0011b60 comp rx=0 tx=0).stop 2024-04-25T01:04:45.485 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.795+0000 7fe7b4600700 1 -- 172.21.15.31:0/1515807681 shutdown_connections 2024-04-25T01:04:45.485 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.795+0000 7fe7b4600700 1 --2- 172.21.15.31:0/1515807681 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7fe794038ef0 0x7fe79403b3b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:45.485 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.795+0000 7fe7b4600700 1 --2- 172.21.15.31:0/1515807681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7ac07de30 0x7fe7ac19df70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:45.485 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.795+0000 7fe7b4600700 1 -- 172.21.15.31:0/1515807681 >> 172.21.15.31:0/1515807681 conn(0x7fe7ac103420 msgr2=0x7fe7ac103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:04:45.485 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.795+0000 7fe7b4600700 1 -- 172.21.15.31:0/1515807681 shutdown_connections 2024-04-25T01:04:45.485 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:44.795+0000 7fe7b4600700 1 -- 172.21.15.31:0/1515807681 wait complete. 2024-04-25T01:04:45.485 INFO:teuthology.orchestra.run.smithi031.stdout:mgr is available 2024-04-25T01:04:45.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:45 smithi031 ceph-mon[28053]: mgrmap e4: x(active, since 2s) 2024-04-25T01:04:45.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:45 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1515807681' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T01:04:46.999 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:46 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1668270189' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T01:04:47.020 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T01:04:47.020 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T01:04:47.020 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout fsid = 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:04:47.020 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T01:04:47.020 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T01:04:47.020 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T01:04:47.020 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T01:04:47.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T01:04:47.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T01:04:47.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T01:04:47.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T01:04:47.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T01:04:47.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-04-25T01:04:47.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T01:04:47.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T01:04:47.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T01:04:47.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T01:04:47.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T01:04:47.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.116+0000 7f2084200700 1 Processor -- start 2024-04-25T01:04:47.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.117+0000 7f2084200700 1 -- start start 2024-04-25T01:04:47.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.117+0000 7f2084200700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f207c106d50 0x7f207c107170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:47.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.117+0000 7f2084200700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f207c107740 con 0x7f207c106d50 2024-04-25T01:04:47.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.117+0000 7f2081800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f207c106d50 0x7f207c107170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:47.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.117+0000 7f2081800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f207c106d50 0x7f207c107170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48306/0 (socket says 172.21.15.31:48306) 2024-04-25T01:04:47.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.117+0000 7f2081800700 1 -- 172.21.15.31:0/1412863901 learned_addr learned my addr 172.21.15.31:0/1412863901 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:04:47.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.118+0000 7f2081800700 1 -- 172.21.15.31:0/1412863901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207c107f50 con 0x7f207c106d50 2024-04-25T01:04:47.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.118+0000 7f2081800700 1 --2- 172.21.15.31:0/1412863901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f207c106d50 0x7f207c107170 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f20700098a0 tx=0x7f2070009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7079d480daf72be8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:47.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.119+0000 7f207be00700 1 -- 172.21.15.31:0/1412863901 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2070004160 con 0x7f207c106d50 2024-04-25T01:04:47.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.119+0000 7f207be00700 1 -- 172.21.15.31:0/1412863901 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2070004300 con 0x7f207c106d50 2024-04-25T01:04:47.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.119+0000 7f207be00700 1 -- 172.21.15.31:0/1412863901 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2070011b50 con 0x7f207c106d50 2024-04-25T01:04:47.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.119+0000 7f2084200700 1 -- 172.21.15.31:0/1412863901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f207c106d50 msgr2=0x7f207c107170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:47.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.119+0000 7f2084200700 1 --2- 172.21.15.31:0/1412863901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f207c106d50 0x7f207c107170 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f20700098a0 tx=0x7f2070009bb0 comp rx=0 tx=0).stop 2024-04-25T01:04:47.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.119+0000 7f2084200700 1 -- 172.21.15.31:0/1412863901 shutdown_connections 2024-04-25T01:04:47.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.119+0000 7f2084200700 1 --2- 172.21.15.31:0/1412863901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f207c106d50 0x7f207c107170 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:47.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.119+0000 7f2084200700 1 -- 172.21.15.31:0/1412863901 >> 172.21.15.31:0/1412863901 conn(0x7f207c102330 msgr2=0x7f207c104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:04:47.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.119+0000 7f2084200700 1 -- 172.21.15.31:0/1412863901 shutdown_connections 2024-04-25T01:04:47.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.119+0000 7f2084200700 1 -- 172.21.15.31:0/1412863901 wait complete. 2024-04-25T01:04:47.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.120+0000 7f2084200700 1 Processor -- start 2024-04-25T01:04:47.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.121+0000 7f2084200700 1 -- start start 2024-04-25T01:04:47.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.121+0000 7f2084200700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f207c106d50 0x7f207c19e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:47.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.121+0000 7f2084200700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f207c10b300 con 0x7f207c106d50 2024-04-25T01:04:47.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.121+0000 7f2081800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f207c106d50 0x7f207c19e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:47.026 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.121+0000 7f2081800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f207c106d50 0x7f207c19e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48318/0 (socket says 172.21.15.31:48318) 2024-04-25T01:04:47.026 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.121+0000 7f2081800700 1 -- 172.21.15.31:0/1668270189 learned_addr learned my addr 172.21.15.31:0/1668270189 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:04:47.026 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.122+0000 7f2081800700 1 -- 172.21.15.31:0/1668270189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207c19e5d0 con 0x7f207c106d50 2024-04-25T01:04:47.026 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.122+0000 7f2081800700 1 --2- 172.21.15.31:0/1668270189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f207c106d50 0x7f207c19e090 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2070011850 tx=0x7f2070011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:47.026 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.122+0000 7f207a000700 1 -- 172.21.15.31:0/1668270189 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2070003af0 con 0x7f207c106d50 2024-04-25T01:04:47.026 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.122+0000 7f207a000700 1 -- 172.21.15.31:0/1668270189 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2070020070 con 0x7f207c106d50 2024-04-25T01:04:47.027 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.122+0000 7f207a000700 1 -- 172.21.15.31:0/1668270189 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20700195a0 con 0x7f207c106d50 2024-04-25T01:04:47.027 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.122+0000 7f2084200700 1 -- 172.21.15.31:0/1668270189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f207c19e860 con 0x7f207c106d50 2024-04-25T01:04:47.027 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.123+0000 7f2084200700 1 -- 172.21.15.31:0/1668270189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f207c1a1550 con 0x7f207c106d50 2024-04-25T01:04:47.027 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.124+0000 7f2084200700 1 -- 172.21.15.31:0/1668270189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f207c063890 con 0x7f207c106d50 2024-04-25T01:04:47.027 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.124+0000 7f207a000700 1 -- 172.21.15.31:0/1668270189 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f2070019850 con 0x7f207c106d50 2024-04-25T01:04:47.027 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.125+0000 7f207a000700 1 --2- 172.21.15.31:0/1668270189 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f2064039050 0x7f206403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:47.028 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.125+0000 7f207a000700 1 -- 172.21.15.31:0/1668270189 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2070011e00 con 0x7f207c106d50 2024-04-25T01:04:47.028 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.126+0000 7f2080e00700 1 --2- 172.21.15.31:0/1668270189 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f2064039050 0x7f206403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:47.028 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.126+0000 7f2080e00700 1 --2- 172.21.15.31:0/1668270189 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f2064039050 0x7f206403b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2068006fd0 tx=0x7f2068006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:47.028 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.130+0000 7f207a000700 1 -- 172.21.15.31:0/1668270189 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f207000e260 con 0x7f207c106d50 2024-04-25T01:04:47.028 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.355+0000 7f2084200700 1 -- 172.21.15.31:0/1668270189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f207c1a1720 con 0x7f207c106d50 2024-04-25T01:04:47.028 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.357+0000 7f207a000700 1 -- 172.21.15.31:0/1668270189 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+398 (secure 0 0 0) 0x7f207c1a1720 con 0x7f207c106d50 2024-04-25T01:04:47.029 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.359+0000 7f2084200700 1 -- 172.21.15.31:0/1668270189 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f2064039050 msgr2=0x7f206403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:47.029 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.359+0000 7f2084200700 1 --2- 172.21.15.31:0/1668270189 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f2064039050 0x7f206403b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2068006fd0 tx=0x7f2068006e40 comp rx=0 tx=0).stop 2024-04-25T01:04:47.029 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.359+0000 7f2084200700 1 -- 172.21.15.31:0/1668270189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f207c106d50 msgr2=0x7f207c19e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:47.029 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.359+0000 7f2084200700 1 --2- 172.21.15.31:0/1668270189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f207c106d50 0x7f207c19e090 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2070011850 tx=0x7f2070011c10 comp rx=0 tx=0).stop 2024-04-25T01:04:47.029 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.360+0000 7f2084200700 1 -- 172.21.15.31:0/1668270189 shutdown_connections 2024-04-25T01:04:47.029 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.360+0000 7f2084200700 1 --2- 172.21.15.31:0/1668270189 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f2064039050 0x7f206403b510 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:47.029 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.360+0000 7f2084200700 1 --2- 172.21.15.31:0/1668270189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f207c106d50 0x7f207c19e090 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:47.030 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.360+0000 7f2084200700 1 -- 172.21.15.31:0/1668270189 >> 172.21.15.31:0/1668270189 conn(0x7f207c102330 msgr2=0x7f207c102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:04:47.030 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.360+0000 7f2084200700 1 -- 172.21.15.31:0/1668270189 shutdown_connections 2024-04-25T01:04:47.030 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:46.360+0000 7f2084200700 1 -- 172.21.15.31:0/1668270189 wait complete. 2024-04-25T01:04:47.030 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling cephadm module... 2024-04-25T01:04:48.245 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:47 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3885470644' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T01:04:48.245 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:48 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: ignoring --setuser ceph since I am not root 2024-04-25T01:04:48.245 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:48 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: ignoring --setgroup ceph since I am not root 2024-04-25T01:04:48.516 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:48 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:48.247+0000 7f893592c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T01:04:48.516 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:48 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:48.492+0000 7f893592c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:04:48.641 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.616+0000 7f50ede00700 1 Processor -- start 2024-04-25T01:04:48.641 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.616+0000 7f50ede00700 1 -- start start 2024-04-25T01:04:48.642 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.616+0000 7f50ede00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50e810a150 0x7f50e810a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:48.642 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.617+0000 7f50ede00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f50e810ab40 con 0x7f50e810a150 2024-04-25T01:04:48.642 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.617+0000 7f50e6a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50e810a150 0x7f50e810a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:48.642 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.617+0000 7f50e6a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50e810a150 0x7f50e810a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33256/0 (socket says 172.21.15.31:33256) 2024-04-25T01:04:48.642 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.617+0000 7f50e6a00700 1 -- 172.21.15.31:0/25171901 learned_addr learned my addr 172.21.15.31:0/25171901 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:04:48.642 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.618+0000 7f50e6a00700 1 -- 172.21.15.31:0/25171901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e810b220 con 0x7f50e810a150 2024-04-25T01:04:48.642 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.618+0000 7f50e6a00700 1 --2- 172.21.15.31:0/25171901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50e810a150 0x7f50e810a570 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f50d80098a0 tx=0x7f50d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ac495fbb59c954 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:48.642 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.618+0000 7f50e5600700 1 -- 172.21.15.31:0/25171901 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50d8004160 con 0x7f50e810a150 2024-04-25T01:04:48.642 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.618+0000 7f50e5600700 1 -- 172.21.15.31:0/25171901 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f50d8004300 con 0x7f50e810a150 2024-04-25T01:04:48.642 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.618+0000 7f50e5600700 1 -- 172.21.15.31:0/25171901 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50d8011b50 con 0x7f50e810a150 2024-04-25T01:04:48.642 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.619+0000 7f50ede00700 1 -- 172.21.15.31:0/25171901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50e810a150 msgr2=0x7f50e810a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:48.642 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.619+0000 7f50ede00700 1 --2- 172.21.15.31:0/25171901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50e810a150 0x7f50e810a570 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f50d80098a0 tx=0x7f50d8009bb0 comp rx=0 tx=0).stop 2024-04-25T01:04:48.643 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.619+0000 7f50ede00700 1 -- 172.21.15.31:0/25171901 shutdown_connections 2024-04-25T01:04:48.643 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.619+0000 7f50ede00700 1 --2- 172.21.15.31:0/25171901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50e810a150 0x7f50e810a570 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:48.643 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.619+0000 7f50ede00700 1 -- 172.21.15.31:0/25171901 >> 172.21.15.31:0/25171901 conn(0x7f50e807d520 msgr2=0x7f50e807d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:04:48.643 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.619+0000 7f50ede00700 1 -- 172.21.15.31:0/25171901 shutdown_connections 2024-04-25T01:04:48.643 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.619+0000 7f50ede00700 1 -- 172.21.15.31:0/25171901 wait complete. 2024-04-25T01:04:48.643 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.620+0000 7f50ede00700 1 Processor -- start 2024-04-25T01:04:48.643 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.621+0000 7f50ede00700 1 -- start start 2024-04-25T01:04:48.643 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.621+0000 7f50ede00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50e810a150 0x7f50e819dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:48.643 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.621+0000 7f50ede00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f50e810e5d0 con 0x7f50e810a150 2024-04-25T01:04:48.643 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.621+0000 7f50e6a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50e810a150 0x7f50e819dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:48.643 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.621+0000 7f50e6a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50e810a150 0x7f50e819dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33262/0 (socket says 172.21.15.31:33262) 2024-04-25T01:04:48.643 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.621+0000 7f50e6a00700 1 -- 172.21.15.31:0/3885470644 learned_addr learned my addr 172.21.15.31:0/3885470644 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:04:48.643 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.622+0000 7f50e6a00700 1 -- 172.21.15.31:0/3885470644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e819e520 con 0x7f50e810a150 2024-04-25T01:04:48.643 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.622+0000 7f50e6a00700 1 --2- 172.21.15.31:0/3885470644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50e810a150 0x7f50e819dfe0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f50d8011850 tx=0x7f50d8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:48.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.622+0000 7f50eca00700 1 -- 172.21.15.31:0/3885470644 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50d8003af0 con 0x7f50e810a150 2024-04-25T01:04:48.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.622+0000 7f50eca00700 1 -- 172.21.15.31:0/3885470644 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f50d8020070 con 0x7f50e810a150 2024-04-25T01:04:48.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.622+0000 7f50eca00700 1 -- 172.21.15.31:0/3885470644 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50d80195a0 con 0x7f50e810a150 2024-04-25T01:04:48.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.622+0000 7f50ede00700 1 -- 172.21.15.31:0/3885470644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50e819e7b0 con 0x7f50e810a150 2024-04-25T01:04:48.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.622+0000 7f50ede00700 1 -- 172.21.15.31:0/3885470644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50e81a14a0 con 0x7f50e810a150 2024-04-25T01:04:48.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.624+0000 7f50eca00700 1 -- 172.21.15.31:0/3885470644 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f50d8019850 con 0x7f50e810a150 2024-04-25T01:04:48.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.624+0000 7f50ede00700 1 -- 172.21.15.31:0/3885470644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50e8063890 con 0x7f50e810a150 2024-04-25T01:04:48.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.624+0000 7f50eca00700 1 --2- 172.21.15.31:0/3885470644 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f50cc039050 0x7f50cc03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:48.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.624+0000 7f50eca00700 1 -- 172.21.15.31:0/3885470644 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f50d8011e00 con 0x7f50e810a150 2024-04-25T01:04:48.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.625+0000 7f50e6000700 1 --2- 172.21.15.31:0/3885470644 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f50cc039050 0x7f50cc03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:48.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.625+0000 7f50e6000700 1 --2- 172.21.15.31:0/3885470644 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f50cc039050 0x7f50cc03b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f50d4006fd0 tx=0x7f50d4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:48.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.630+0000 7f50eca00700 1 -- 172.21.15.31:0/3885470644 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f50d800e260 con 0x7f50e810a150 2024-04-25T01:04:48.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.915+0000 7f50ede00700 1 -- 172.21.15.31:0/3885470644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f50e81a1670 con 0x7f50e810a150 2024-04-25T01:04:48.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.968+0000 7f50eca00700 1 -- 172.21.15.31:0/3885470644 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f50e81a1670 con 0x7f50e810a150 2024-04-25T01:04:48.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.972+0000 7f50eca00700 1 -- 172.21.15.31:0/3885470644 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f50d801a430 con 0x7f50e810a150 2024-04-25T01:04:48.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.972+0000 7f50ede00700 1 -- 172.21.15.31:0/3885470644 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f50cc039050 msgr2=0x7f50cc03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:48.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.972+0000 7f50ede00700 1 --2- 172.21.15.31:0/3885470644 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f50cc039050 0x7f50cc03b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f50d4006fd0 tx=0x7f50d4006e40 comp rx=0 tx=0).stop 2024-04-25T01:04:48.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.972+0000 7f50ede00700 1 -- 172.21.15.31:0/3885470644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50e810a150 msgr2=0x7f50e819dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:48.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.972+0000 7f50ede00700 1 --2- 172.21.15.31:0/3885470644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50e810a150 0x7f50e819dfe0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f50d8011850 tx=0x7f50d8011c10 comp rx=0 tx=0).stop 2024-04-25T01:04:48.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.972+0000 7f50ede00700 1 -- 172.21.15.31:0/3885470644 shutdown_connections 2024-04-25T01:04:48.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.972+0000 7f50ede00700 1 --2- 172.21.15.31:0/3885470644 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f50cc039050 0x7f50cc03b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:48.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.972+0000 7f50ede00700 1 --2- 172.21.15.31:0/3885470644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50e810a150 0x7f50e819dfe0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:48.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.972+0000 7f50ede00700 1 -- 172.21.15.31:0/3885470644 >> 172.21.15.31:0/3885470644 conn(0x7f50e807d520 msgr2=0x7f50e8107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:04:48.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.973+0000 7f50ede00700 1 -- 172.21.15.31:0/3885470644 shutdown_connections 2024-04-25T01:04:48.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:47.973+0000 7f50ede00700 1 -- 172.21.15.31:0/3885470644 wait complete. 2024-04-25T01:04:48.914 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:48 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:48.627+0000 7f893592c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T01:04:49.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:48 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3885470644' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T01:04:49.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:48 smithi031 ceph-mon[28053]: mgrmap e5: x(active, since 5s) 2024-04-25T01:04:49.234 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:49 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:49.044+0000 7f893592c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T01:04:49.234 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:49 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:49.161+0000 7f893592c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T01:04:50.169 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:49 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1871790332' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T01:04:50.185 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-25T01:04:50.185 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T01:04:50.185 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T01:04:50.185 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T01:04:50.185 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T01:04:50.186 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-25T01:04:50.186 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.236+0000 7f224e600700 1 Processor -- start 2024-04-25T01:04:50.186 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.236+0000 7f224e600700 1 -- start start 2024-04-25T01:04:50.186 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.237+0000 7f224e600700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2248106d30 0x7f2248107150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:50.186 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.237+0000 7f224e600700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2248107720 con 0x7f2248106d30 2024-04-25T01:04:50.186 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.237+0000 7f2247400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2248106d30 0x7f2248107150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:50.186 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.237+0000 7f2247400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2248106d30 0x7f2248107150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33296/0 (socket says 172.21.15.31:33296) 2024-04-25T01:04:50.186 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.237+0000 7f2247400700 1 -- 172.21.15.31:0/1677283451 learned_addr learned my addr 172.21.15.31:0/1677283451 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:04:50.186 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.238+0000 7f2247400700 1 -- 172.21.15.31:0/1677283451 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2248107f30 con 0x7f2248106d30 2024-04-25T01:04:50.187 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.238+0000 7f2247400700 1 --2- 172.21.15.31:0/1677283451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2248106d30 0x7f2248107150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f22380098a0 tx=0x7f2238009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab2c8737442d1fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:50.187 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.238+0000 7f2246000700 1 -- 172.21.15.31:0/1677283451 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2238004160 con 0x7f2248106d30 2024-04-25T01:04:50.187 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.238+0000 7f2246000700 1 -- 172.21.15.31:0/1677283451 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2238004300 con 0x7f2248106d30 2024-04-25T01:04:50.187 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.238+0000 7f2246000700 1 -- 172.21.15.31:0/1677283451 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2238011b50 con 0x7f2248106d30 2024-04-25T01:04:50.187 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.239+0000 7f224e600700 1 -- 172.21.15.31:0/1677283451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2248106d30 msgr2=0x7f2248107150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:50.187 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.239+0000 7f224e600700 1 --2- 172.21.15.31:0/1677283451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2248106d30 0x7f2248107150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f22380098a0 tx=0x7f2238009bb0 comp rx=0 tx=0).stop 2024-04-25T01:04:50.187 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.240+0000 7f224e600700 1 -- 172.21.15.31:0/1677283451 shutdown_connections 2024-04-25T01:04:50.187 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.240+0000 7f224e600700 1 --2- 172.21.15.31:0/1677283451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2248106d30 0x7f2248107150 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:50.187 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.240+0000 7f224e600700 1 -- 172.21.15.31:0/1677283451 >> 172.21.15.31:0/1677283451 conn(0x7f2248102370 msgr2=0x7f2248104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:04:50.188 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.240+0000 7f224e600700 1 -- 172.21.15.31:0/1677283451 shutdown_connections 2024-04-25T01:04:50.188 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.240+0000 7f224e600700 1 -- 172.21.15.31:0/1677283451 wait complete. 2024-04-25T01:04:50.188 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.241+0000 7f224e600700 1 Processor -- start 2024-04-25T01:04:50.188 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.241+0000 7f224e600700 1 -- start start 2024-04-25T01:04:50.188 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.241+0000 7f224e600700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2248106d30 0x7f224819e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:50.188 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.241+0000 7f224e600700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f224807b860 con 0x7f2248106d30 2024-04-25T01:04:50.188 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.242+0000 7f2247400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2248106d30 0x7f224819e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:50.188 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.242+0000 7f2247400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2248106d30 0x7f224819e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33310/0 (socket says 172.21.15.31:33310) 2024-04-25T01:04:50.188 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.242+0000 7f2247400700 1 -- 172.21.15.31:0/1871790332 learned_addr learned my addr 172.21.15.31:0/1871790332 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:04:50.188 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.242+0000 7f2247400700 1 -- 172.21.15.31:0/1871790332 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224819e540 con 0x7f2248106d30 2024-04-25T01:04:50.188 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.242+0000 7f2247400700 1 --2- 172.21.15.31:0/1871790332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2248106d30 0x7f224819e000 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2238011850 tx=0x7f2238011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:50.188 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.242+0000 7f223fe00700 1 -- 172.21.15.31:0/1871790332 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2238003af0 con 0x7f2248106d30 2024-04-25T01:04:50.188 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.243+0000 7f224e600700 1 -- 172.21.15.31:0/1871790332 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224819e7d0 con 0x7f2248106d30 2024-04-25T01:04:50.188 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.243+0000 7f223fe00700 1 -- 172.21.15.31:0/1871790332 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2238020070 con 0x7f2248106d30 2024-04-25T01:04:50.189 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.243+0000 7f223fe00700 1 -- 172.21.15.31:0/1871790332 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22380195a0 con 0x7f2248106d30 2024-04-25T01:04:50.189 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.243+0000 7f224e600700 1 -- 172.21.15.31:0/1871790332 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22481a14c0 con 0x7f2248106d30 2024-04-25T01:04:50.189 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.244+0000 7f224e600700 1 -- 172.21.15.31:0/1871790332 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2248063890 con 0x7f2248106d30 2024-04-25T01:04:50.189 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.244+0000 7f223fe00700 1 -- 172.21.15.31:0/1871790332 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f2238019870 con 0x7f2248106d30 2024-04-25T01:04:50.189 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.245+0000 7f223fe00700 1 --2- 172.21.15.31:0/1871790332 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f222c0390a0 0x7f222c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:50.189 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.245+0000 7f223fe00700 1 -- 172.21.15.31:0/1871790332 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2238011e00 con 0x7f2248106d30 2024-04-25T01:04:50.189 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.245+0000 7f2246a00700 1 -- 172.21.15.31:0/1871790332 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f222c0390a0 msgr2=0x7f222c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2638309002 2024-04-25T01:04:50.189 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.245+0000 7f2246a00700 1 --2- 172.21.15.31:0/1871790332 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f222c0390a0 0x7f222c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:04:50.189 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.250+0000 7f223fe00700 1 -- 172.21.15.31:0/1871790332 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f223800e260 con 0x7f2248106d30 2024-04-25T01:04:50.189 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.446+0000 7f2246a00700 1 -- 172.21.15.31:0/1871790332 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f222c0390a0 msgr2=0x7f222c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2638309002 2024-04-25T01:04:50.189 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.446+0000 7f2246a00700 1 --2- 172.21.15.31:0/1871790332 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f222c0390a0 0x7f222c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:04:50.189 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.530+0000 7f224e600700 1 -- 172.21.15.31:0/1871790332 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f22480008d0 con 0x7f2248106d30 2024-04-25T01:04:50.190 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.531+0000 7f223fe00700 1 -- 172.21.15.31:0/1871790332 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f2238019b60 con 0x7f2248106d30 2024-04-25T01:04:50.190 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.533+0000 7f224e600700 1 -- 172.21.15.31:0/1871790332 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f222c0390a0 msgr2=0x7f222c03b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:04:50.190 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.533+0000 7f224e600700 1 --2- 172.21.15.31:0/1871790332 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f222c0390a0 0x7f222c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:50.190 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.533+0000 7f224e600700 1 -- 172.21.15.31:0/1871790332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2248106d30 msgr2=0x7f224819e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:50.190 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.533+0000 7f224e600700 1 --2- 172.21.15.31:0/1871790332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2248106d30 0x7f224819e000 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2238011850 tx=0x7f2238011c10 comp rx=0 tx=0).stop 2024-04-25T01:04:50.190 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.534+0000 7f224e600700 1 -- 172.21.15.31:0/1871790332 shutdown_connections 2024-04-25T01:04:50.191 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.534+0000 7f224e600700 1 --2- 172.21.15.31:0/1871790332 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f222c0390a0 0x7f222c03b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:50.191 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.534+0000 7f224e600700 1 --2- 172.21.15.31:0/1871790332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2248106d30 0x7f224819e000 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:50.191 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.534+0000 7f224e600700 1 -- 172.21.15.31:0/1871790332 >> 172.21.15.31:0/1871790332 conn(0x7f2248102370 msgr2=0x7f2248102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:04:50.191 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.534+0000 7f224e600700 1 -- 172.21.15.31:0/1871790332 shutdown_connections 2024-04-25T01:04:50.191 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:49.534+0000 7f224e600700 1 -- 172.21.15.31:0/1871790332 wait complete. 2024-04-25T01:04:50.191 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for the mgr to restart... 2024-04-25T01:04:50.191 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr epoch 5... 2024-04-25T01:04:50.982 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:50 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:50.777+0000 7f893592c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T01:04:51.266 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:51 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:51.012+0000 7f893592c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T01:04:51.266 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:51 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:51.230+0000 7f893592c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T01:04:51.749 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:51 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:51.625+0000 7f893592c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T01:04:51.749 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:51 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:51.751+0000 7f893592c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T01:04:52.265 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:52 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:52.010+0000 7f893592c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T01:04:52.765 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:52 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:52.275+0000 7f893592c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:04:53.515 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:53 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:53.087+0000 7f893592c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T01:04:53.786 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:53 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:53.663+0000 7f893592c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T01:04:54.079 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:53 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:53.798+0000 7f893592c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T01:04:54.079 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:54 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:54.081+0000 7f893592c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T01:04:55.016 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:54 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:54.755+0000 7f893592c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T01:04:55.016 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:54 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:54.883+0000 7f893592c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T01:04:55.402 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:55 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:55.218+0000 7f893592c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T01:04:55.692 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:55 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:55.404+0000 7f893592c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T01:04:55.693 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:55 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:55.534+0000 7f893592c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T01:04:55.693 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:55 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:55.694+0000 7f893592c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T01:04:56.015 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:55 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:55.821+0000 7f893592c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T01:04:57.142 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:56 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:56.854+0000 7f893592c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T01:04:57.142 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:04:57 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:04:57.144+0000 7f893592c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T01:04:57.517 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:57 smithi031 ceph-mon[28053]: Active manager daemon x restarted 2024-04-25T01:04:57.517 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:57 smithi031 ceph-mon[28053]: Activating manager daemon x 2024-04-25T01:04:57.517 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:57 smithi031 ceph-mon[28053]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T01:04:57.517 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:57 smithi031 ceph-mon[28053]: mgrmap e6: x(active, starting, since 0.00889004s) 2024-04-25T01:04:57.517 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:57 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:04:57.517 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:57 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T01:04:57.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:57 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T01:04:57.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:57 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T01:04:57.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:57 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T01:04:57.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:57 smithi031 ceph-mon[28053]: Manager daemon x is now available 2024-04-25T01:04:57.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:57 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' 2024-04-25T01:04:58.503 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:58 smithi031 ceph-mon[28053]: Found migration_current of "None". Setting to last migration. 2024-04-25T01:04:58.503 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:58 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' 2024-04-25T01:04:58.503 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:58 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:04:58.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:58 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:04:58.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:58 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T01:04:58.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:58 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T01:04:58.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:58 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' 2024-04-25T01:04:58.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:58 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' 2024-04-25T01:04:58.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:58 smithi031 ceph-mon[28053]: mgrmap e7: x(active, since 1.01597s) 2024-04-25T01:04:58.865 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-25T01:04:58.873 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T01:04:58.873 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T01:04:58.873 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-25T01:04:58.873 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.775+0000 7f1a0aa00700 1 Processor -- start 2024-04-25T01:04:58.873 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.775+0000 7f1a0aa00700 1 -- start start 2024-04-25T01:04:58.873 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.776+0000 7f1a0aa00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a04107d20 0x7f1a04108140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:58.873 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.776+0000 7f1a0aa00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1a04108710 con 0x7f1a04107d20 2024-04-25T01:04:58.873 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.776+0000 7f1a09600700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a04107d20 0x7f1a04108140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:58.873 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.776+0000 7f1a09600700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a04107d20 0x7f1a04108140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33320/0 (socket says 172.21.15.31:33320) 2024-04-25T01:04:58.873 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.776+0000 7f1a09600700 1 -- 172.21.15.31:0/3688944823 learned_addr learned my addr 172.21.15.31:0/3688944823 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:04:58.874 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.777+0000 7f1a09600700 1 -- 172.21.15.31:0/3688944823 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a04108e80 con 0x7f1a04107d20 2024-04-25T01:04:58.874 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.777+0000 7f1a09600700 1 --2- 172.21.15.31:0/3688944823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a04107d20 0x7f1a04108140 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f19f80098a0 tx=0x7f19f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3636273964ae38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:58.874 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.777+0000 7f1a03e00700 1 -- 172.21.15.31:0/3688944823 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19f8004160 con 0x7f1a04107d20 2024-04-25T01:04:58.874 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.777+0000 7f1a03e00700 1 -- 172.21.15.31:0/3688944823 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f19f8004300 con 0x7f1a04107d20 2024-04-25T01:04:58.874 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.777+0000 7f1a03e00700 1 -- 172.21.15.31:0/3688944823 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19f8011b50 con 0x7f1a04107d20 2024-04-25T01:04:58.874 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.778+0000 7f1a0aa00700 1 -- 172.21.15.31:0/3688944823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a04107d20 msgr2=0x7f1a04108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:58.874 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.778+0000 7f1a0aa00700 1 --2- 172.21.15.31:0/3688944823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a04107d20 0x7f1a04108140 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f19f80098a0 tx=0x7f19f8009bb0 comp rx=0 tx=0).stop 2024-04-25T01:04:58.874 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.778+0000 7f1a0aa00700 1 -- 172.21.15.31:0/3688944823 shutdown_connections 2024-04-25T01:04:58.874 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.778+0000 7f1a0aa00700 1 --2- 172.21.15.31:0/3688944823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a04107d20 0x7f1a04108140 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:58.874 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.778+0000 7f1a0aa00700 1 -- 172.21.15.31:0/3688944823 >> 172.21.15.31:0/3688944823 conn(0x7f1a041032e0 msgr2=0x7f1a04105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:04:58.874 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.779+0000 7f1a0aa00700 1 -- 172.21.15.31:0/3688944823 shutdown_connections 2024-04-25T01:04:58.874 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.779+0000 7f1a0aa00700 1 -- 172.21.15.31:0/3688944823 wait complete. 2024-04-25T01:04:58.874 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.780+0000 7f1a0aa00700 1 Processor -- start 2024-04-25T01:04:58.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.780+0000 7f1a0aa00700 1 -- start start 2024-04-25T01:04:58.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.780+0000 7f1a0aa00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a04107d20 0x7f1a0419de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:58.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.780+0000 7f1a0aa00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0410c230 con 0x7f1a04107d20 2024-04-25T01:04:58.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.780+0000 7f1a09600700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a04107d20 0x7f1a0419de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:58.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.780+0000 7f1a09600700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a04107d20 0x7f1a0419de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33322/0 (socket says 172.21.15.31:33322) 2024-04-25T01:04:58.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.780+0000 7f1a09600700 1 -- 172.21.15.31:0/1068957765 learned_addr learned my addr 172.21.15.31:0/1068957765 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:04:58.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.781+0000 7f1a09600700 1 -- 172.21.15.31:0/1068957765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0419e3b0 con 0x7f1a04107d20 2024-04-25T01:04:58.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.781+0000 7f1a09600700 1 --2- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a04107d20 0x7f1a0419de70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f19f8011850 tx=0x7f19f8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:58.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.781+0000 7f1a02000700 1 -- 172.21.15.31:0/1068957765 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19f8003af0 con 0x7f1a04107d20 2024-04-25T01:04:58.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.781+0000 7f1a02000700 1 -- 172.21.15.31:0/1068957765 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f19f8020070 con 0x7f1a04107d20 2024-04-25T01:04:58.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.781+0000 7f1a0aa00700 1 -- 172.21.15.31:0/1068957765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a0419e640 con 0x7f1a04107d20 2024-04-25T01:04:58.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.781+0000 7f1a02000700 1 -- 172.21.15.31:0/1068957765 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19f80195a0 con 0x7f1a04107d20 2024-04-25T01:04:58.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.781+0000 7f1a0aa00700 1 -- 172.21.15.31:0/1068957765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a041a1330 con 0x7f1a04107d20 2024-04-25T01:04:58.875 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.782+0000 7f1a02000700 1 -- 172.21.15.31:0/1068957765 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f19f801a430 con 0x7f1a04107d20 2024-04-25T01:04:58.876 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.783+0000 7f1a02000700 1 --2- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f19ec0390a0 0x7f19ec03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:58.876 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.783+0000 7f1a02000700 1 -- 172.21.15.31:0/1068957765 --> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f19ec03bc70 con 0x7f19ec0390a0 2024-04-25T01:04:58.876 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.783+0000 7f1a02000700 1 -- 172.21.15.31:0/1068957765 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f19f8011e00 con 0x7f1a04107d20 2024-04-25T01:04:58.876 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.783+0000 7f1a08c00700 1 -- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f19ec0390a0 msgr2=0x7f19ec03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2638309002 2024-04-25T01:04:58.876 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.783+0000 7f1a08c00700 1 --2- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f19ec0390a0 0x7f19ec03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:04:58.876 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.984+0000 7f1a08c00700 1 -- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f19ec0390a0 msgr2=0x7f19ec03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2638309002 2024-04-25T01:04:58.876 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:50.984+0000 7f1a08c00700 1 --2- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f19ec0390a0 0x7f19ec03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:04:58.876 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:51.385+0000 7f1a08c00700 1 -- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f19ec0390a0 msgr2=0x7f19ec03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2638309002 2024-04-25T01:04:58.876 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:51.385+0000 7f1a08c00700 1 --2- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f19ec0390a0 0x7f19ec03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:04:58.876 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:52.186+0000 7f1a08c00700 1 -- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f19ec0390a0 msgr2=0x7f19ec03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2638309002 2024-04-25T01:04:58.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:52.186+0000 7f1a08c00700 1 --2- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f19ec0390a0 0x7f19ec03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:04:58.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:53.788+0000 7f1a08c00700 1 -- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f19ec0390a0 msgr2=0x7f19ec03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2638309002 2024-04-25T01:04:58.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:53.788+0000 7f1a08c00700 1 --2- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f19ec0390a0 0x7f19ec03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T01:04:58.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:56.991+0000 7f1a08c00700 1 -- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f19ec0390a0 msgr2=0x7f19ec03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2638309002 2024-04-25T01:04:58.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:56.991+0000 7f1a08c00700 1 --2- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f19ec0390a0 0x7f19ec03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-04-25T01:04:58.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:57.154+0000 7f1a02000700 1 -- 172.21.15.31:0/1068957765 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f19f801a770 con 0x7f1a04107d20 2024-04-25T01:04:58.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:57.154+0000 7f1a02000700 1 -- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f19ec0390a0 msgr2=0x7f19ec03b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:04:58.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:57.154+0000 7f1a02000700 1 --2- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f19ec0390a0 0x7f19ec03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:58.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:58.158+0000 7f1a02000700 1 -- 172.21.15.31:0/1068957765 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f19f804e920 con 0x7f1a04107d20 2024-04-25T01:04:58.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:58.158+0000 7f1a02000700 1 --2- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f19ec03c9d0 0x7f19ec03edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:04:58.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:58.158+0000 7f1a02000700 1 -- 172.21.15.31:0/1068957765 --> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f19f8001d30 con 0x7f19ec03c9d0 2024-04-25T01:04:58.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:58.160+0000 7f1a08c00700 1 --2- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f19ec03c9d0 0x7f19ec03edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:04:58.877 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:58.163+0000 7f1a08c00700 1 --2- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f19ec03c9d0 0x7f19ec03edc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f19f0003a10 tx=0x7f19f00092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:04:58.878 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:58.165+0000 7f1a02000700 1 -- 172.21.15.31:0/1068957765 <== mgr.14120 v2:172.21.15.31:6800/1901973432 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f19f8001d30 con 0x7f19ec03c9d0 2024-04-25T01:04:58.878 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:58.172+0000 7f19f7400700 1 -- 172.21.15.31:0/1068957765 --> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f19e0002600 con 0x7f19ec03c9d0 2024-04-25T01:04:58.878 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:58.173+0000 7f1a02000700 1 -- 172.21.15.31:0/1068957765 <== mgr.14120 v2:172.21.15.31:6800/1901973432 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f19e0002600 con 0x7f19ec03c9d0 2024-04-25T01:04:58.878 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:58.173+0000 7f19f7400700 1 -- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f19ec03c9d0 msgr2=0x7f19ec03edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:58.878 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:58.173+0000 7f19f7400700 1 --2- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f19ec03c9d0 0x7f19ec03edc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f19f0003a10 tx=0x7f19f00092b0 comp rx=0 tx=0).stop 2024-04-25T01:04:58.878 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:58.173+0000 7f19f7400700 1 -- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a04107d20 msgr2=0x7f1a0419de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:04:58.878 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:58.173+0000 7f19f7400700 1 --2- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a04107d20 0x7f1a0419de70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f19f8011850 tx=0x7f19f8011c10 comp rx=0 tx=0).stop 2024-04-25T01:04:58.878 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:58.174+0000 7f19f7400700 1 -- 172.21.15.31:0/1068957765 shutdown_connections 2024-04-25T01:04:58.878 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:58.174+0000 7f19f7400700 1 --2- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f19ec03c9d0 0x7f19ec03edc0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:58.878 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:58.174+0000 7f19f7400700 1 --2- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a04107d20 0x7f1a0419de70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:58.879 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:58.174+0000 7f19f7400700 1 --2- 172.21.15.31:0/1068957765 >> [v2:172.21.15.31:6800/2638309002,v1:172.21.15.31:6801/2638309002] conn(0x7f19ec0390a0 0x7f19ec03b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:04:58.879 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:58.174+0000 7f19f7400700 1 -- 172.21.15.31:0/1068957765 >> 172.21.15.31:0/1068957765 conn(0x7f1a041032e0 msgr2=0x7f1a04103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:04:58.879 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:58.174+0000 7f19f7400700 1 -- 172.21.15.31:0/1068957765 shutdown_connections 2024-04-25T01:04:58.879 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:58.174+0000 7f19f7400700 1 -- 172.21.15.31:0/1068957765 wait complete. 2024-04-25T01:04:58.879 INFO:teuthology.orchestra.run.smithi031.stdout:mgr epoch 5 is available 2024-04-25T01:04:58.880 INFO:teuthology.orchestra.run.smithi031.stdout:Setting orchestrator backend to cephadm... 2024-04-25T01:04:59.362 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:59 smithi031 ceph-mon[28053]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T01:04:59.362 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:59 smithi031 ceph-mon[28053]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T01:04:59.362 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:59 smithi031 ceph-mon[28053]: [25/Apr/2024:01:04:58] ENGINE Bus STARTING 2024-04-25T01:04:59.362 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:59 smithi031 ceph-mon[28053]: [25/Apr/2024:01:04:58] ENGINE Serving on https://172.21.15.31:7150 2024-04-25T01:04:59.363 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:59 smithi031 ceph-mon[28053]: [25/Apr/2024:01:04:58] ENGINE Serving on http://172.21.15.31:8765 2024-04-25T01:04:59.363 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:59 smithi031 ceph-mon[28053]: [25/Apr/2024:01:04:58] ENGINE Bus STARTED 2024-04-25T01:04:59.363 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:04:59 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:05:00.481 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:00 smithi031 ceph-mon[28053]: mgrmap e8: x(active, since 2s) 2024-04-25T01:05:00.481 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:00 smithi031 ceph-mon[28053]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:05:00.481 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:00 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' 2024-04-25T01:05:00.481 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:00 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:05:00.501 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.535+0000 7f53c2e00700 1 Processor -- start 2024-04-25T01:05:00.507 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.535+0000 7f53c2e00700 1 -- start start 2024-04-25T01:05:00.507 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.535+0000 7f53c2e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53bc10a150 0x7f53bc10a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:00.507 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.536+0000 7f53c2e00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc10ab40 con 0x7f53bc10a150 2024-04-25T01:05:00.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.536+0000 7f53bbe00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53bc10a150 0x7f53bc10a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:00.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.536+0000 7f53bbe00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53bc10a150 0x7f53bc10a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42878/0 (socket says 172.21.15.31:42878) 2024-04-25T01:05:00.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.536+0000 7f53bbe00700 1 -- 172.21.15.31:0/208146288 learned_addr learned my addr 172.21.15.31:0/208146288 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:00.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.536+0000 7f53bbe00700 1 -- 172.21.15.31:0/208146288 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53bc10b220 con 0x7f53bc10a150 2024-04-25T01:05:00.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.537+0000 7f53bbe00700 1 --2- 172.21.15.31:0/208146288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53bc10a150 0x7f53bc10a570 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0098a0 tx=0x7f53ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf4cbc9830d60e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:00.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.537+0000 7f53bb400700 1 -- 172.21.15.31:0/208146288 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53ac004160 con 0x7f53bc10a150 2024-04-25T01:05:00.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.537+0000 7f53bb400700 1 -- 172.21.15.31:0/208146288 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f53ac004300 con 0x7f53bc10a150 2024-04-25T01:05:00.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.537+0000 7f53bb400700 1 -- 172.21.15.31:0/208146288 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53ac011b50 con 0x7f53bc10a150 2024-04-25T01:05:00.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.538+0000 7f53c2e00700 1 -- 172.21.15.31:0/208146288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53bc10a150 msgr2=0x7f53bc10a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:00.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.538+0000 7f53c2e00700 1 --2- 172.21.15.31:0/208146288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53bc10a150 0x7f53bc10a570 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0098a0 tx=0x7f53ac009bb0 comp rx=0 tx=0).stop 2024-04-25T01:05:00.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.538+0000 7f53c2e00700 1 -- 172.21.15.31:0/208146288 shutdown_connections 2024-04-25T01:05:00.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.538+0000 7f53c2e00700 1 --2- 172.21.15.31:0/208146288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53bc10a150 0x7f53bc10a570 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:00.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.538+0000 7f53c2e00700 1 -- 172.21.15.31:0/208146288 >> 172.21.15.31:0/208146288 conn(0x7f53bc07d520 msgr2=0x7f53bc07d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:00.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.538+0000 7f53c2e00700 1 -- 172.21.15.31:0/208146288 shutdown_connections 2024-04-25T01:05:00.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.538+0000 7f53c2e00700 1 -- 172.21.15.31:0/208146288 wait complete. 2024-04-25T01:05:00.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.539+0000 7f53c2e00700 1 Processor -- start 2024-04-25T01:05:00.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.540+0000 7f53c2e00700 1 -- start start 2024-04-25T01:05:00.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.540+0000 7f53c2e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53bc10a150 0x7f53bc19dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:00.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.540+0000 7f53c2e00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc10e5d0 con 0x7f53bc10a150 2024-04-25T01:05:00.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.540+0000 7f53bbe00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53bc10a150 0x7f53bc19dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:00.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.540+0000 7f53bbe00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53bc10a150 0x7f53bc19dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42890/0 (socket says 172.21.15.31:42890) 2024-04-25T01:05:00.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.540+0000 7f53bbe00700 1 -- 172.21.15.31:0/1371714477 learned_addr learned my addr 172.21.15.31:0/1371714477 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:00.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.541+0000 7f53bbe00700 1 -- 172.21.15.31:0/1371714477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53bc19e520 con 0x7f53bc10a150 2024-04-25T01:05:00.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.541+0000 7f53bbe00700 1 --2- 172.21.15.31:0/1371714477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53bc10a150 0x7f53bc19dfe0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f53ac011850 tx=0x7f53ac011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:00.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.541+0000 7f53b9600700 1 -- 172.21.15.31:0/1371714477 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53ac003af0 con 0x7f53bc10a150 2024-04-25T01:05:00.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.541+0000 7f53b9600700 1 -- 172.21.15.31:0/1371714477 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f53ac020070 con 0x7f53bc10a150 2024-04-25T01:05:00.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.541+0000 7f53b9600700 1 -- 172.21.15.31:0/1371714477 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53ac0195a0 con 0x7f53bc10a150 2024-04-25T01:05:00.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.541+0000 7f53c2e00700 1 -- 172.21.15.31:0/1371714477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53bc19e7b0 con 0x7f53bc10a150 2024-04-25T01:05:00.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.541+0000 7f53c2e00700 1 -- 172.21.15.31:0/1371714477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53bc1a14a0 con 0x7f53bc10a150 2024-04-25T01:05:00.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.543+0000 7f53b9600700 1 -- 172.21.15.31:0/1371714477 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f53ac0198a0 con 0x7f53bc10a150 2024-04-25T01:05:00.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.542+0000 7f53c2e00700 1 -- 172.21.15.31:0/1371714477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53bc063890 con 0x7f53bc10a150 2024-04-25T01:05:00.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.543+0000 7f53b9600700 1 --2- 172.21.15.31:0/1371714477 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f53a0039050 0x7f53a003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:00.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.543+0000 7f53b9600700 1 -- 172.21.15.31:0/1371714477 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f53ac011e00 con 0x7f53bc10a150 2024-04-25T01:05:00.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.543+0000 7f53b3e00700 1 --2- 172.21.15.31:0/1371714477 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f53a0039050 0x7f53a003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:00.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.544+0000 7f53b3e00700 1 --2- 172.21.15.31:0/1371714477 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f53a0039050 0x7f53a003b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f53a4006fd0 tx=0x7f53a4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:00.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.549+0000 7f53b9600700 1 -- 172.21.15.31:0/1371714477 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f53ac00e260 con 0x7f53bc10a150 2024-04-25T01:05:00.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.794+0000 7f53c2e00700 1 -- 172.21.15.31:0/1371714477 --> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f53bc108500 con 0x7f53a0039050 2024-04-25T01:05:00.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.805+0000 7f53b9600700 1 -- 172.21.15.31:0/1371714477 <== mgr.14120 v2:172.21.15.31:6800/1901973432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f53bc108500 con 0x7f53a0039050 2024-04-25T01:05:00.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.808+0000 7f53c2e00700 1 -- 172.21.15.31:0/1371714477 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f53a0039050 msgr2=0x7f53a003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:00.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.809+0000 7f53c2e00700 1 --2- 172.21.15.31:0/1371714477 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f53a0039050 0x7f53a003b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f53a4006fd0 tx=0x7f53a4006e40 comp rx=0 tx=0).stop 2024-04-25T01:05:00.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.809+0000 7f53c2e00700 1 -- 172.21.15.31:0/1371714477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53bc10a150 msgr2=0x7f53bc19dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:00.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.809+0000 7f53c2e00700 1 --2- 172.21.15.31:0/1371714477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53bc10a150 0x7f53bc19dfe0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f53ac011850 tx=0x7f53ac011c10 comp rx=0 tx=0).stop 2024-04-25T01:05:00.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.809+0000 7f53c2e00700 1 -- 172.21.15.31:0/1371714477 shutdown_connections 2024-04-25T01:05:00.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.809+0000 7f53c2e00700 1 --2- 172.21.15.31:0/1371714477 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f53a0039050 0x7f53a003b510 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:00.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.809+0000 7f53c2e00700 1 --2- 172.21.15.31:0/1371714477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53bc10a150 0x7f53bc19dfe0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:00.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.809+0000 7f53c2e00700 1 -- 172.21.15.31:0/1371714477 >> 172.21.15.31:0/1371714477 conn(0x7f53bc07d520 msgr2=0x7f53bc107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:00.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.809+0000 7f53c2e00700 1 -- 172.21.15.31:0/1371714477 shutdown_connections 2024-04-25T01:05:00.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:04:59.809+0000 7f53c2e00700 1 -- 172.21.15.31:0/1371714477 wait complete. 2024-04-25T01:05:01.977 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T01:05:01.977 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.090+0000 7fa1bd600700 1 Processor -- start 2024-04-25T01:05:01.977 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.090+0000 7fa1bd600700 1 -- start start 2024-04-25T01:05:01.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.090+0000 7fa1bd600700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1b807ec40 0x7fa1b807f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:01.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.090+0000 7fa1bd600700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b807f630 con 0x7fa1b807ec40 2024-04-25T01:05:01.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.091+0000 7fa1b7e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1b807ec40 0x7fa1b807f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:01.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.091+0000 7fa1b7e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1b807ec40 0x7fa1b807f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42892/0 (socket says 172.21.15.31:42892) 2024-04-25T01:05:01.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.091+0000 7fa1b7e00700 1 -- 172.21.15.31:0/2994818729 learned_addr learned my addr 172.21.15.31:0/2994818729 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:01.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.092+0000 7fa1b7e00700 1 -- 172.21.15.31:0/2994818729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b807fe40 con 0x7fa1b807ec40 2024-04-25T01:05:01.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.092+0000 7fa1b7e00700 1 --2- 172.21.15.31:0/2994818729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1b807ec40 0x7fa1b807f060 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80098a0 tx=0x7fa1a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fcaf6e44d0416e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:01.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.093+0000 7fa1b6a00700 1 -- 172.21.15.31:0/2994818729 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1a8004160 con 0x7fa1b807ec40 2024-04-25T01:05:01.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.093+0000 7fa1b6a00700 1 -- 172.21.15.31:0/2994818729 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa1a8004300 con 0x7fa1b807ec40 2024-04-25T01:05:01.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.093+0000 7fa1b6a00700 1 -- 172.21.15.31:0/2994818729 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1a8011b50 con 0x7fa1b807ec40 2024-04-25T01:05:01.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.094+0000 7fa1bd600700 1 -- 172.21.15.31:0/2994818729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1b807ec40 msgr2=0x7fa1b807f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:01.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.094+0000 7fa1bd600700 1 --2- 172.21.15.31:0/2994818729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1b807ec40 0x7fa1b807f060 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80098a0 tx=0x7fa1a8009bb0 comp rx=0 tx=0).stop 2024-04-25T01:05:01.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.094+0000 7fa1bd600700 1 -- 172.21.15.31:0/2994818729 shutdown_connections 2024-04-25T01:05:01.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.094+0000 7fa1bd600700 1 --2- 172.21.15.31:0/2994818729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1b807ec40 0x7fa1b807f060 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:01.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.094+0000 7fa1bd600700 1 -- 172.21.15.31:0/2994818729 >> 172.21.15.31:0/2994818729 conn(0x7fa1b807d360 msgr2=0x7fa1b807d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:01.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.094+0000 7fa1bd600700 1 -- 172.21.15.31:0/2994818729 shutdown_connections 2024-04-25T01:05:01.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.094+0000 7fa1bd600700 1 -- 172.21.15.31:0/2994818729 wait complete. 2024-04-25T01:05:01.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.095+0000 7fa1bd600700 1 Processor -- start 2024-04-25T01:05:01.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.095+0000 7fa1bd600700 1 -- start start 2024-04-25T01:05:01.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.095+0000 7fa1bd600700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1b807ec40 0x7fa1b81a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:01.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.095+0000 7fa1bd600700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b8081200 con 0x7fa1b807ec40 2024-04-25T01:05:01.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.096+0000 7fa1b7e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1b807ec40 0x7fa1b81a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:01.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.096+0000 7fa1b7e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1b807ec40 0x7fa1b81a2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42906/0 (socket says 172.21.15.31:42906) 2024-04-25T01:05:01.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.096+0000 7fa1b7e00700 1 -- 172.21.15.31:0/135915090 learned_addr learned my addr 172.21.15.31:0/135915090 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:01.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.096+0000 7fa1b7e00700 1 -- 172.21.15.31:0/135915090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b81a26c0 con 0x7fa1b807ec40 2024-04-25T01:05:01.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.096+0000 7fa1b7e00700 1 --2- 172.21.15.31:0/135915090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1b807ec40 0x7fa1b81a2180 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8011b60 tx=0x7fa1a8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:01.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.097+0000 7fa1b4c00700 1 -- 172.21.15.31:0/135915090 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1a8003b20 con 0x7fa1b807ec40 2024-04-25T01:05:01.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.097+0000 7fa1b4c00700 1 -- 172.21.15.31:0/135915090 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa1a8003cc0 con 0x7fa1b807ec40 2024-04-25T01:05:01.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.097+0000 7fa1bd600700 1 -- 172.21.15.31:0/135915090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b81a2950 con 0x7fa1b807ec40 2024-04-25T01:05:01.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.097+0000 7fa1b4c00700 1 -- 172.21.15.31:0/135915090 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1a8024840 con 0x7fa1b807ec40 2024-04-25T01:05:01.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.097+0000 7fa1bd600700 1 -- 172.21.15.31:0/135915090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b81a5640 con 0x7fa1b807ec40 2024-04-25T01:05:01.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.098+0000 7fa1bd600700 1 -- 172.21.15.31:0/135915090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa198005360 con 0x7fa1b807ec40 2024-04-25T01:05:01.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.098+0000 7fa1b4c00700 1 -- 172.21.15.31:0/135915090 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa1a8022020 con 0x7fa1b807ec40 2024-04-25T01:05:01.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.099+0000 7fa1b4c00700 1 --2- 172.21.15.31:0/135915090 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7fa19c0390a0 0x7fa19c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:01.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.099+0000 7fa1b4c00700 1 -- 172.21.15.31:0/135915090 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa1a8011e00 con 0x7fa1b807ec40 2024-04-25T01:05:01.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.099+0000 7fa1b7400700 1 --2- 172.21.15.31:0/135915090 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7fa19c0390a0 0x7fa19c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:01.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.100+0000 7fa1b7400700 1 --2- 172.21.15.31:0/135915090 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7fa19c0390a0 0x7fa19c03b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4006fd0 tx=0x7fa1a4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:01.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.105+0000 7fa1b4c00700 1 -- 172.21.15.31:0/135915090 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa1a80172b0 con 0x7fa1b807ec40 2024-04-25T01:05:01.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.334+0000 7fa1bd600700 1 -- 172.21.15.31:0/135915090 --> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fa198000bf0 con 0x7fa19c0390a0 2024-04-25T01:05:01.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.336+0000 7fa1b4c00700 1 -- 172.21.15.31:0/135915090 <== mgr.14120 v2:172.21.15.31:6800/1901973432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fa198000bf0 con 0x7fa19c0390a0 2024-04-25T01:05:01.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.339+0000 7fa1bd600700 1 -- 172.21.15.31:0/135915090 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7fa19c0390a0 msgr2=0x7fa19c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:01.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.339+0000 7fa1bd600700 1 --2- 172.21.15.31:0/135915090 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7fa19c0390a0 0x7fa19c03b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4006fd0 tx=0x7fa1a4006e40 comp rx=0 tx=0).stop 2024-04-25T01:05:01.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.339+0000 7fa1bd600700 1 -- 172.21.15.31:0/135915090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1b807ec40 msgr2=0x7fa1b81a2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:01.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.339+0000 7fa1bd600700 1 --2- 172.21.15.31:0/135915090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1b807ec40 0x7fa1b81a2180 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8011b60 tx=0x7fa1a8011c40 comp rx=0 tx=0).stop 2024-04-25T01:05:01.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.340+0000 7fa1bd600700 1 -- 172.21.15.31:0/135915090 shutdown_connections 2024-04-25T01:05:01.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.340+0000 7fa1bd600700 1 --2- 172.21.15.31:0/135915090 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7fa19c0390a0 0x7fa19c03b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:01.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.340+0000 7fa1bd600700 1 --2- 172.21.15.31:0/135915090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1b807ec40 0x7fa1b81a2180 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:01.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.340+0000 7fa1bd600700 1 -- 172.21.15.31:0/135915090 >> 172.21.15.31:0/135915090 conn(0x7fa1b807d360 msgr2=0x7fa1b8108850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:01.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.340+0000 7fa1bd600700 1 -- 172.21.15.31:0/135915090 shutdown_connections 2024-04-25T01:05:01.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:01.340+0000 7fa1bd600700 1 -- 172.21.15.31:0/135915090 wait complete. 2024-04-25T01:05:01.986 INFO:teuthology.orchestra.run.smithi031.stdout:Generating ssh key... 2024-04-25T01:05:02.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:02 smithi031 ceph-mon[28053]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:05:03.381 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: Generating public/private rsa key pair. 2024-04-25T01:05:03.381 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: Your identification has been saved in /tmp/tmpw8c55d1p/key. 2024-04-25T01:05:03.381 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: Your public key has been saved in /tmp/tmpw8c55d1p/key.pub. 2024-04-25T01:05:03.381 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: The key fingerprint is: 2024-04-25T01:05:03.381 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: SHA256:vYfZHpbKQbGgEhsib1xy7oHKIZf33VyRnqwTktDY+fY ceph-95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:05:03.381 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: The key's randomart image is: 2024-04-25T01:05:03.381 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: +---[RSA 3072]----+ 2024-04-25T01:05:03.381 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: | | 2024-04-25T01:05:03.381 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: | + . . | 2024-04-25T01:05:03.381 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: |. o = o = . o | 2024-04-25T01:05:03.381 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: | + O + o = = o | 2024-04-25T01:05:03.381 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: |..B * . S B = | 2024-04-25T01:05:03.382 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: |o+.o + . * X . | 2024-04-25T01:05:03.382 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: |.. . . . X E | 2024-04-25T01:05:03.382 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: | . B . | 2024-04-25T01:05:03.382 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: | o . | 2024-04-25T01:05:03.382 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: +----[SHA256]-----+ 2024-04-25T01:05:03.632 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.603+0000 7fdcc7a00700 1 Processor -- start 2024-04-25T01:05:03.633 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.603+0000 7fdcc7a00700 1 -- start start 2024-04-25T01:05:03.633 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.604+0000 7fdcc7a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcc010a150 0x7fdcc010a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:03.633 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.604+0000 7fdcc7a00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc010ab40 con 0x7fdcc010a150 2024-04-25T01:05:03.633 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.604+0000 7fdcc5000700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcc010a150 0x7fdcc010a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:03.634 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.605+0000 7fdcc5000700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcc010a150 0x7fdcc010a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42912/0 (socket says 172.21.15.31:42912) 2024-04-25T01:05:03.634 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.605+0000 7fdcc5000700 1 -- 172.21.15.31:0/1619958361 learned_addr learned my addr 172.21.15.31:0/1619958361 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:03.634 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.605+0000 7fdcc5000700 1 -- 172.21.15.31:0/1619958361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc010b220 con 0x7fdcc010a150 2024-04-25T01:05:03.634 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.605+0000 7fdcc5000700 1 --2- 172.21.15.31:0/1619958361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcc010a150 0x7fdcc010a570 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40098a0 tx=0x7fdcb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33a124506255bfa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:03.634 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.606+0000 7fdcbf400700 1 -- 172.21.15.31:0/1619958361 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdcb4004160 con 0x7fdcc010a150 2024-04-25T01:05:03.634 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.606+0000 7fdcbf400700 1 -- 172.21.15.31:0/1619958361 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdcb4004300 con 0x7fdcc010a150 2024-04-25T01:05:03.634 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.606+0000 7fdcbf400700 1 -- 172.21.15.31:0/1619958361 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdcb4011b50 con 0x7fdcc010a150 2024-04-25T01:05:03.637 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.606+0000 7fdcc7a00700 1 -- 172.21.15.31:0/1619958361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcc010a150 msgr2=0x7fdcc010a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:03.637 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.606+0000 7fdcc7a00700 1 --2- 172.21.15.31:0/1619958361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcc010a150 0x7fdcc010a570 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40098a0 tx=0x7fdcb4009bb0 comp rx=0 tx=0).stop 2024-04-25T01:05:03.637 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.606+0000 7fdcc7a00700 1 -- 172.21.15.31:0/1619958361 shutdown_connections 2024-04-25T01:05:03.637 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.606+0000 7fdcc7a00700 1 --2- 172.21.15.31:0/1619958361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcc010a150 0x7fdcc010a570 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:03.637 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.606+0000 7fdcc7a00700 1 -- 172.21.15.31:0/1619958361 >> 172.21.15.31:0/1619958361 conn(0x7fdcc007d520 msgr2=0x7fdcc007d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:03.637 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.607+0000 7fdcc7a00700 1 -- 172.21.15.31:0/1619958361 shutdown_connections 2024-04-25T01:05:03.637 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.607+0000 7fdcc7a00700 1 -- 172.21.15.31:0/1619958361 wait complete. 2024-04-25T01:05:03.637 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.608+0000 7fdcc7a00700 1 Processor -- start 2024-04-25T01:05:03.637 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.608+0000 7fdcc7a00700 1 -- start start 2024-04-25T01:05:03.637 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.608+0000 7fdcc7a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcc010a150 0x7fdcc019dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:03.637 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.608+0000 7fdcc7a00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc010e5d0 con 0x7fdcc010a150 2024-04-25T01:05:03.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.608+0000 7fdcc5000700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcc010a150 0x7fdcc019dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:03.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.609+0000 7fdcc5000700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcc010a150 0x7fdcc019dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42924/0 (socket says 172.21.15.31:42924) 2024-04-25T01:05:03.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.609+0000 7fdcc5000700 1 -- 172.21.15.31:0/1085542354 learned_addr learned my addr 172.21.15.31:0/1085542354 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:03.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.609+0000 7fdcc5000700 1 -- 172.21.15.31:0/1085542354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc019e520 con 0x7fdcc010a150 2024-04-25T01:05:03.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.609+0000 7fdcc5000700 1 --2- 172.21.15.31:0/1085542354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcc010a150 0x7fdcc019dfe0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4011850 tx=0x7fdcb4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:03.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.609+0000 7fdcbd600700 1 -- 172.21.15.31:0/1085542354 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdcb4003af0 con 0x7fdcc010a150 2024-04-25T01:05:03.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.610+0000 7fdcbd600700 1 -- 172.21.15.31:0/1085542354 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdcb4020070 con 0x7fdcc010a150 2024-04-25T01:05:03.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.610+0000 7fdcc7a00700 1 -- 172.21.15.31:0/1085542354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc019e7b0 con 0x7fdcc010a150 2024-04-25T01:05:03.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.610+0000 7fdcbd600700 1 -- 172.21.15.31:0/1085542354 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdcb40195a0 con 0x7fdcc010a150 2024-04-25T01:05:03.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.610+0000 7fdcc7a00700 1 -- 172.21.15.31:0/1085542354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc01a14a0 con 0x7fdcc010a150 2024-04-25T01:05:03.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.611+0000 7fdcc7a00700 1 -- 172.21.15.31:0/1085542354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc0063890 con 0x7fdcc010a150 2024-04-25T01:05:03.640 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.611+0000 7fdcbd600700 1 -- 172.21.15.31:0/1085542354 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fdcb4019850 con 0x7fdcc010a150 2024-04-25T01:05:03.640 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.612+0000 7fdcbd600700 1 --2- 172.21.15.31:0/1085542354 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7fdca80390a0 0x7fdca803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:03.640 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.612+0000 7fdcbd600700 1 -- 172.21.15.31:0/1085542354 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdcb4011e00 con 0x7fdcc010a150 2024-04-25T01:05:03.641 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.612+0000 7fdcbfe00700 1 --2- 172.21.15.31:0/1085542354 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7fdca80390a0 0x7fdca803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:03.641 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.613+0000 7fdcbfe00700 1 --2- 172.21.15.31:0/1085542354 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7fdca80390a0 0x7fdca803b560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdcac006fd0 tx=0x7fdcac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:03.641 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.617+0000 7fdcbd600700 1 -- 172.21.15.31:0/1085542354 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdcb400e260 con 0x7fdcc010a150 2024-04-25T01:05:03.641 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:02.847+0000 7fdcc7a00700 1 -- 172.21.15.31:0/1085542354 --> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fdcc0108500 con 0x7fdca80390a0 2024-04-25T01:05:03.641 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:03.057+0000 7fdcbd600700 1 -- 172.21.15.31:0/1085542354 <== mgr.14120 v2:172.21.15.31:6800/1901973432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdcc0108500 con 0x7fdca80390a0 2024-04-25T01:05:03.641 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:03.060+0000 7fdcc7a00700 1 -- 172.21.15.31:0/1085542354 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7fdca80390a0 msgr2=0x7fdca803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:03.641 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:03.060+0000 7fdcc7a00700 1 --2- 172.21.15.31:0/1085542354 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7fdca80390a0 0x7fdca803b560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdcac006fd0 tx=0x7fdcac006e40 comp rx=0 tx=0).stop 2024-04-25T01:05:03.641 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:03.060+0000 7fdcc7a00700 1 -- 172.21.15.31:0/1085542354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcc010a150 msgr2=0x7fdcc019dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:03.641 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:03.060+0000 7fdcc7a00700 1 --2- 172.21.15.31:0/1085542354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcc010a150 0x7fdcc019dfe0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4011850 tx=0x7fdcb4011c10 comp rx=0 tx=0).stop 2024-04-25T01:05:03.641 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:03.060+0000 7fdcc7a00700 1 -- 172.21.15.31:0/1085542354 shutdown_connections 2024-04-25T01:05:03.641 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:03.060+0000 7fdcc7a00700 1 --2- 172.21.15.31:0/1085542354 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7fdca80390a0 0x7fdca803b560 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:03.641 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:03.060+0000 7fdcc7a00700 1 --2- 172.21.15.31:0/1085542354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcc010a150 0x7fdcc019dfe0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:03.641 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:03.060+0000 7fdcc7a00700 1 -- 172.21.15.31:0/1085542354 >> 172.21.15.31:0/1085542354 conn(0x7fdcc007d520 msgr2=0x7fdcc0107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:03.641 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:03.060+0000 7fdcc7a00700 1 -- 172.21.15.31:0/1085542354 shutdown_connections 2024-04-25T01:05:03.641 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:03.060+0000 7fdcc7a00700 1 -- 172.21.15.31:0/1085542354 wait complete. 2024-04-25T01:05:04.130 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:04 smithi031 ceph-mon[28053]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:05:04.140 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:04 smithi031 ceph-mon[28053]: Generating ssh key... 2024-04-25T01:05:04.140 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:04 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' 2024-04-25T01:05:04.140 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:04 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' 2024-04-25T01:05:05.227 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:05 smithi031 ceph-mon[28053]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:05:05.310 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQwe7FRAzi79xdQ01F4nxQCtlivRddUKmjPHosaGboRFNlS9nebaurEweJKBtyi5qe/fZR5KWpXGJCpcgNVX+bnzEdMla+AggRzOFUlh9Xcp0GMPSEbkljHUJeoHFjIidYTE7JUtBWpC4tpH/UiyvlYnpQgSZk7R0ud55xE6l3VLS7yV3m4F3mi3uif4YTA5DdUrQzd3qLGpp8oVFJwDtUFI8+MV4yqdX7Hl5eBCLLh63l9LfRkpcvwt6/dz4zj43+KklNSC0t+W47OVLu0Us5dfTCAsfLZR1WrSHtvOchNw9f9daJU8wqSFbE5r1U4B6EpXpYDmIrqU0amsk6yavOJekj6ZjvMmSw4xa2uHKs1d8W6CQc2zNDtW3q9uiwb344On6Ja+Dl2BHGFG9+EAm53KM8HGjJ0/bVitF/hF4wIS1obGiNmXA9AzlK2tj+YlIxe45KrTXcnSfxb4lsmZJmLaxj8CRBxnKXQl4DkZeo1hJ7+0Tvf1AV8T+vMpDsodU= ceph-95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:05:05.310 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.301+0000 7f52c0a00700 1 Processor -- start 2024-04-25T01:05:05.310 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.302+0000 7f52c0a00700 1 -- start start 2024-04-25T01:05:05.310 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.302+0000 7f52c0a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52bc07e640 0x7f52bc07cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:05.311 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.302+0000 7f52c0a00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc07d0f0 con 0x7f52bc07e640 2024-04-25T01:05:05.311 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.302+0000 7f52b9e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52bc07e640 0x7f52bc07cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:05.311 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.302+0000 7f52b9e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52bc07e640 0x7f52bc07cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42928/0 (socket says 172.21.15.31:42928) 2024-04-25T01:05:05.314 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.303+0000 7f52b9e00700 1 -- 172.21.15.31:0/3124658971 learned_addr learned my addr 172.21.15.31:0/3124658971 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:05.314 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.303+0000 7f52b9e00700 1 -- 172.21.15.31:0/3124658971 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52bc07d270 con 0x7f52bc07e640 2024-04-25T01:05:05.314 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.303+0000 7f52b9e00700 1 --2- 172.21.15.31:0/3124658971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52bc07e640 0x7f52bc07cbb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0098a0 tx=0x7f52ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6835c0b12125f94d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:05.314 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.304+0000 7f52b8a00700 1 -- 172.21.15.31:0/3124658971 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52ac004160 con 0x7f52bc07e640 2024-04-25T01:05:05.314 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.304+0000 7f52b8a00700 1 -- 172.21.15.31:0/3124658971 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f52ac004300 con 0x7f52bc07e640 2024-04-25T01:05:05.314 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.304+0000 7f52b8a00700 1 -- 172.21.15.31:0/3124658971 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52ac011b50 con 0x7f52bc07e640 2024-04-25T01:05:05.314 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.304+0000 7f52c0a00700 1 -- 172.21.15.31:0/3124658971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52bc07e640 msgr2=0x7f52bc07cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:05.314 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.304+0000 7f52c0a00700 1 --2- 172.21.15.31:0/3124658971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52bc07e640 0x7f52bc07cbb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0098a0 tx=0x7f52ac009bb0 comp rx=0 tx=0).stop 2024-04-25T01:05:05.314 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.304+0000 7f52c0a00700 1 -- 172.21.15.31:0/3124658971 shutdown_connections 2024-04-25T01:05:05.314 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.304+0000 7f52c0a00700 1 --2- 172.21.15.31:0/3124658971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52bc07e640 0x7f52bc07cbb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:05.315 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.304+0000 7f52c0a00700 1 -- 172.21.15.31:0/3124658971 >> 172.21.15.31:0/3124658971 conn(0x7f52bc1034e0 msgr2=0x7f52bc105900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:05.315 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.305+0000 7f52c0a00700 1 -- 172.21.15.31:0/3124658971 shutdown_connections 2024-04-25T01:05:05.315 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.305+0000 7f52c0a00700 1 -- 172.21.15.31:0/3124658971 wait complete. 2024-04-25T01:05:05.315 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.306+0000 7f52c0a00700 1 Processor -- start 2024-04-25T01:05:05.315 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.306+0000 7f52c0a00700 1 -- start start 2024-04-25T01:05:05.315 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.306+0000 7f52c0a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52bc07e640 0x7f52bc19dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:05.315 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.306+0000 7f52c0a00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc10e2c0 con 0x7f52bc07e640 2024-04-25T01:05:05.315 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.306+0000 7f52b9e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52bc07e640 0x7f52bc19dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:05.315 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.306+0000 7f52b9e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52bc07e640 0x7f52bc19dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42936/0 (socket says 172.21.15.31:42936) 2024-04-25T01:05:05.315 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.306+0000 7f52b9e00700 1 -- 172.21.15.31:0/2717082252 learned_addr learned my addr 172.21.15.31:0/2717082252 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:05.315 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.307+0000 7f52b9e00700 1 -- 172.21.15.31:0/2717082252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52bc19e500 con 0x7f52bc07e640 2024-04-25T01:05:05.315 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.307+0000 7f52b9e00700 1 --2- 172.21.15.31:0/2717082252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52bc07e640 0x7f52bc19dfc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f52ac009fb0 tx=0x7f52ac011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:05.315 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.307+0000 7f52b2a00700 1 -- 172.21.15.31:0/2717082252 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52ac0038b0 con 0x7f52bc07e640 2024-04-25T01:05:05.316 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.307+0000 7f52b2a00700 1 -- 172.21.15.31:0/2717082252 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f52ac011e50 con 0x7f52bc07e640 2024-04-25T01:05:05.316 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.307+0000 7f52b2a00700 1 -- 172.21.15.31:0/2717082252 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52ac019420 con 0x7f52bc07e640 2024-04-25T01:05:05.316 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.307+0000 7f52c0a00700 1 -- 172.21.15.31:0/2717082252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52bc19e790 con 0x7f52bc07e640 2024-04-25T01:05:05.316 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.308+0000 7f52c0a00700 1 -- 172.21.15.31:0/2717082252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52bc1a1480 con 0x7f52bc07e640 2024-04-25T01:05:05.318 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.309+0000 7f52b2a00700 1 -- 172.21.15.31:0/2717082252 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f52ac003a50 con 0x7f52bc07e640 2024-04-25T01:05:05.318 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.309+0000 7f52c0a00700 1 -- 172.21.15.31:0/2717082252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52bc063890 con 0x7f52bc07e640 2024-04-25T01:05:05.318 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.309+0000 7f52b2a00700 1 --2- 172.21.15.31:0/2717082252 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f52a0039050 0x7f52a003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:05.318 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.309+0000 7f52b2a00700 1 -- 172.21.15.31:0/2717082252 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f52ac04d2f0 con 0x7f52bc07e640 2024-04-25T01:05:05.318 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.310+0000 7f52b9400700 1 --2- 172.21.15.31:0/2717082252 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f52a0039050 0x7f52a003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:05.318 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.310+0000 7f52b9400700 1 --2- 172.21.15.31:0/2717082252 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f52a0039050 0x7f52a003b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f52a8006fd0 tx=0x7f52a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:05.318 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.315+0000 7f52b2a00700 1 -- 172.21.15.31:0/2717082252 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f52ac04cc40 con 0x7f52bc07e640 2024-04-25T01:05:05.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.571+0000 7f52c0a00700 1 -- 172.21.15.31:0/2717082252 --> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f52bc104780 con 0x7f52a0039050 2024-04-25T01:05:05.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.572+0000 7f52b2a00700 1 -- 172.21.15.31:0/2717082252 <== mgr.14120 v2:172.21.15.31:6800/1901973432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f52bc104780 con 0x7f52a0039050 2024-04-25T01:05:05.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.575+0000 7f52c0a00700 1 -- 172.21.15.31:0/2717082252 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f52a0039050 msgr2=0x7f52a003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:05.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.575+0000 7f52c0a00700 1 --2- 172.21.15.31:0/2717082252 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f52a0039050 0x7f52a003b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f52a8006fd0 tx=0x7f52a8006e40 comp rx=0 tx=0).stop 2024-04-25T01:05:05.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.575+0000 7f52c0a00700 1 -- 172.21.15.31:0/2717082252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52bc07e640 msgr2=0x7f52bc19dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:05.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.575+0000 7f52c0a00700 1 --2- 172.21.15.31:0/2717082252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52bc07e640 0x7f52bc19dfc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f52ac009fb0 tx=0x7f52ac011bf0 comp rx=0 tx=0).stop 2024-04-25T01:05:05.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.576+0000 7f52c0a00700 1 -- 172.21.15.31:0/2717082252 shutdown_connections 2024-04-25T01:05:05.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.576+0000 7f52c0a00700 1 --2- 172.21.15.31:0/2717082252 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f52a0039050 0x7f52a003b510 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:05.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.576+0000 7f52c0a00700 1 --2- 172.21.15.31:0/2717082252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52bc07e640 0x7f52bc19dfc0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:05.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.576+0000 7f52c0a00700 1 -- 172.21.15.31:0/2717082252 >> 172.21.15.31:0/2717082252 conn(0x7f52bc1034e0 msgr2=0x7f52bc104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:05.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.577+0000 7f52c0a00700 1 -- 172.21.15.31:0/2717082252 shutdown_connections 2024-04-25T01:05:05.319 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:04.577+0000 7f52c0a00700 1 -- 172.21.15.31:0/2717082252 wait complete. 2024-04-25T01:05:05.319 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T01:05:05.319 INFO:teuthology.orchestra.run.smithi031.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T01:05:05.323 INFO:teuthology.orchestra.run.smithi031.stdout:Adding host smithi031... 2024-04-25T01:05:07.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:07 smithi031 ceph-mon[28053]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi031", "addr": "172.21.15.31", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:05:08.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:08 smithi031 ceph-mon[28053]: Deploying cephadm binary to smithi031 2024-04-25T01:05:13.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Added host 'smithi031' with addr '172.21.15.31' 2024-04-25T01:05:13.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.919+0000 7f6d91c00700 1 Processor -- start 2024-04-25T01:05:13.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.919+0000 7f6d91c00700 1 -- start start 2024-04-25T01:05:13.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.920+0000 7f6d91c00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d8c109f20 0x7f6d8c10a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:13.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.920+0000 7f6d91c00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c10a880 con 0x7f6d8c109f20 2024-04-25T01:05:13.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.921+0000 7f6d8be00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d8c109f20 0x7f6d8c10a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:13.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.921+0000 7f6d8be00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d8c109f20 0x7f6d8c10a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42942/0 (socket says 172.21.15.31:42942) 2024-04-25T01:05:13.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.921+0000 7f6d8be00700 1 -- 172.21.15.31:0/3850961078 learned_addr learned my addr 172.21.15.31:0/3850961078 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:13.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.921+0000 7f6d8be00700 1 -- 172.21.15.31:0/3850961078 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8c10afe0 con 0x7f6d8c109f20 2024-04-25T01:05:13.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.921+0000 7f6d8be00700 1 --2- 172.21.15.31:0/3850961078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d8c109f20 0x7f6d8c10a340 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0098a0 tx=0x7f6d7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b8199e3c564ae8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:13.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.922+0000 7f6d8aa00700 1 -- 172.21.15.31:0/3850961078 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d7c004160 con 0x7f6d8c109f20 2024-04-25T01:05:13.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.922+0000 7f6d8aa00700 1 -- 172.21.15.31:0/3850961078 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6d7c004300 con 0x7f6d8c109f20 2024-04-25T01:05:13.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.922+0000 7f6d8aa00700 1 -- 172.21.15.31:0/3850961078 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d7c011b50 con 0x7f6d8c109f20 2024-04-25T01:05:13.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.922+0000 7f6d91c00700 1 -- 172.21.15.31:0/3850961078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d8c109f20 msgr2=0x7f6d8c10a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:13.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.922+0000 7f6d91c00700 1 --2- 172.21.15.31:0/3850961078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d8c109f20 0x7f6d8c10a340 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0098a0 tx=0x7f6d7c009bb0 comp rx=0 tx=0).stop 2024-04-25T01:05:13.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.923+0000 7f6d91c00700 1 -- 172.21.15.31:0/3850961078 shutdown_connections 2024-04-25T01:05:13.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.923+0000 7f6d91c00700 1 --2- 172.21.15.31:0/3850961078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d8c109f20 0x7f6d8c10a340 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:13.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.923+0000 7f6d91c00700 1 -- 172.21.15.31:0/3850961078 >> 172.21.15.31:0/3850961078 conn(0x7f6d8c07d330 msgr2=0x7f6d8c07d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:13.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.923+0000 7f6d91c00700 1 -- 172.21.15.31:0/3850961078 shutdown_connections 2024-04-25T01:05:13.286 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.923+0000 7f6d91c00700 1 -- 172.21.15.31:0/3850961078 wait complete. 2024-04-25T01:05:13.286 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.924+0000 7f6d91c00700 1 Processor -- start 2024-04-25T01:05:13.286 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.924+0000 7f6d91c00700 1 -- start start 2024-04-25T01:05:13.286 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.924+0000 7f6d91c00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d8c109f20 0x7f6d8c19de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:13.286 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.924+0000 7f6d91c00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c10e2a0 con 0x7f6d8c109f20 2024-04-25T01:05:13.286 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.925+0000 7f6d8be00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d8c109f20 0x7f6d8c19de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:13.286 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.925+0000 7f6d8be00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d8c109f20 0x7f6d8c19de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42946/0 (socket says 172.21.15.31:42946) 2024-04-25T01:05:13.286 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.925+0000 7f6d8be00700 1 -- 172.21.15.31:0/76146801 learned_addr learned my addr 172.21.15.31:0/76146801 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:13.286 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.925+0000 7f6d8be00700 1 -- 172.21.15.31:0/76146801 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8c19e360 con 0x7f6d8c109f20 2024-04-25T01:05:13.286 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.925+0000 7f6d8be00700 1 --2- 172.21.15.31:0/76146801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d8c109f20 0x7f6d8c19de20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c011b60 tx=0x7f6d7c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:13.286 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.926+0000 7f6d88c00700 1 -- 172.21.15.31:0/76146801 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d7c003970 con 0x7f6d8c109f20 2024-04-25T01:05:13.287 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.926+0000 7f6d88c00700 1 -- 172.21.15.31:0/76146801 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6d7c003b10 con 0x7f6d8c109f20 2024-04-25T01:05:13.287 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.926+0000 7f6d91c00700 1 -- 172.21.15.31:0/76146801 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d8c19e5f0 con 0x7f6d8c109f20 2024-04-25T01:05:13.287 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.926+0000 7f6d88c00700 1 -- 172.21.15.31:0/76146801 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d7c024610 con 0x7f6d8c109f20 2024-04-25T01:05:13.287 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.926+0000 7f6d91c00700 1 -- 172.21.15.31:0/76146801 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d8c1a12e0 con 0x7f6d8c109f20 2024-04-25T01:05:13.287 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.927+0000 7f6d91c00700 1 -- 172.21.15.31:0/76146801 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d8c10e420 con 0x7f6d8c109f20 2024-04-25T01:05:13.287 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.927+0000 7f6d88c00700 1 -- 172.21.15.31:0/76146801 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f6d7c01a9f0 con 0x7f6d8c109f20 2024-04-25T01:05:13.287 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.928+0000 7f6d88c00700 1 --2- 172.21.15.31:0/76146801 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f6d700390a0 0x7f6d7003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:13.287 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.928+0000 7f6d88c00700 1 -- 172.21.15.31:0/76146801 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6d7c011e00 con 0x7f6d8c109f20 2024-04-25T01:05:13.287 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.928+0000 7f6d8b400700 1 --2- 172.21.15.31:0/76146801 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f6d700390a0 0x7f6d7003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:13.287 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.929+0000 7f6d8b400700 1 --2- 172.21.15.31:0/76146801 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f6d700390a0 0x7f6d7003b560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6d78006fd0 tx=0x7f6d78006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:13.287 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:05.934+0000 7f6d88c00700 1 -- 172.21.15.31:0/76146801 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6d7c04cc10 con 0x7f6d8c109f20 2024-04-25T01:05:13.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:06.172+0000 7f6d91c00700 1 -- 172.21.15.31:0/76146801 --> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi031", "addr": "172.21.15.31", "target": ["mon-mgr", ""]}) -- 0x7f6d8c107c10 con 0x7f6d700390a0 2024-04-25T01:05:13.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:12.595+0000 7f6d88c00700 1 -- 172.21.15.31:0/76146801 <== mgr.14120 v2:172.21.15.31:6800/1901973432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f6d8c107c10 con 0x7f6d700390a0 2024-04-25T01:05:13.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:12.599+0000 7f6d91c00700 1 -- 172.21.15.31:0/76146801 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f6d700390a0 msgr2=0x7f6d7003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:13.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:12.599+0000 7f6d91c00700 1 --2- 172.21.15.31:0/76146801 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f6d700390a0 0x7f6d7003b560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6d78006fd0 tx=0x7f6d78006e40 comp rx=0 tx=0).stop 2024-04-25T01:05:13.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:12.600+0000 7f6d91c00700 1 -- 172.21.15.31:0/76146801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d8c109f20 msgr2=0x7f6d8c19de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:13.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:12.600+0000 7f6d91c00700 1 --2- 172.21.15.31:0/76146801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d8c109f20 0x7f6d8c19de20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c011b60 tx=0x7f6d7c011c40 comp rx=0 tx=0).stop 2024-04-25T01:05:13.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:12.600+0000 7f6d91c00700 1 -- 172.21.15.31:0/76146801 shutdown_connections 2024-04-25T01:05:13.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:12.600+0000 7f6d91c00700 1 --2- 172.21.15.31:0/76146801 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f6d700390a0 0x7f6d7003b560 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:13.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:12.600+0000 7f6d91c00700 1 --2- 172.21.15.31:0/76146801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d8c109f20 0x7f6d8c19de20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:13.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:12.600+0000 7f6d91c00700 1 -- 172.21.15.31:0/76146801 >> 172.21.15.31:0/76146801 conn(0x7f6d8c07d330 msgr2=0x7f6d8c107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:13.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:12.600+0000 7f6d91c00700 1 -- 172.21.15.31:0/76146801 shutdown_connections 2024-04-25T01:05:13.289 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:12.600+0000 7f6d91c00700 1 -- 172.21.15.31:0/76146801 wait complete. 2024-04-25T01:05:13.289 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying unmanaged mon service... 2024-04-25T01:05:13.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:13 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' 2024-04-25T01:05:13.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:13 smithi031 ceph-mon[28053]: Added host smithi031 2024-04-25T01:05:13.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:13 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:05:15.515 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:15 smithi031 ceph-mon[28053]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:05:15.515 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:15 smithi031 ceph-mon[28053]: Saving service mon spec with placement count:5 2024-04-25T01:05:15.515 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:15 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' 2024-04-25T01:05:15.684 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T01:05:15.685 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.963+0000 7fe93b200700 1 Processor -- start 2024-04-25T01:05:15.685 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.963+0000 7fe93b200700 1 -- start start 2024-04-25T01:05:15.685 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.964+0000 7fe93b200700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe93c107cf0 0x7fe93c108110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:15.685 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.964+0000 7fe93b200700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c1086e0 con 0x7fe93c107cf0 2024-04-25T01:05:15.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.964+0000 7fe939e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe93c107cf0 0x7fe93c108110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:15.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.964+0000 7fe939e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe93c107cf0 0x7fe93c108110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45350/0 (socket says 172.21.15.31:45350) 2024-04-25T01:05:15.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.964+0000 7fe939e00700 1 -- 172.21.15.31:0/2030431753 learned_addr learned my addr 172.21.15.31:0/2030431753 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:15.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.965+0000 7fe939e00700 1 -- 172.21.15.31:0/2030431753 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93c108e50 con 0x7fe93c107cf0 2024-04-25T01:05:15.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.965+0000 7fe939e00700 1 --2- 172.21.15.31:0/2030431753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe93c107cf0 0x7fe93c108110 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0098a0 tx=0x7fe92c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628dc50808cfb9a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:15.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.966+0000 7fe938a00700 1 -- 172.21.15.31:0/2030431753 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe92c004160 con 0x7fe93c107cf0 2024-04-25T01:05:15.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.966+0000 7fe938a00700 1 -- 172.21.15.31:0/2030431753 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe92c004300 con 0x7fe93c107cf0 2024-04-25T01:05:15.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.966+0000 7fe938a00700 1 -- 172.21.15.31:0/2030431753 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe92c011b50 con 0x7fe93c107cf0 2024-04-25T01:05:15.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.966+0000 7fe93b200700 1 -- 172.21.15.31:0/2030431753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe93c107cf0 msgr2=0x7fe93c108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:15.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.966+0000 7fe93b200700 1 --2- 172.21.15.31:0/2030431753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe93c107cf0 0x7fe93c108110 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0098a0 tx=0x7fe92c009bb0 comp rx=0 tx=0).stop 2024-04-25T01:05:15.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.967+0000 7fe93b200700 1 -- 172.21.15.31:0/2030431753 shutdown_connections 2024-04-25T01:05:15.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.967+0000 7fe93b200700 1 --2- 172.21.15.31:0/2030431753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe93c107cf0 0x7fe93c108110 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:15.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.967+0000 7fe93b200700 1 -- 172.21.15.31:0/2030431753 >> 172.21.15.31:0/2030431753 conn(0x7fe93c1032f0 msgr2=0x7fe93c105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:15.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.967+0000 7fe93b200700 1 -- 172.21.15.31:0/2030431753 shutdown_connections 2024-04-25T01:05:15.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.967+0000 7fe93b200700 1 -- 172.21.15.31:0/2030431753 wait complete. 2024-04-25T01:05:15.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.968+0000 7fe93b200700 1 Processor -- start 2024-04-25T01:05:15.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.968+0000 7fe93b200700 1 -- start start 2024-04-25T01:05:15.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.969+0000 7fe93b200700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe93c107cf0 0x7fe93c07e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:15.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.969+0000 7fe93b200700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c10c200 con 0x7fe93c107cf0 2024-04-25T01:05:15.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.969+0000 7fe939e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe93c107cf0 0x7fe93c07e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:15.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.969+0000 7fe939e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe93c107cf0 0x7fe93c07e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45356/0 (socket says 172.21.15.31:45356) 2024-04-25T01:05:15.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.969+0000 7fe939e00700 1 -- 172.21.15.31:0/830988539 learned_addr learned my addr 172.21.15.31:0/830988539 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:15.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.969+0000 7fe939e00700 1 -- 172.21.15.31:0/830988539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93c07e940 con 0x7fe93c107cf0 2024-04-25T01:05:15.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.970+0000 7fe939e00700 1 --2- 172.21.15.31:0/830988539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe93c107cf0 0x7fe93c07e400 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe92c000c00 tx=0x7fe92c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:15.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.970+0000 7fe932a00700 1 -- 172.21.15.31:0/830988539 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe92c003b20 con 0x7fe93c107cf0 2024-04-25T01:05:15.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.970+0000 7fe932a00700 1 -- 172.21.15.31:0/830988539 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe92c003cc0 con 0x7fe93c107cf0 2024-04-25T01:05:15.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.970+0000 7fe932a00700 1 -- 172.21.15.31:0/830988539 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe92c024910 con 0x7fe93c107cf0 2024-04-25T01:05:15.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.970+0000 7fe93b200700 1 -- 172.21.15.31:0/830988539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe93c07cb30 con 0x7fe93c107cf0 2024-04-25T01:05:15.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.970+0000 7fe93b200700 1 -- 172.21.15.31:0/830988539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe93c07d070 con 0x7fe93c107cf0 2024-04-25T01:05:15.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.971+0000 7fe93b200700 1 -- 172.21.15.31:0/830988539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe93c063890 con 0x7fe93c107cf0 2024-04-25T01:05:15.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.971+0000 7fe932a00700 1 -- 172.21.15.31:0/830988539 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe92c022020 con 0x7fe93c107cf0 2024-04-25T01:05:15.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.972+0000 7fe932a00700 1 --2- 172.21.15.31:0/830988539 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7fe920039050 0x7fe92003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:15.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.972+0000 7fe932a00700 1 -- 172.21.15.31:0/830988539 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe92c011e00 con 0x7fe93c107cf0 2024-04-25T01:05:15.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.972+0000 7fe939400700 1 --2- 172.21.15.31:0/830988539 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7fe920039050 0x7fe92003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:15.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.973+0000 7fe939400700 1 --2- 172.21.15.31:0/830988539 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7fe920039050 0x7fe92003b510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe928006fd0 tx=0x7fe928006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:15.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:13.977+0000 7fe932a00700 1 -- 172.21.15.31:0/830988539 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe92c00f2e0 con 0x7fe93c107cf0 2024-04-25T01:05:15.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:14.228+0000 7fe93b200700 1 -- 172.21.15.31:0/830988539 --> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fe93c07d8c0 con 0x7fe920039050 2024-04-25T01:05:15.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:14.239+0000 7fe932a00700 1 -- 172.21.15.31:0/830988539 <== mgr.14120 v2:172.21.15.31:6800/1901973432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe93c07d8c0 con 0x7fe920039050 2024-04-25T01:05:15.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:14.242+0000 7fe93b200700 1 -- 172.21.15.31:0/830988539 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7fe920039050 msgr2=0x7fe92003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:15.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:14.242+0000 7fe93b200700 1 --2- 172.21.15.31:0/830988539 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7fe920039050 0x7fe92003b510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe928006fd0 tx=0x7fe928006e40 comp rx=0 tx=0).stop 2024-04-25T01:05:15.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:14.242+0000 7fe93b200700 1 -- 172.21.15.31:0/830988539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe93c107cf0 msgr2=0x7fe93c07e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:15.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:14.242+0000 7fe93b200700 1 --2- 172.21.15.31:0/830988539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe93c107cf0 0x7fe93c07e400 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe92c000c00 tx=0x7fe92c011c10 comp rx=0 tx=0).stop 2024-04-25T01:05:15.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:14.242+0000 7fe93b200700 1 -- 172.21.15.31:0/830988539 shutdown_connections 2024-04-25T01:05:15.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:14.242+0000 7fe93b200700 1 --2- 172.21.15.31:0/830988539 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7fe920039050 0x7fe92003b510 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:15.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:14.242+0000 7fe93b200700 1 --2- 172.21.15.31:0/830988539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe93c107cf0 0x7fe93c07e400 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:15.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:14.242+0000 7fe93b200700 1 -- 172.21.15.31:0/830988539 >> 172.21.15.31:0/830988539 conn(0x7fe93c1032f0 msgr2=0x7fe93c103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:15.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:14.243+0000 7fe93b200700 1 -- 172.21.15.31:0/830988539 shutdown_connections 2024-04-25T01:05:15.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:14.243+0000 7fe93b200700 1 -- 172.21.15.31:0/830988539 wait complete. 2024-04-25T01:05:15.694 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying unmanaged mgr service... 2024-04-25T01:05:17.515 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:17 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' 2024-04-25T01:05:17.515 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:17 smithi031 ceph-mon[28053]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:05:17.515 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:17 smithi031 ceph-mon[28053]: Saving service mgr spec with placement count:2 2024-04-25T01:05:17.515 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:17 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' 2024-04-25T01:05:17.515 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:17 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' 2024-04-25T01:05:17.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T01:05:17.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.554+0000 7f9e4aa00700 1 Processor -- start 2024-04-25T01:05:17.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.555+0000 7f9e4aa00700 1 -- start start 2024-04-25T01:05:17.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.555+0000 7f9e4aa00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e44107f00 0x7f9e44108320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:17.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.555+0000 7f9e4aa00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9e44108860 con 0x7f9e44107f00 2024-04-25T01:05:17.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.556+0000 7f9e43e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e44107f00 0x7f9e44108320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:17.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.556+0000 7f9e43e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e44107f00 0x7f9e44108320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45358/0 (socket says 172.21.15.31:45358) 2024-04-25T01:05:17.533 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.556+0000 7f9e43e00700 1 -- 172.21.15.31:0/2834514102 learned_addr learned my addr 172.21.15.31:0/2834514102 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:17.533 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.556+0000 7f9e43e00700 1 -- 172.21.15.31:0/2834514102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e44108fc0 con 0x7f9e44107f00 2024-04-25T01:05:17.534 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.557+0000 7f9e43e00700 1 --2- 172.21.15.31:0/2834514102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e44107f00 0x7f9e44108320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9e340098a0 tx=0x7f9e34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755b1eac8d0ae3fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:17.534 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.557+0000 7f9e42a00700 1 -- 172.21.15.31:0/2834514102 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e34004160 con 0x7f9e44107f00 2024-04-25T01:05:17.534 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.557+0000 7f9e42a00700 1 -- 172.21.15.31:0/2834514102 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e34004300 con 0x7f9e44107f00 2024-04-25T01:05:17.534 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.557+0000 7f9e42a00700 1 -- 172.21.15.31:0/2834514102 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e34011b20 con 0x7f9e44107f00 2024-04-25T01:05:17.534 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.558+0000 7f9e4aa00700 1 -- 172.21.15.31:0/2834514102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e44107f00 msgr2=0x7f9e44108320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:17.534 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.558+0000 7f9e4aa00700 1 --2- 172.21.15.31:0/2834514102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e44107f00 0x7f9e44108320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9e340098a0 tx=0x7f9e34009bb0 comp rx=0 tx=0).stop 2024-04-25T01:05:17.534 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.558+0000 7f9e4aa00700 1 -- 172.21.15.31:0/2834514102 shutdown_connections 2024-04-25T01:05:17.534 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.558+0000 7f9e4aa00700 1 --2- 172.21.15.31:0/2834514102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e44107f00 0x7f9e44108320 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:17.534 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.558+0000 7f9e4aa00700 1 -- 172.21.15.31:0/2834514102 >> 172.21.15.31:0/2834514102 conn(0x7f9e441034c0 msgr2=0x7f9e441058e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:17.534 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.558+0000 7f9e4aa00700 1 -- 172.21.15.31:0/2834514102 shutdown_connections 2024-04-25T01:05:17.534 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.558+0000 7f9e4aa00700 1 -- 172.21.15.31:0/2834514102 wait complete. 2024-04-25T01:05:17.534 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.559+0000 7f9e4aa00700 1 Processor -- start 2024-04-25T01:05:17.535 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.559+0000 7f9e4aa00700 1 -- start start 2024-04-25T01:05:17.535 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.560+0000 7f9e4aa00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e44107f00 0x7f9e4419df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:17.535 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.560+0000 7f9e4aa00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4410c400 con 0x7f9e44107f00 2024-04-25T01:05:17.535 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.560+0000 7f9e43e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e44107f00 0x7f9e4419df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:17.535 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.560+0000 7f9e43e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e44107f00 0x7f9e4419df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45362/0 (socket says 172.21.15.31:45362) 2024-04-25T01:05:17.535 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.560+0000 7f9e43e00700 1 -- 172.21.15.31:0/88603700 learned_addr learned my addr 172.21.15.31:0/88603700 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:17.535 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.560+0000 7f9e43e00700 1 -- 172.21.15.31:0/88603700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e4419e4b0 con 0x7f9e44107f00 2024-04-25T01:05:17.535 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.560+0000 7f9e43e00700 1 --2- 172.21.15.31:0/88603700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e44107f00 0x7f9e4419df70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9e34011b60 tx=0x7f9e34011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:17.535 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.561+0000 7f9e40c00700 1 -- 172.21.15.31:0/88603700 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e34003b60 con 0x7f9e44107f00 2024-04-25T01:05:17.535 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.561+0000 7f9e40c00700 1 -- 172.21.15.31:0/88603700 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e3401a910 con 0x7f9e44107f00 2024-04-25T01:05:17.535 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.561+0000 7f9e4aa00700 1 -- 172.21.15.31:0/88603700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e4419e740 con 0x7f9e44107f00 2024-04-25T01:05:17.535 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.561+0000 7f9e40c00700 1 -- 172.21.15.31:0/88603700 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e34024930 con 0x7f9e44107f00 2024-04-25T01:05:17.535 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.561+0000 7f9e4aa00700 1 -- 172.21.15.31:0/88603700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e4419eba0 con 0x7f9e44107f00 2024-04-25T01:05:17.536 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.562+0000 7f9e40c00700 1 -- 172.21.15.31:0/88603700 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f9e34022020 con 0x7f9e44107f00 2024-04-25T01:05:17.536 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.563+0000 7f9e4aa00700 1 -- 172.21.15.31:0/88603700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e44063890 con 0x7f9e44107f00 2024-04-25T01:05:17.536 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.563+0000 7f9e40c00700 1 --2- 172.21.15.31:0/88603700 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f9e28039050 0x7f9e2803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:17.536 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.563+0000 7f9e40c00700 1 -- 172.21.15.31:0/88603700 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9e3404d260 con 0x7f9e44107f00 2024-04-25T01:05:17.536 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.563+0000 7f9e43400700 1 --2- 172.21.15.31:0/88603700 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f9e28039050 0x7f9e2803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:17.536 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.564+0000 7f9e43400700 1 --2- 172.21.15.31:0/88603700 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f9e28039050 0x7f9e2803b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9e30006fd0 tx=0x7f9e30006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:17.536 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.569+0000 7f9e40c00700 1 -- 172.21.15.31:0/88603700 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9e34024c50 con 0x7f9e44107f00 2024-04-25T01:05:17.536 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.838+0000 7f9e4aa00700 1 -- 172.21.15.31:0/88603700 --> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f9e441045c0 con 0x7f9e28039050 2024-04-25T01:05:17.536 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.843+0000 7f9e40c00700 1 -- 172.21.15.31:0/88603700 <== mgr.14120 v2:172.21.15.31:6800/1901973432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9e441045c0 con 0x7f9e28039050 2024-04-25T01:05:17.536 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.846+0000 7f9e4aa00700 1 -- 172.21.15.31:0/88603700 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f9e28039050 msgr2=0x7f9e2803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:17.536 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.846+0000 7f9e4aa00700 1 --2- 172.21.15.31:0/88603700 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f9e28039050 0x7f9e2803b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9e30006fd0 tx=0x7f9e30006e40 comp rx=0 tx=0).stop 2024-04-25T01:05:17.536 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.846+0000 7f9e4aa00700 1 -- 172.21.15.31:0/88603700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e44107f00 msgr2=0x7f9e4419df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:17.536 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.846+0000 7f9e4aa00700 1 --2- 172.21.15.31:0/88603700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e44107f00 0x7f9e4419df70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9e34011b60 tx=0x7f9e34011c40 comp rx=0 tx=0).stop 2024-04-25T01:05:17.536 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.847+0000 7f9e4aa00700 1 -- 172.21.15.31:0/88603700 shutdown_connections 2024-04-25T01:05:17.537 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.847+0000 7f9e4aa00700 1 --2- 172.21.15.31:0/88603700 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f9e28039050 0x7f9e2803b510 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:17.537 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.847+0000 7f9e4aa00700 1 --2- 172.21.15.31:0/88603700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e44107f00 0x7f9e4419df70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:17.537 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.847+0000 7f9e4aa00700 1 -- 172.21.15.31:0/88603700 >> 172.21.15.31:0/88603700 conn(0x7f9e441034c0 msgr2=0x7f9e44103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:17.537 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.847+0000 7f9e4aa00700 1 -- 172.21.15.31:0/88603700 shutdown_connections 2024-04-25T01:05:17.537 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:16.847+0000 7f9e4aa00700 1 -- 172.21.15.31:0/88603700 wait complete. 2024-04-25T01:05:18.414 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:18 smithi031 ceph-mon[28053]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:05:19.694 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:19 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2815395260' entity='client.admin' 2024-04-25T01:05:19.705 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.178+0000 7f51a8c00700 1 Processor -- start 2024-04-25T01:05:19.705 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.178+0000 7f51a8c00700 1 -- start start 2024-04-25T01:05:19.706 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.178+0000 7f51a8c00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a410a100 0x7f51a410a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:19.706 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.178+0000 7f51a8c00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f51a410aaf0 con 0x7f51a410a100 2024-04-25T01:05:19.706 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.179+0000 7f51a3400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a410a100 0x7f51a410a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:19.706 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.179+0000 7f51a3400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a410a100 0x7f51a410a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46942/0 (socket says 172.21.15.31:46942) 2024-04-25T01:05:19.706 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.179+0000 7f51a3400700 1 -- 172.21.15.31:0/940012581 learned_addr learned my addr 172.21.15.31:0/940012581 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:19.706 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.179+0000 7f51a3400700 1 -- 172.21.15.31:0/940012581 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a410b020 con 0x7f51a410a100 2024-04-25T01:05:19.706 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.179+0000 7f51a3400700 1 --2- 172.21.15.31:0/940012581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a410a100 0x7f51a410a520 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f51940098a0 tx=0x7f5194009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02b6a2fe3e6a7e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:19.706 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.180+0000 7f51a2000700 1 -- 172.21.15.31:0/940012581 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5194004160 con 0x7f51a410a100 2024-04-25T01:05:19.706 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.180+0000 7f51a2000700 1 -- 172.21.15.31:0/940012581 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5194004300 con 0x7f51a410a100 2024-04-25T01:05:19.706 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.180+0000 7f51a2000700 1 -- 172.21.15.31:0/940012581 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5194011b20 con 0x7f51a410a100 2024-04-25T01:05:19.706 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.180+0000 7f51a8c00700 1 -- 172.21.15.31:0/940012581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a410a100 msgr2=0x7f51a410a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:19.706 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.180+0000 7f51a8c00700 1 --2- 172.21.15.31:0/940012581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a410a100 0x7f51a410a520 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f51940098a0 tx=0x7f5194009bb0 comp rx=0 tx=0).stop 2024-04-25T01:05:19.706 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.181+0000 7f51a8c00700 1 -- 172.21.15.31:0/940012581 shutdown_connections 2024-04-25T01:05:19.707 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.181+0000 7f51a8c00700 1 --2- 172.21.15.31:0/940012581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a410a100 0x7f51a410a520 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:19.707 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.181+0000 7f51a8c00700 1 -- 172.21.15.31:0/940012581 >> 172.21.15.31:0/940012581 conn(0x7f51a407d340 msgr2=0x7f51a407d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:19.707 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.181+0000 7f51a8c00700 1 -- 172.21.15.31:0/940012581 shutdown_connections 2024-04-25T01:05:19.707 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.181+0000 7f51a8c00700 1 -- 172.21.15.31:0/940012581 wait complete. 2024-04-25T01:05:19.707 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.182+0000 7f51a8c00700 1 Processor -- start 2024-04-25T01:05:19.707 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.182+0000 7f51a8c00700 1 -- start start 2024-04-25T01:05:19.707 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.182+0000 7f51a8c00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a410a100 0x7f51a419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:19.707 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.182+0000 7f51a8c00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f51a410e460 con 0x7f51a410a100 2024-04-25T01:05:19.707 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.183+0000 7f51a3400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a410a100 0x7f51a419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:19.707 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.183+0000 7f51a3400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a410a100 0x7f51a419ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46950/0 (socket says 172.21.15.31:46950) 2024-04-25T01:05:19.707 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.183+0000 7f51a3400700 1 -- 172.21.15.31:0/2815395260 learned_addr learned my addr 172.21.15.31:0/2815395260 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:19.707 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.183+0000 7f51a3400700 1 -- 172.21.15.31:0/2815395260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a419e2f0 con 0x7f51a410a100 2024-04-25T01:05:19.710 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.183+0000 7f51a3400700 1 --2- 172.21.15.31:0/2815395260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a410a100 0x7f51a419ddb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5194011b60 tx=0x7f5194011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:19.710 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.184+0000 7f519be00700 1 -- 172.21.15.31:0/2815395260 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5194003b60 con 0x7f51a410a100 2024-04-25T01:05:19.710 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.184+0000 7f519be00700 1 -- 172.21.15.31:0/2815395260 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f519401a910 con 0x7f51a410a100 2024-04-25T01:05:19.710 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.184+0000 7f519be00700 1 -- 172.21.15.31:0/2815395260 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5194024930 con 0x7f51a410a100 2024-04-25T01:05:19.710 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.184+0000 7f51a8c00700 1 -- 172.21.15.31:0/2815395260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a419e580 con 0x7f51a410a100 2024-04-25T01:05:19.711 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.184+0000 7f51a8c00700 1 -- 172.21.15.31:0/2815395260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a419e9e0 con 0x7f51a410a100 2024-04-25T01:05:19.711 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.185+0000 7f51a8c00700 1 -- 172.21.15.31:0/2815395260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a4063890 con 0x7f51a410a100 2024-04-25T01:05:19.711 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.185+0000 7f519be00700 1 -- 172.21.15.31:0/2815395260 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f5194022020 con 0x7f51a410a100 2024-04-25T01:05:19.711 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.186+0000 7f519be00700 1 --2- 172.21.15.31:0/2815395260 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f5188039050 0x7f518803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:19.711 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.186+0000 7f519be00700 1 -- 172.21.15.31:0/2815395260 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f519404d310 con 0x7f51a410a100 2024-04-25T01:05:19.711 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.186+0000 7f51a2a00700 1 --2- 172.21.15.31:0/2815395260 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f5188039050 0x7f518803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:19.711 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.187+0000 7f51a2a00700 1 --2- 172.21.15.31:0/2815395260 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f5188039050 0x7f518803b510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5190006fd0 tx=0x7f5190006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:19.711 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.192+0000 7f519be00700 1 -- 172.21.15.31:0/2815395260 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f519404cc60 con 0x7f51a410a100 2024-04-25T01:05:19.711 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.417+0000 7f51a8c00700 1 -- 172.21.15.31:0/2815395260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f51a41a1270 con 0x7f51a410a100 2024-04-25T01:05:19.711 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.426+0000 7f519be00700 1 -- 172.21.15.31:0/2815395260 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f5194024c50 con 0x7f51a410a100 2024-04-25T01:05:19.711 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.428+0000 7f51a8c00700 1 -- 172.21.15.31:0/2815395260 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f5188039050 msgr2=0x7f518803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:19.711 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.428+0000 7f51a8c00700 1 --2- 172.21.15.31:0/2815395260 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f5188039050 0x7f518803b510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5190006fd0 tx=0x7f5190006e40 comp rx=0 tx=0).stop 2024-04-25T01:05:19.711 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.428+0000 7f51a8c00700 1 -- 172.21.15.31:0/2815395260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a410a100 msgr2=0x7f51a419ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:19.711 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.428+0000 7f51a8c00700 1 --2- 172.21.15.31:0/2815395260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a410a100 0x7f51a419ddb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5194011b60 tx=0x7f5194011c40 comp rx=0 tx=0).stop 2024-04-25T01:05:19.712 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.429+0000 7f51a8c00700 1 -- 172.21.15.31:0/2815395260 shutdown_connections 2024-04-25T01:05:19.712 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.429+0000 7f51a8c00700 1 --2- 172.21.15.31:0/2815395260 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f5188039050 0x7f518803b510 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:19.712 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.429+0000 7f51a8c00700 1 --2- 172.21.15.31:0/2815395260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a410a100 0x7f51a419ddb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:19.712 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.429+0000 7f51a8c00700 1 -- 172.21.15.31:0/2815395260 >> 172.21.15.31:0/2815395260 conn(0x7f51a407d340 msgr2=0x7f51a4107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:19.712 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.429+0000 7f51a8c00700 1 -- 172.21.15.31:0/2815395260 shutdown_connections 2024-04-25T01:05:19.712 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:18.429+0000 7f51a8c00700 1 -- 172.21.15.31:0/2815395260 wait complete. 2024-04-25T01:05:20.560 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:20 smithi031 ceph-mon[28053]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:05:20.561 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:20 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' 2024-04-25T01:05:21.283 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.317+0000 7f67ff000700 1 Processor -- start 2024-04-25T01:05:21.283 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.317+0000 7f67ff000700 1 -- start start 2024-04-25T01:05:21.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.317+0000 7f67ff000700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f8108940 0x7f67f810ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:21.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.317+0000 7f67ff000700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f67f807be30 con 0x7f67f8108940 2024-04-25T01:05:21.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.318+0000 7f67f7e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f8108940 0x7f67f810ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:21.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.318+0000 7f67f7e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f8108940 0x7f67f810ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46966/0 (socket says 172.21.15.31:46966) 2024-04-25T01:05:21.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.318+0000 7f67f7e00700 1 -- 172.21.15.31:0/690852948 learned_addr learned my addr 172.21.15.31:0/690852948 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:21.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.318+0000 7f67f7e00700 1 -- 172.21.15.31:0/690852948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f810b270 con 0x7f67f8108940 2024-04-25T01:05:21.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.319+0000 7f67f7e00700 1 --2- 172.21.15.31:0/690852948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f8108940 0x7f67f810ad30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f67e80098a0 tx=0x7f67e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2212dc5080318705 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:21.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.319+0000 7f67f6a00700 1 -- 172.21.15.31:0/690852948 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67e8004160 con 0x7f67f8108940 2024-04-25T01:05:21.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.319+0000 7f67f6a00700 1 -- 172.21.15.31:0/690852948 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67e8004300 con 0x7f67f8108940 2024-04-25T01:05:21.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.319+0000 7f67f6a00700 1 -- 172.21.15.31:0/690852948 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67e8011b20 con 0x7f67f8108940 2024-04-25T01:05:21.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.320+0000 7f67ff000700 1 -- 172.21.15.31:0/690852948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f8108940 msgr2=0x7f67f810ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:21.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.320+0000 7f67ff000700 1 --2- 172.21.15.31:0/690852948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f8108940 0x7f67f810ad30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f67e80098a0 tx=0x7f67e8009bb0 comp rx=0 tx=0).stop 2024-04-25T01:05:21.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.320+0000 7f67ff000700 1 -- 172.21.15.31:0/690852948 shutdown_connections 2024-04-25T01:05:21.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.320+0000 7f67ff000700 1 --2- 172.21.15.31:0/690852948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f8108940 0x7f67f810ad30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:21.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.320+0000 7f67ff000700 1 -- 172.21.15.31:0/690852948 >> 172.21.15.31:0/690852948 conn(0x7f67f8102330 msgr2=0x7f67f8104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:21.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.320+0000 7f67ff000700 1 -- 172.21.15.31:0/690852948 shutdown_connections 2024-04-25T01:05:21.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.320+0000 7f67ff000700 1 -- 172.21.15.31:0/690852948 wait complete. 2024-04-25T01:05:21.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.321+0000 7f67ff000700 1 Processor -- start 2024-04-25T01:05:21.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.322+0000 7f67ff000700 1 -- start start 2024-04-25T01:05:21.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.322+0000 7f67ff000700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f8108940 0x7f67f81081c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:21.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.322+0000 7f67ff000700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f67f810c270 con 0x7f67f8108940 2024-04-25T01:05:21.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.323+0000 7f67f7e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f8108940 0x7f67f81081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:21.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.323+0000 7f67f7e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f8108940 0x7f67f81081c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46970/0 (socket says 172.21.15.31:46970) 2024-04-25T01:05:21.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.323+0000 7f67f7e00700 1 -- 172.21.15.31:0/3832085845 learned_addr learned my addr 172.21.15.31:0/3832085845 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:21.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.324+0000 7f67f7e00700 1 -- 172.21.15.31:0/3832085845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f8108700 con 0x7f67f8108940 2024-04-25T01:05:21.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.324+0000 7f67f7e00700 1 --2- 172.21.15.31:0/3832085845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f8108940 0x7f67f81081c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f67e8011b60 tx=0x7f67e8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:21.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.324+0000 7f67f4c00700 1 -- 172.21.15.31:0/3832085845 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67e8003b60 con 0x7f67f8108940 2024-04-25T01:05:21.288 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.324+0000 7f67f4c00700 1 -- 172.21.15.31:0/3832085845 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67e801a910 con 0x7f67f8108940 2024-04-25T01:05:21.289 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.324+0000 7f67f4c00700 1 -- 172.21.15.31:0/3832085845 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67e8024930 con 0x7f67f8108940 2024-04-25T01:05:21.289 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.325+0000 7f67ff000700 1 -- 172.21.15.31:0/3832085845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f81068f0 con 0x7f67f8108940 2024-04-25T01:05:21.289 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.325+0000 7f67ff000700 1 -- 172.21.15.31:0/3832085845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f8106db0 con 0x7f67f8108940 2024-04-25T01:05:21.289 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.326+0000 7f67f4c00700 1 -- 172.21.15.31:0/3832085845 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f67e8022020 con 0x7f67f8108940 2024-04-25T01:05:21.289 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.327+0000 7f67f4c00700 1 --2- 172.21.15.31:0/3832085845 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f67dc0390a0 0x7f67dc03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:21.289 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.327+0000 7f67f4c00700 1 -- 172.21.15.31:0/3832085845 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f67e804d300 con 0x7f67f8108940 2024-04-25T01:05:21.289 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.327+0000 7f67ff000700 1 -- 172.21.15.31:0/3832085845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67f8063890 con 0x7f67f8108940 2024-04-25T01:05:21.289 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.328+0000 7f67f7400700 1 --2- 172.21.15.31:0/3832085845 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f67dc0390a0 0x7f67dc03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:21.289 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.329+0000 7f67f7400700 1 --2- 172.21.15.31:0/3832085845 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f67dc0390a0 0x7f67dc03b560 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f67e4006fd0 tx=0x7f67e4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:21.289 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.333+0000 7f67f4c00700 1 -- 172.21.15.31:0/3832085845 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f67e800b3e0 con 0x7f67f8108940 2024-04-25T01:05:21.289 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.564+0000 7f67ff000700 1 -- 172.21.15.31:0/3832085845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f67f8107570 con 0x7f67f8108940 2024-04-25T01:05:21.289 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.570+0000 7f67f4c00700 1 -- 172.21.15.31:0/3832085845 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f67e804cc50 con 0x7f67f8108940 2024-04-25T01:05:21.289 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.573+0000 7f67ff000700 1 -- 172.21.15.31:0/3832085845 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f67dc0390a0 msgr2=0x7f67dc03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:21.290 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.573+0000 7f67ff000700 1 --2- 172.21.15.31:0/3832085845 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f67dc0390a0 0x7f67dc03b560 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f67e4006fd0 tx=0x7f67e4006e40 comp rx=0 tx=0).stop 2024-04-25T01:05:21.290 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.573+0000 7f67ff000700 1 -- 172.21.15.31:0/3832085845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f8108940 msgr2=0x7f67f81081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:21.290 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.573+0000 7f67ff000700 1 --2- 172.21.15.31:0/3832085845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f8108940 0x7f67f81081c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f67e8011b60 tx=0x7f67e8011c40 comp rx=0 tx=0).stop 2024-04-25T01:05:21.290 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.573+0000 7f67ff000700 1 -- 172.21.15.31:0/3832085845 shutdown_connections 2024-04-25T01:05:21.290 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.573+0000 7f67ff000700 1 --2- 172.21.15.31:0/3832085845 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f67dc0390a0 0x7f67dc03b560 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:21.290 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.573+0000 7f67ff000700 1 --2- 172.21.15.31:0/3832085845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f8108940 0x7f67f81081c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:21.290 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.573+0000 7f67ff000700 1 -- 172.21.15.31:0/3832085845 >> 172.21.15.31:0/3832085845 conn(0x7f67f8102330 msgr2=0x7f67f8102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:21.290 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.573+0000 7f67ff000700 1 -- 172.21.15.31:0/3832085845 shutdown_connections 2024-04-25T01:05:21.290 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:20.574+0000 7f67ff000700 1 -- 172.21.15.31:0/3832085845 wait complete. 2024-04-25T01:05:21.290 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling the dashboard module... 2024-04-25T01:05:21.878 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:21 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3832085845' entity='client.admin' 2024-04-25T01:05:22.973 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:22 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: ignoring --setuser ceph since I am not root 2024-04-25T01:05:22.973 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:22 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: ignoring --setgroup ceph since I am not root 2024-04-25T01:05:22.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:22 smithi031 ceph-mon[28053]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:05:22.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:22 smithi031 ceph-mon[28053]: from='mgr.14120 172.21.15.31:0/3147237190' entity='mgr.x' 2024-04-25T01:05:22.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:22 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2945126780' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T01:05:23.265 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:23 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:05:23.005+0000 7fdac5631200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T01:05:23.605 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:23 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:05:23.272+0000 7fdac5631200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:05:23.606 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:23 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:05:23.410+0000 7fdac5631200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T01:05:23.901 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:23 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2945126780' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T01:05:23.901 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:23 smithi031 ceph-mon[28053]: mgrmap e9: x(active, since 25s) 2024-04-25T01:05:23.901 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:23 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:05:23.820+0000 7fdac5631200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T01:05:24.245 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:23 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:05:23.934+0000 7fdac5631200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T01:05:24.256 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.878+0000 7f433c800700 1 Processor -- start 2024-04-25T01:05:24.256 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.878+0000 7f433c800700 1 -- start start 2024-04-25T01:05:24.256 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.878+0000 7f433c800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f433407e6d0 0x7f433407cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:24.256 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.878+0000 7f433c800700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f433407d110 con 0x7f433407e6d0 2024-04-25T01:05:24.256 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.878+0000 7f4339e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f433407e6d0 0x7f433407cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:24.256 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.879+0000 7f4339e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f433407e6d0 0x7f433407cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46972/0 (socket says 172.21.15.31:46972) 2024-04-25T01:05:24.256 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.879+0000 7f4339e00700 1 -- 172.21.15.31:0/944773866 learned_addr learned my addr 172.21.15.31:0/944773866 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:24.256 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.879+0000 7f4339e00700 1 -- 172.21.15.31:0/944773866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433407d290 con 0x7f433407e6d0 2024-04-25T01:05:24.256 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.879+0000 7f4339e00700 1 --2- 172.21.15.31:0/944773866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f433407e6d0 0x7f433407cbd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f43280098a0 tx=0x7f4328009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af4c9e0ce124468 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:24.256 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.880+0000 7f4338a00700 1 -- 172.21.15.31:0/944773866 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4328004160 con 0x7f433407e6d0 2024-04-25T01:05:24.256 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.880+0000 7f4338a00700 1 -- 172.21.15.31:0/944773866 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4328004300 con 0x7f433407e6d0 2024-04-25T01:05:24.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.880+0000 7f4338a00700 1 -- 172.21.15.31:0/944773866 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4328011b20 con 0x7f433407e6d0 2024-04-25T01:05:24.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.880+0000 7f433c800700 1 -- 172.21.15.31:0/944773866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f433407e6d0 msgr2=0x7f433407cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:24.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.880+0000 7f433c800700 1 --2- 172.21.15.31:0/944773866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f433407e6d0 0x7f433407cbd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f43280098a0 tx=0x7f4328009bb0 comp rx=0 tx=0).stop 2024-04-25T01:05:24.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.881+0000 7f433c800700 1 -- 172.21.15.31:0/944773866 shutdown_connections 2024-04-25T01:05:24.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.881+0000 7f433c800700 1 --2- 172.21.15.31:0/944773866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f433407e6d0 0x7f433407cbd0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:24.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.881+0000 7f433c800700 1 -- 172.21.15.31:0/944773866 >> 172.21.15.31:0/944773866 conn(0x7f4334103480 msgr2=0x7f43341058e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:24.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.881+0000 7f433c800700 1 -- 172.21.15.31:0/944773866 shutdown_connections 2024-04-25T01:05:24.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.881+0000 7f433c800700 1 -- 172.21.15.31:0/944773866 wait complete. 2024-04-25T01:05:24.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.882+0000 7f433c800700 1 Processor -- start 2024-04-25T01:05:24.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.882+0000 7f433c800700 1 -- start start 2024-04-25T01:05:24.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.882+0000 7f433c800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f433407e6d0 0x7f43341a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:24.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.882+0000 7f433c800700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f433410a050 con 0x7f433407e6d0 2024-04-25T01:05:24.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.883+0000 7f4339e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f433407e6d0 0x7f43341a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:24.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.883+0000 7f4339e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f433407e6d0 0x7f43341a2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46984/0 (socket says 172.21.15.31:46984) 2024-04-25T01:05:24.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.883+0000 7f4339e00700 1 -- 172.21.15.31:0/2945126780 learned_addr learned my addr 172.21.15.31:0/2945126780 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:24.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.883+0000 7f4339e00700 1 -- 172.21.15.31:0/2945126780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43341a28d0 con 0x7f433407e6d0 2024-04-25T01:05:24.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.883+0000 7f4339e00700 1 --2- 172.21.15.31:0/2945126780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f433407e6d0 0x7f43341a2390 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4328011b60 tx=0x7f4328011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:24.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.884+0000 7f432ea00700 1 -- 172.21.15.31:0/2945126780 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4328003b60 con 0x7f433407e6d0 2024-04-25T01:05:24.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.884+0000 7f432ea00700 1 -- 172.21.15.31:0/2945126780 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f432801a910 con 0x7f433407e6d0 2024-04-25T01:05:24.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.884+0000 7f432ea00700 1 -- 172.21.15.31:0/2945126780 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4328024930 con 0x7f433407e6d0 2024-04-25T01:05:24.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.884+0000 7f433c800700 1 -- 172.21.15.31:0/2945126780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43341a2b60 con 0x7f433407e6d0 2024-04-25T01:05:24.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.884+0000 7f433c800700 1 -- 172.21.15.31:0/2945126780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43341a2fc0 con 0x7f433407e6d0 2024-04-25T01:05:24.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.885+0000 7f432ea00700 1 -- 172.21.15.31:0/2945126780 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f4328022020 con 0x7f433407e6d0 2024-04-25T01:05:24.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.885+0000 7f433c800700 1 -- 172.21.15.31:0/2945126780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4334063890 con 0x7f433407e6d0 2024-04-25T01:05:24.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.886+0000 7f432ea00700 1 --2- 172.21.15.31:0/2945126780 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f431c0390a0 0x7f431c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:24.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.886+0000 7f432ea00700 1 -- 172.21.15.31:0/2945126780 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f432804d240 con 0x7f433407e6d0 2024-04-25T01:05:24.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.886+0000 7f4339400700 1 --2- 172.21.15.31:0/2945126780 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f431c0390a0 0x7f431c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:24.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.887+0000 7f4339400700 1 --2- 172.21.15.31:0/2945126780 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f431c0390a0 0x7f431c03b560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4324006fd0 tx=0x7f4324006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:24.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:21.891+0000 7f432ea00700 1 -- 172.21.15.31:0/2945126780 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f432801a4e0 con 0x7f433407e6d0 2024-04-25T01:05:24.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:22.187+0000 7f433c800700 1 -- 172.21.15.31:0/2945126780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f4334062e40 con 0x7f433407e6d0 2024-04-25T01:05:24.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:22.678+0000 7f432ea00700 1 -- 172.21.15.31:0/2945126780 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f432802d3f0 con 0x7f433407e6d0 2024-04-25T01:05:24.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:22.682+0000 7f432ea00700 1 -- 172.21.15.31:0/2945126780 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f4328024ad0 con 0x7f433407e6d0 2024-04-25T01:05:24.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:22.682+0000 7f433c800700 1 -- 172.21.15.31:0/2945126780 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f431c0390a0 msgr2=0x7f431c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:24.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:22.682+0000 7f433c800700 1 --2- 172.21.15.31:0/2945126780 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f431c0390a0 0x7f431c03b560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4324006fd0 tx=0x7f4324006e40 comp rx=0 tx=0).stop 2024-04-25T01:05:24.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:22.682+0000 7f433c800700 1 -- 172.21.15.31:0/2945126780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f433407e6d0 msgr2=0x7f43341a2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:24.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:22.682+0000 7f433c800700 1 --2- 172.21.15.31:0/2945126780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f433407e6d0 0x7f43341a2390 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4328011b60 tx=0x7f4328011c40 comp rx=0 tx=0).stop 2024-04-25T01:05:24.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:22.683+0000 7f433c800700 1 -- 172.21.15.31:0/2945126780 shutdown_connections 2024-04-25T01:05:24.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:22.683+0000 7f433c800700 1 --2- 172.21.15.31:0/2945126780 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f431c0390a0 0x7f431c03b560 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:24.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:22.683+0000 7f433c800700 1 --2- 172.21.15.31:0/2945126780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f433407e6d0 0x7f43341a2390 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:24.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:22.683+0000 7f433c800700 1 -- 172.21.15.31:0/2945126780 >> 172.21.15.31:0/2945126780 conn(0x7f4334103480 msgr2=0x7f43341058e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:24.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:22.683+0000 7f433c800700 1 -- 172.21.15.31:0/2945126780 shutdown_connections 2024-04-25T01:05:24.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:22.685+0000 7f433c800700 1 -- 172.21.15.31:0/2945126780 wait complete. 2024-04-25T01:05:25.431 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:25 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1347967317' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T01:05:25.764 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:25 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:05:25.559+0000 7fdac5631200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T01:05:25.832 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-25T01:05:25.832 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T01:05:25.832 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T01:05:25.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T01:05:25.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T01:05:25.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-25T01:05:25.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.798+0000 7f7641a00700 1 Processor -- start 2024-04-25T01:05:25.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.798+0000 7f7641a00700 1 -- start start 2024-04-25T01:05:25.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.798+0000 7f7641a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f763c07de10 0x7f763c07c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:25.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.798+0000 7f7641a00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f763c07c800 con 0x7f763c07de10 2024-04-25T01:05:25.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.799+0000 7f763aa00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f763c07de10 0x7f763c07c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:25.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.799+0000 7f763aa00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f763c07de10 0x7f763c07c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47026/0 (socket says 172.21.15.31:47026) 2024-04-25T01:05:25.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.799+0000 7f763aa00700 1 -- 172.21.15.31:0/3970418893 learned_addr learned my addr 172.21.15.31:0/3970418893 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:25.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.800+0000 7f763aa00700 1 -- 172.21.15.31:0/3970418893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763c07c980 con 0x7f763c07de10 2024-04-25T01:05:25.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.800+0000 7f763aa00700 1 --2- 172.21.15.31:0/3970418893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f763c07de10 0x7f763c07c2c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f762c0098a0 tx=0x7f762c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfb12928515ac6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:25.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.800+0000 7f763a000700 1 -- 172.21.15.31:0/3970418893 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f762c004160 con 0x7f763c07de10 2024-04-25T01:05:25.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.800+0000 7f763a000700 1 -- 172.21.15.31:0/3970418893 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f762c004300 con 0x7f763c07de10 2024-04-25T01:05:25.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.800+0000 7f763a000700 1 -- 172.21.15.31:0/3970418893 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f762c011b20 con 0x7f763c07de10 2024-04-25T01:05:25.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.801+0000 7f7641a00700 1 -- 172.21.15.31:0/3970418893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f763c07de10 msgr2=0x7f763c07c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:25.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.801+0000 7f7641a00700 1 --2- 172.21.15.31:0/3970418893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f763c07de10 0x7f763c07c2c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f762c0098a0 tx=0x7f762c009bb0 comp rx=0 tx=0).stop 2024-04-25T01:05:25.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.801+0000 7f7641a00700 1 -- 172.21.15.31:0/3970418893 shutdown_connections 2024-04-25T01:05:25.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.801+0000 7f7641a00700 1 --2- 172.21.15.31:0/3970418893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f763c07de10 0x7f763c07c2c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:25.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.802+0000 7f7641a00700 1 -- 172.21.15.31:0/3970418893 >> 172.21.15.31:0/3970418893 conn(0x7f763c103420 msgr2=0x7f763c105840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:25.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.802+0000 7f7641a00700 1 -- 172.21.15.31:0/3970418893 shutdown_connections 2024-04-25T01:05:25.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.802+0000 7f7641a00700 1 -- 172.21.15.31:0/3970418893 wait complete. 2024-04-25T01:05:25.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.803+0000 7f7641a00700 1 Processor -- start 2024-04-25T01:05:25.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.803+0000 7f7641a00700 1 -- start start 2024-04-25T01:05:25.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.803+0000 7f7641a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f763c07de10 0x7f763c19df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:25.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.803+0000 7f7641a00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f763c10e440 con 0x7f763c07de10 2024-04-25T01:05:25.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.803+0000 7f763aa00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f763c07de10 0x7f763c19df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:25.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.804+0000 7f763aa00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f763c07de10 0x7f763c19df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47038/0 (socket says 172.21.15.31:47038) 2024-04-25T01:05:25.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.804+0000 7f763aa00700 1 -- 172.21.15.31:0/1347967317 learned_addr learned my addr 172.21.15.31:0/1347967317 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:25.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.804+0000 7f763aa00700 1 -- 172.21.15.31:0/1347967317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763c19e440 con 0x7f763c07de10 2024-04-25T01:05:25.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.804+0000 7f763aa00700 1 --2- 172.21.15.31:0/1347967317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f763c07de10 0x7f763c19df00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f762c0117e0 tx=0x7f762c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:25.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.804+0000 7f7633e00700 1 -- 172.21.15.31:0/1347967317 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f762c003ba0 con 0x7f763c07de10 2024-04-25T01:05:25.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.805+0000 7f7633e00700 1 -- 172.21.15.31:0/1347967317 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f762c01a910 con 0x7f763c07de10 2024-04-25T01:05:25.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.805+0000 7f7641a00700 1 -- 172.21.15.31:0/1347967317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f763c19e6d0 con 0x7f763c07de10 2024-04-25T01:05:25.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.805+0000 7f7633e00700 1 -- 172.21.15.31:0/1347967317 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f762c0248f0 con 0x7f763c07de10 2024-04-25T01:05:25.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.805+0000 7f7641a00700 1 -- 172.21.15.31:0/1347967317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f763c19eb30 con 0x7f763c07de10 2024-04-25T01:05:25.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.806+0000 7f7641a00700 1 -- 172.21.15.31:0/1347967317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f763c063890 con 0x7f763c07de10 2024-04-25T01:05:25.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.806+0000 7f7633e00700 1 -- 172.21.15.31:0/1347967317 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f762c022020 con 0x7f763c07de10 2024-04-25T01:05:25.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.807+0000 7f7633e00700 1 --2- 172.21.15.31:0/1347967317 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f7620039100 0x7f762003b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:25.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.807+0000 7f7633e00700 1 -- 172.21.15.31:0/1347967317 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f762c04d4c0 con 0x7f763c07de10 2024-04-25T01:05:25.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.807+0000 7f7632000700 1 -- 172.21.15.31:0/1347967317 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f7620039100 msgr2=0x7f762003b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1901973432 2024-04-25T01:05:25.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.807+0000 7f7632000700 1 --2- 172.21.15.31:0/1347967317 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f7620039100 0x7f762003b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:05:25.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:24.812+0000 7f7633e00700 1 -- 172.21.15.31:0/1347967317 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f762c01ab70 con 0x7f763c07de10 2024-04-25T01:05:25.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:25.008+0000 7f7632000700 1 -- 172.21.15.31:0/1347967317 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f7620039100 msgr2=0x7f762003b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1901973432 2024-04-25T01:05:25.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:25.008+0000 7f7632000700 1 --2- 172.21.15.31:0/1347967317 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f7620039100 0x7f762003b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:05:25.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:25.103+0000 7f7641a00700 1 -- 172.21.15.31:0/1347967317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f763c1a15a0 con 0x7f763c07de10 2024-04-25T01:05:25.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:25.104+0000 7f7633e00700 1 -- 172.21.15.31:0/1347967317 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f762c0212c0 con 0x7f763c07de10 2024-04-25T01:05:25.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:25.107+0000 7f7641a00700 1 -- 172.21.15.31:0/1347967317 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f7620039100 msgr2=0x7f762003b5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:05:25.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:25.107+0000 7f7641a00700 1 --2- 172.21.15.31:0/1347967317 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f7620039100 0x7f762003b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:25.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:25.107+0000 7f7641a00700 1 -- 172.21.15.31:0/1347967317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f763c07de10 msgr2=0x7f763c19df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:25.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:25.107+0000 7f7641a00700 1 --2- 172.21.15.31:0/1347967317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f763c07de10 0x7f763c19df00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f762c0117e0 tx=0x7f762c011ba0 comp rx=0 tx=0).stop 2024-04-25T01:05:25.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:25.107+0000 7f7641a00700 1 -- 172.21.15.31:0/1347967317 shutdown_connections 2024-04-25T01:05:25.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:25.107+0000 7f7641a00700 1 --2- 172.21.15.31:0/1347967317 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7f7620039100 0x7f762003b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:25.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:25.107+0000 7f7641a00700 1 --2- 172.21.15.31:0/1347967317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f763c07de10 0x7f763c19df00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:25.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:25.107+0000 7f7641a00700 1 -- 172.21.15.31:0/1347967317 >> 172.21.15.31:0/1347967317 conn(0x7f763c103420 msgr2=0x7f763c103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:25.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:25.108+0000 7f7641a00700 1 -- 172.21.15.31:0/1347967317 shutdown_connections 2024-04-25T01:05:25.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:25.108+0000 7f7641a00700 1 -- 172.21.15.31:0/1347967317 wait complete. 2024-04-25T01:05:25.837 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for the mgr to restart... 2024-04-25T01:05:25.837 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr epoch 9... 2024-04-25T01:05:26.158 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:25 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:05:25.802+0000 7fdac5631200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T01:05:26.159 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:26 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:05:26.030+0000 7fdac5631200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T01:05:26.738 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:26 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:05:26.442+0000 7fdac5631200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T01:05:26.738 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:26 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:05:26.572+0000 7fdac5631200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T01:05:27.015 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:26 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:05:26.832+0000 7fdac5631200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T01:05:27.515 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:27 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:05:27.094+0000 7fdac5631200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:05:28.265 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:27 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:05:27.906+0000 7fdac5631200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T01:05:28.765 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:28 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:05:28.477+0000 7fdac5631200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T01:05:28.765 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:28 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:05:28.608+0000 7fdac5631200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T01:05:29.265 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:28 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:05:28.880+0000 7fdac5631200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T01:05:29.765 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:29 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:05:29.547+0000 7fdac5631200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T01:05:29.765 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:29 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:05:29.673+0000 7fdac5631200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T01:05:30.265 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:30 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:05:30.004+0000 7fdac5631200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T01:05:30.265 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:30 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:05:30.188+0000 7fdac5631200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T01:05:30.600 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:30 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:05:30.319+0000 7fdac5631200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T01:05:30.600 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:30 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:05:30.477+0000 7fdac5631200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T01:05:31.014 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:30 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:05:30.604+0000 7fdac5631200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T01:05:31.925 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:31 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:05:31.639+0000 7fdac5631200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T01:05:31.926 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:05:31 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:05:31.930+0000 7fdac5631200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T01:05:32.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:31 smithi031 ceph-mon[28053]: Active manager daemon x restarted 2024-04-25T01:05:32.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:31 smithi031 ceph-mon[28053]: Activating manager daemon x 2024-04-25T01:05:32.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:31 smithi031 ceph-mon[28053]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T01:05:32.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:31 smithi031 ceph-mon[28053]: mgrmap e10: x(active, starting, since 0.00837977s) 2024-04-25T01:05:32.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:31 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:05:32.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:31 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T01:05:32.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:31 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T01:05:32.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:31 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T01:05:32.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:31 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T01:05:32.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:31 smithi031 ceph-mon[28053]: Manager daemon x is now available 2024-04-25T01:05:33.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:32 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:05:33.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:32 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T01:05:33.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:32 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T01:05:33.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:32 smithi031 ceph-mon[28053]: [25/Apr/2024:01:05:32] ENGINE Bus STARTING 2024-04-25T01:05:33.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:32 smithi031 ceph-mon[28053]: [25/Apr/2024:01:05:32] ENGINE Serving on http://172.21.15.31:8765 2024-04-25T01:05:33.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:32 smithi031 ceph-mon[28053]: [25/Apr/2024:01:05:32] ENGINE Serving on https://172.21.15.31:7150 2024-04-25T01:05:33.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:32 smithi031 ceph-mon[28053]: [25/Apr/2024:01:05:32] ENGINE Bus STARTED 2024-04-25T01:05:33.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:32 smithi031 ceph-mon[28053]: mgrmap e11: x(active, since 1.00839s) 2024-04-25T01:05:33.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-25T01:05:33.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T01:05:33.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T01:05:33.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-25T01:05:33.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.450+0000 7feb8ca00700 1 Processor -- start 2024-04-25T01:05:33.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.450+0000 7feb8ca00700 1 -- start start 2024-04-25T01:05:33.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.451+0000 7feb8ca00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb8807de00 0x7feb8807c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:33.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.451+0000 7feb8ca00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7feb8807c840 con 0x7feb8807de00 2024-04-25T01:05:33.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.451+0000 7feb86800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb8807de00 0x7feb8807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:33.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.451+0000 7feb86800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb8807de00 0x7feb8807c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47040/0 (socket says 172.21.15.31:47040) 2024-04-25T01:05:33.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.451+0000 7feb86800700 1 -- 172.21.15.31:0/1914315094 learned_addr learned my addr 172.21.15.31:0/1914315094 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:33.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.452+0000 7feb86800700 1 -- 172.21.15.31:0/1914315094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb8807c9c0 con 0x7feb8807de00 2024-04-25T01:05:33.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.452+0000 7feb86800700 1 --2- 172.21.15.31:0/1914315094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb8807de00 0x7feb8807c300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb780098a0 tx=0x7feb78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b93af686b7352337 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:33.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.452+0000 7feb85400700 1 -- 172.21.15.31:0/1914315094 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb78004160 con 0x7feb8807de00 2024-04-25T01:05:33.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.452+0000 7feb85400700 1 -- 172.21.15.31:0/1914315094 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb78004300 con 0x7feb8807de00 2024-04-25T01:05:33.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.452+0000 7feb85400700 1 -- 172.21.15.31:0/1914315094 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb78011b20 con 0x7feb8807de00 2024-04-25T01:05:33.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.453+0000 7feb8ca00700 1 -- 172.21.15.31:0/1914315094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb8807de00 msgr2=0x7feb8807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:33.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.453+0000 7feb8ca00700 1 --2- 172.21.15.31:0/1914315094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb8807de00 0x7feb8807c300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb780098a0 tx=0x7feb78009bb0 comp rx=0 tx=0).stop 2024-04-25T01:05:33.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.454+0000 7feb8ca00700 1 -- 172.21.15.31:0/1914315094 shutdown_connections 2024-04-25T01:05:33.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.454+0000 7feb8ca00700 1 --2- 172.21.15.31:0/1914315094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb8807de00 0x7feb8807c300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:33.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.454+0000 7feb8ca00700 1 -- 172.21.15.31:0/1914315094 >> 172.21.15.31:0/1914315094 conn(0x7feb881032e0 msgr2=0x7feb88105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:33.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.454+0000 7feb8ca00700 1 -- 172.21.15.31:0/1914315094 shutdown_connections 2024-04-25T01:05:33.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.454+0000 7feb8ca00700 1 -- 172.21.15.31:0/1914315094 wait complete. 2024-04-25T01:05:33.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.455+0000 7feb8ca00700 1 Processor -- start 2024-04-25T01:05:33.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.456+0000 7feb8ca00700 1 -- start start 2024-04-25T01:05:33.656 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.456+0000 7feb8ca00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb8807de00 0x7feb8819dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:33.656 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.456+0000 7feb8ca00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7feb8810e2c0 con 0x7feb8807de00 2024-04-25T01:05:33.656 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.456+0000 7feb86800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb8807de00 0x7feb8819dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:33.656 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.456+0000 7feb86800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb8807de00 0x7feb8819dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47048/0 (socket says 172.21.15.31:47048) 2024-04-25T01:05:33.656 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.456+0000 7feb86800700 1 -- 172.21.15.31:0/2256158115 learned_addr learned my addr 172.21.15.31:0/2256158115 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:33.656 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.457+0000 7feb86800700 1 -- 172.21.15.31:0/2256158115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb8819e2e0 con 0x7feb8807de00 2024-04-25T01:05:33.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.457+0000 7feb86800700 1 --2- 172.21.15.31:0/2256158115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb8807de00 0x7feb8819dda0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feb78011b60 tx=0x7feb78011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:33.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.457+0000 7feb7f400700 1 -- 172.21.15.31:0/2256158115 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb78003b60 con 0x7feb8807de00 2024-04-25T01:05:33.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.457+0000 7feb7f400700 1 -- 172.21.15.31:0/2256158115 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb7801a910 con 0x7feb8807de00 2024-04-25T01:05:33.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.457+0000 7feb7f400700 1 -- 172.21.15.31:0/2256158115 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb78024930 con 0x7feb8807de00 2024-04-25T01:05:33.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.457+0000 7feb8ca00700 1 -- 172.21.15.31:0/2256158115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb8819e570 con 0x7feb8807de00 2024-04-25T01:05:33.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.458+0000 7feb8ca00700 1 -- 172.21.15.31:0/2256158115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb8819e9d0 con 0x7feb8807de00 2024-04-25T01:05:33.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.459+0000 7feb7f400700 1 -- 172.21.15.31:0/2256158115 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7feb78022020 con 0x7feb8807de00 2024-04-25T01:05:33.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.460+0000 7feb7f400700 1 --2- 172.21.15.31:0/2256158115 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7feb6c0390a0 0x7feb6c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:33.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.460+0000 7feb7f400700 1 -- 172.21.15.31:0/2256158115 --> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7feb6c03bc70 con 0x7feb6c0390a0 2024-04-25T01:05:33.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.460+0000 7feb7f400700 1 -- 172.21.15.31:0/2256158115 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7feb7804d360 con 0x7feb8807de00 2024-04-25T01:05:33.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.460+0000 7feb85e00700 1 -- 172.21.15.31:0/2256158115 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7feb6c0390a0 msgr2=0x7feb6c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1901973432 2024-04-25T01:05:33.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.460+0000 7feb85e00700 1 --2- 172.21.15.31:0/2256158115 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7feb6c0390a0 0x7feb6c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:05:33.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.661+0000 7feb85e00700 1 -- 172.21.15.31:0/2256158115 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7feb6c0390a0 msgr2=0x7feb6c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1901973432 2024-04-25T01:05:33.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:26.661+0000 7feb85e00700 1 --2- 172.21.15.31:0/2256158115 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7feb6c0390a0 0x7feb6c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:05:33.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:27.062+0000 7feb85e00700 1 -- 172.21.15.31:0/2256158115 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7feb6c0390a0 msgr2=0x7feb6c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1901973432 2024-04-25T01:05:33.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:27.062+0000 7feb85e00700 1 --2- 172.21.15.31:0/2256158115 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7feb6c0390a0 0x7feb6c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:05:33.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:27.863+0000 7feb85e00700 1 -- 172.21.15.31:0/2256158115 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7feb6c0390a0 msgr2=0x7feb6c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1901973432 2024-04-25T01:05:33.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:27.863+0000 7feb85e00700 1 --2- 172.21.15.31:0/2256158115 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7feb6c0390a0 0x7feb6c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:05:33.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:29.465+0000 7feb85e00700 1 -- 172.21.15.31:0/2256158115 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7feb6c0390a0 msgr2=0x7feb6c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1901973432 2024-04-25T01:05:33.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:29.465+0000 7feb85e00700 1 --2- 172.21.15.31:0/2256158115 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7feb6c0390a0 0x7feb6c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T01:05:33.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:31.936+0000 7feb7f400700 1 -- 172.21.15.31:0/2256158115 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7feb7801a430 con 0x7feb8807de00 2024-04-25T01:05:33.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:31.937+0000 7feb7f400700 1 -- 172.21.15.31:0/2256158115 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7feb6c0390a0 msgr2=0x7feb6c03b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:05:33.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:31.937+0000 7feb7f400700 1 --2- 172.21.15.31:0/2256158115 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7feb6c0390a0 0x7feb6c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:33.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:32.938+0000 7feb7f400700 1 -- 172.21.15.31:0/2256158115 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7feb7800bd10 con 0x7feb8807de00 2024-04-25T01:05:33.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:32.938+0000 7feb7f400700 1 --2- 172.21.15.31:0/2256158115 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7feb6c03cb60 0x7feb6c03ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:33.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:32.938+0000 7feb7f400700 1 -- 172.21.15.31:0/2256158115 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7feb7800fec0 con 0x7feb6c03cb60 2024-04-25T01:05:33.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:32.940+0000 7feb85e00700 1 --2- 172.21.15.31:0/2256158115 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7feb6c03cb60 0x7feb6c03ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:33.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:32.941+0000 7feb85e00700 1 --2- 172.21.15.31:0/2256158115 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7feb6c03cb60 0x7feb6c03ef50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7feb74003a10 tx=0x7feb740092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:33.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:32.942+0000 7feb7f400700 1 -- 172.21.15.31:0/2256158115 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7feb7800fec0 con 0x7feb6c03cb60 2024-04-25T01:05:33.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:32.948+0000 7feb8ca00700 1 -- 172.21.15.31:0/2256158115 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7feb88063890 con 0x7feb6c03cb60 2024-04-25T01:05:33.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:32.949+0000 7feb7f400700 1 -- 172.21.15.31:0/2256158115 <== mgr.14152 v2:172.21.15.31:6800/2874594057 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7feb88063890 con 0x7feb6c03cb60 2024-04-25T01:05:33.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:32.949+0000 7feb8ca00700 1 -- 172.21.15.31:0/2256158115 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7feb6c03cb60 msgr2=0x7feb6c03ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:33.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:32.949+0000 7feb8ca00700 1 --2- 172.21.15.31:0/2256158115 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7feb6c03cb60 0x7feb6c03ef50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7feb74003a10 tx=0x7feb740092b0 comp rx=0 tx=0).stop 2024-04-25T01:05:33.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:32.949+0000 7feb8ca00700 1 -- 172.21.15.31:0/2256158115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb8807de00 msgr2=0x7feb8819dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:33.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:32.949+0000 7feb8ca00700 1 --2- 172.21.15.31:0/2256158115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb8807de00 0x7feb8819dda0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feb78011b60 tx=0x7feb78011c40 comp rx=0 tx=0).stop 2024-04-25T01:05:33.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:32.950+0000 7feb8ca00700 1 -- 172.21.15.31:0/2256158115 shutdown_connections 2024-04-25T01:05:33.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:32.950+0000 7feb8ca00700 1 --2- 172.21.15.31:0/2256158115 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7feb6c03cb60 0x7feb6c03ef50 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:33.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:32.950+0000 7feb8ca00700 1 --2- 172.21.15.31:0/2256158115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb8807de00 0x7feb8819dda0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:33.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:32.950+0000 7feb8ca00700 1 --2- 172.21.15.31:0/2256158115 >> [v2:172.21.15.31:6800/1901973432,v1:172.21.15.31:6801/1901973432] conn(0x7feb6c0390a0 0x7feb6c03b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:33.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:32.950+0000 7feb8ca00700 1 -- 172.21.15.31:0/2256158115 >> 172.21.15.31:0/2256158115 conn(0x7feb881032e0 msgr2=0x7feb88103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:33.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:32.950+0000 7feb8ca00700 1 -- 172.21.15.31:0/2256158115 shutdown_connections 2024-04-25T01:05:33.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:32.950+0000 7feb8ca00700 1 -- 172.21.15.31:0/2256158115 wait complete. 2024-04-25T01:05:33.660 INFO:teuthology.orchestra.run.smithi031.stdout:mgr epoch 9 is available 2024-04-25T01:05:33.660 INFO:teuthology.orchestra.run.smithi031.stdout:Generating a dashboard self-signed certificate... 2024-04-25T01:05:34.231 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:33 smithi031 ceph-mon[28053]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T01:05:34.231 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:33 smithi031 ceph-mon[28053]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T01:05:35.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:34 smithi031 ceph-mon[28053]: mgrmap e12: x(active, since 2s) 2024-04-25T01:05:35.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:34 smithi031 ceph-mon[28053]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:05:35.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:34 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:05:35.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:34 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:05:35.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:34 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:05:35.655 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T01:05:35.655 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.558+0000 7f5fbfe00700 1 Processor -- start 2024-04-25T01:05:35.655 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.558+0000 7f5fbfe00700 1 -- start start 2024-04-25T01:05:35.655 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.558+0000 7f5fbfe00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fc0109f20 0x7f5fc010a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:35.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.558+0000 7f5fbfe00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc010a880 con 0x7f5fc0109f20 2024-04-25T01:05:35.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.559+0000 7f5fbea00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fc0109f20 0x7f5fc010a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:35.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.559+0000 7f5fbea00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fc0109f20 0x7f5fc010a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60456/0 (socket says 172.21.15.31:60456) 2024-04-25T01:05:35.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.559+0000 7f5fbea00700 1 -- 172.21.15.31:0/872126196 learned_addr learned my addr 172.21.15.31:0/872126196 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:35.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.559+0000 7f5fbea00700 1 -- 172.21.15.31:0/872126196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc010afe0 con 0x7f5fc0109f20 2024-04-25T01:05:35.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.560+0000 7f5fbea00700 1 --2- 172.21.15.31:0/872126196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fc0109f20 0x7f5fc010a340 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00098a0 tx=0x7f5fb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15a48625725c506 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:35.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.560+0000 7f5fbe000700 1 -- 172.21.15.31:0/872126196 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fb0004160 con 0x7f5fc0109f20 2024-04-25T01:05:35.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.560+0000 7f5fbe000700 1 -- 172.21.15.31:0/872126196 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fb0004300 con 0x7f5fc0109f20 2024-04-25T01:05:35.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.560+0000 7f5fbe000700 1 -- 172.21.15.31:0/872126196 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fb0011b20 con 0x7f5fc0109f20 2024-04-25T01:05:35.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.561+0000 7f5fbfe00700 1 -- 172.21.15.31:0/872126196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fc0109f20 msgr2=0x7f5fc010a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:35.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.561+0000 7f5fbfe00700 1 --2- 172.21.15.31:0/872126196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fc0109f20 0x7f5fc010a340 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00098a0 tx=0x7f5fb0009bb0 comp rx=0 tx=0).stop 2024-04-25T01:05:35.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.561+0000 7f5fbfe00700 1 -- 172.21.15.31:0/872126196 shutdown_connections 2024-04-25T01:05:35.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.561+0000 7f5fbfe00700 1 --2- 172.21.15.31:0/872126196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fc0109f20 0x7f5fc010a340 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:35.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.561+0000 7f5fbfe00700 1 -- 172.21.15.31:0/872126196 >> 172.21.15.31:0/872126196 conn(0x7f5fc007d330 msgr2=0x7f5fc007d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:35.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.561+0000 7f5fbfe00700 1 -- 172.21.15.31:0/872126196 shutdown_connections 2024-04-25T01:05:35.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.561+0000 7f5fbfe00700 1 -- 172.21.15.31:0/872126196 wait complete. 2024-04-25T01:05:35.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.562+0000 7f5fbfe00700 1 Processor -- start 2024-04-25T01:05:35.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.563+0000 7f5fbfe00700 1 -- start start 2024-04-25T01:05:35.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.563+0000 7f5fbfe00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fc0109f20 0x7f5fc019ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:35.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.563+0000 7f5fbfe00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc010e420 con 0x7f5fc0109f20 2024-04-25T01:05:35.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.563+0000 7f5fbea00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fc0109f20 0x7f5fc019ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:35.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.563+0000 7f5fbea00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fc0109f20 0x7f5fc019ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60458/0 (socket says 172.21.15.31:60458) 2024-04-25T01:05:35.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.563+0000 7f5fbea00700 1 -- 172.21.15.31:0/2124352818 learned_addr learned my addr 172.21.15.31:0/2124352818 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:35.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.564+0000 7f5fbea00700 1 -- 172.21.15.31:0/2124352818 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc019e2f0 con 0x7f5fc0109f20 2024-04-25T01:05:35.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.564+0000 7f5fbea00700 1 --2- 172.21.15.31:0/2124352818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fc0109f20 0x7f5fc019ddb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00116a0 tx=0x7f5fb0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:35.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.564+0000 7f5fb7e00700 1 -- 172.21.15.31:0/2124352818 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fb0003a40 con 0x7f5fc0109f20 2024-04-25T01:05:35.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.564+0000 7f5fbfe00700 1 -- 172.21.15.31:0/2124352818 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc019e580 con 0x7f5fc0109f20 2024-04-25T01:05:35.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.564+0000 7f5fb7e00700 1 -- 172.21.15.31:0/2124352818 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fb0003be0 con 0x7f5fc0109f20 2024-04-25T01:05:35.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.564+0000 7f5fb7e00700 1 -- 172.21.15.31:0/2124352818 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fb001a750 con 0x7f5fc0109f20 2024-04-25T01:05:35.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.564+0000 7f5fbfe00700 1 -- 172.21.15.31:0/2124352818 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc01a1230 con 0x7f5fc0109f20 2024-04-25T01:05:35.663 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.566+0000 7f5fb7e00700 1 -- 172.21.15.31:0/2124352818 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f5fb001aa00 con 0x7f5fc0109f20 2024-04-25T01:05:35.663 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.566+0000 7f5fbfe00700 1 -- 172.21.15.31:0/2124352818 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fc0063890 con 0x7f5fc0109f20 2024-04-25T01:05:35.663 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.566+0000 7f5fb7e00700 1 --2- 172.21.15.31:0/2124352818 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5fa40390b0 0x7f5fa403b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:35.664 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.566+0000 7f5fb7e00700 1 -- 172.21.15.31:0/2124352818 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5fb004d830 con 0x7f5fc0109f20 2024-04-25T01:05:35.664 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.566+0000 7f5fb6000700 1 --2- 172.21.15.31:0/2124352818 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5fa40390b0 0x7f5fa403b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:35.664 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.567+0000 7f5fb6000700 1 --2- 172.21.15.31:0/2124352818 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5fa40390b0 0x7f5fa403b570 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800ad30 tx=0x7f5fa80093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:35.664 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.572+0000 7f5fb7e00700 1 -- 172.21.15.31:0/2124352818 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5fb0011e20 con 0x7f5fc0109f20 2024-04-25T01:05:35.664 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.810+0000 7f5fbfe00700 1 -- 172.21.15.31:0/2124352818 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f5fc0107b60 con 0x7f5fa40390b0 2024-04-25T01:05:35.664 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.940+0000 7f5fb7e00700 1 -- 172.21.15.31:0/2124352818 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f5fc0107b60 con 0x7f5fa40390b0 2024-04-25T01:05:35.664 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.943+0000 7f5fbfe00700 1 -- 172.21.15.31:0/2124352818 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5fa40390b0 msgr2=0x7f5fa403b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:35.664 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.943+0000 7f5fbfe00700 1 --2- 172.21.15.31:0/2124352818 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5fa40390b0 0x7f5fa403b570 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800ad30 tx=0x7f5fa80093f0 comp rx=0 tx=0).stop 2024-04-25T01:05:35.664 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.943+0000 7f5fbfe00700 1 -- 172.21.15.31:0/2124352818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fc0109f20 msgr2=0x7f5fc019ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:35.664 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.943+0000 7f5fbfe00700 1 --2- 172.21.15.31:0/2124352818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fc0109f20 0x7f5fc019ddb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00116a0 tx=0x7f5fb0011ba0 comp rx=0 tx=0).stop 2024-04-25T01:05:35.664 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.944+0000 7f5fbfe00700 1 -- 172.21.15.31:0/2124352818 shutdown_connections 2024-04-25T01:05:35.664 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.944+0000 7f5fbfe00700 1 --2- 172.21.15.31:0/2124352818 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5fa40390b0 0x7f5fa403b570 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:35.664 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.944+0000 7f5fbfe00700 1 --2- 172.21.15.31:0/2124352818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fc0109f20 0x7f5fc019ddb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:35.664 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.944+0000 7f5fbfe00700 1 -- 172.21.15.31:0/2124352818 >> 172.21.15.31:0/2124352818 conn(0x7f5fc007d330 msgr2=0x7f5fc0107450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:35.665 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.944+0000 7f5fbfe00700 1 -- 172.21.15.31:0/2124352818 shutdown_connections 2024-04-25T01:05:35.665 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:34.944+0000 7f5fbfe00700 1 -- 172.21.15.31:0/2124352818 wait complete. 2024-04-25T01:05:35.665 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial admin user... 2024-04-25T01:05:37.247 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:36 smithi031 ceph-mon[28053]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:05:37.247 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:36 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:05:37.600 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$1rYw6IwUGvQWcA2OBxoLL.tv8KU7J2ybJut1cvWtHsYD38jlE2It6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714007136, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T01:05:37.601 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.317+0000 7f6f23c00700 1 Processor -- start 2024-04-25T01:05:37.601 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.317+0000 7f6f23c00700 1 -- start start 2024-04-25T01:05:37.601 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.317+0000 7f6f23c00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f1c1088c0 0x7f6f1c10acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:37.601 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.317+0000 7f6f23c00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c07beb0 con 0x7f6f1c1088c0 2024-04-25T01:05:37.601 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.318+0000 7f6f21200700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f1c1088c0 0x7f6f1c10acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:37.601 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.318+0000 7f6f21200700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f1c1088c0 0x7f6f1c10acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60462/0 (socket says 172.21.15.31:60462) 2024-04-25T01:05:37.601 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.318+0000 7f6f21200700 1 -- 172.21.15.31:0/2520688205 learned_addr learned my addr 172.21.15.31:0/2520688205 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:37.601 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.318+0000 7f6f21200700 1 -- 172.21.15.31:0/2520688205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1c10b1f0 con 0x7f6f1c1088c0 2024-04-25T01:05:37.601 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.319+0000 7f6f21200700 1 --2- 172.21.15.31:0/2520688205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f1c1088c0 0x7f6f1c10acb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6f100098a0 tx=0x7f6f10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71af7f942b0559a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:37.601 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.319+0000 7f6f1b400700 1 -- 172.21.15.31:0/2520688205 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f10004160 con 0x7f6f1c1088c0 2024-04-25T01:05:37.602 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.319+0000 7f6f1b400700 1 -- 172.21.15.31:0/2520688205 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f10004300 con 0x7f6f1c1088c0 2024-04-25T01:05:37.602 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.319+0000 7f6f1b400700 1 -- 172.21.15.31:0/2520688205 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f10011b20 con 0x7f6f1c1088c0 2024-04-25T01:05:37.602 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.320+0000 7f6f23c00700 1 -- 172.21.15.31:0/2520688205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f1c1088c0 msgr2=0x7f6f1c10acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:37.602 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.320+0000 7f6f23c00700 1 --2- 172.21.15.31:0/2520688205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f1c1088c0 0x7f6f1c10acb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6f100098a0 tx=0x7f6f10009bb0 comp rx=0 tx=0).stop 2024-04-25T01:05:37.602 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.320+0000 7f6f23c00700 1 -- 172.21.15.31:0/2520688205 shutdown_connections 2024-04-25T01:05:37.602 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.320+0000 7f6f23c00700 1 --2- 172.21.15.31:0/2520688205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f1c1088c0 0x7f6f1c10acb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:37.602 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.320+0000 7f6f23c00700 1 -- 172.21.15.31:0/2520688205 >> 172.21.15.31:0/2520688205 conn(0x7f6f1c102330 msgr2=0x7f6f1c104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:37.602 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.320+0000 7f6f23c00700 1 -- 172.21.15.31:0/2520688205 shutdown_connections 2024-04-25T01:05:37.602 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.320+0000 7f6f23c00700 1 -- 172.21.15.31:0/2520688205 wait complete. 2024-04-25T01:05:37.602 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.321+0000 7f6f23c00700 1 Processor -- start 2024-04-25T01:05:37.602 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.321+0000 7f6f23c00700 1 -- start start 2024-04-25T01:05:37.603 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.322+0000 7f6f23c00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f1c1088c0 0x7f6f1c108140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:37.603 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.322+0000 7f6f23c00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c10c260 con 0x7f6f1c1088c0 2024-04-25T01:05:37.603 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.322+0000 7f6f21200700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f1c1088c0 0x7f6f1c108140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:37.603 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.322+0000 7f6f21200700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f1c1088c0 0x7f6f1c108140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60470/0 (socket says 172.21.15.31:60470) 2024-04-25T01:05:37.603 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.322+0000 7f6f21200700 1 -- 172.21.15.31:0/1677258445 learned_addr learned my addr 172.21.15.31:0/1677258445 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:37.603 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.322+0000 7f6f21200700 1 -- 172.21.15.31:0/1677258445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1c108680 con 0x7f6f1c1088c0 2024-04-25T01:05:37.603 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.322+0000 7f6f21200700 1 --2- 172.21.15.31:0/1677258445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f1c1088c0 0x7f6f1c108140 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6f10011b60 tx=0x7f6f10011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:37.603 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.323+0000 7f6f19600700 1 -- 172.21.15.31:0/1677258445 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f10003b60 con 0x7f6f1c1088c0 2024-04-25T01:05:37.603 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.323+0000 7f6f19600700 1 -- 172.21.15.31:0/1677258445 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f1001a910 con 0x7f6f1c1088c0 2024-04-25T01:05:37.603 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.323+0000 7f6f19600700 1 -- 172.21.15.31:0/1677258445 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f10024930 con 0x7f6f1c1088c0 2024-04-25T01:05:37.603 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.323+0000 7f6f23c00700 1 -- 172.21.15.31:0/1677258445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f1c106870 con 0x7f6f1c1088c0 2024-04-25T01:05:37.604 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.323+0000 7f6f23c00700 1 -- 172.21.15.31:0/1677258445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f1c106d30 con 0x7f6f1c1088c0 2024-04-25T01:05:37.604 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.324+0000 7f6f19600700 1 -- 172.21.15.31:0/1677258445 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f6f10022020 con 0x7f6f1c1088c0 2024-04-25T01:05:37.604 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.324+0000 7f6f23c00700 1 -- 172.21.15.31:0/1677258445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f1c063890 con 0x7f6f1c1088c0 2024-04-25T01:05:37.604 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.325+0000 7f6f19600700 1 --2- 172.21.15.31:0/1677258445 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6f04039100 0x7f6f0403b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:37.604 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.325+0000 7f6f19600700 1 -- 172.21.15.31:0/1677258445 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6f1004d400 con 0x7f6f1c1088c0 2024-04-25T01:05:37.604 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.325+0000 7f6f1be00700 1 --2- 172.21.15.31:0/1677258445 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6f04039100 0x7f6f0403b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:37.604 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.326+0000 7f6f1be00700 1 --2- 172.21.15.31:0/1677258445 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6f04039100 0x7f6f0403b5c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6f08006fd0 tx=0x7f6f08006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:37.604 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.331+0000 7f6f19600700 1 -- 172.21.15.31:0/1677258445 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6f1000b3e0 con 0x7f6f1c1088c0 2024-04-25T01:05:37.604 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.588+0000 7f6f23c00700 1 -- 172.21.15.31:0/1677258445 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f6f1c106fb0 con 0x7f6f04039100 2024-04-25T01:05:37.604 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.860+0000 7f6f19600700 1 -- 172.21.15.31:0/1677258445 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f6f1c106fb0 con 0x7f6f04039100 2024-04-25T01:05:37.604 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.864+0000 7f6f23c00700 1 -- 172.21.15.31:0/1677258445 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6f04039100 msgr2=0x7f6f0403b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:37.604 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.864+0000 7f6f23c00700 1 --2- 172.21.15.31:0/1677258445 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6f04039100 0x7f6f0403b5c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6f08006fd0 tx=0x7f6f08006e40 comp rx=0 tx=0).stop 2024-04-25T01:05:37.605 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.864+0000 7f6f23c00700 1 -- 172.21.15.31:0/1677258445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f1c1088c0 msgr2=0x7f6f1c108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:37.605 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.864+0000 7f6f23c00700 1 --2- 172.21.15.31:0/1677258445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f1c1088c0 0x7f6f1c108140 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6f10011b60 tx=0x7f6f10011c40 comp rx=0 tx=0).stop 2024-04-25T01:05:37.605 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.864+0000 7f6f23c00700 1 -- 172.21.15.31:0/1677258445 shutdown_connections 2024-04-25T01:05:37.605 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.864+0000 7f6f23c00700 1 --2- 172.21.15.31:0/1677258445 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6f04039100 0x7f6f0403b5c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:37.605 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.864+0000 7f6f23c00700 1 --2- 172.21.15.31:0/1677258445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f1c1088c0 0x7f6f1c108140 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:37.605 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.864+0000 7f6f23c00700 1 -- 172.21.15.31:0/1677258445 >> 172.21.15.31:0/1677258445 conn(0x7f6f1c102330 msgr2=0x7f6f1c102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:37.605 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.865+0000 7f6f23c00700 1 -- 172.21.15.31:0/1677258445 shutdown_connections 2024-04-25T01:05:37.605 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:36.865+0000 7f6f23c00700 1 -- 172.21.15.31:0/1677258445 wait complete. 2024-04-25T01:05:37.605 INFO:teuthology.orchestra.run.smithi031.stdout:Fetching dashboard port number... 2024-04-25T01:05:38.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:38 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:05:39.514 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:39 smithi031 ceph-mon[28053]: mgrmap e13: x(active, since 6s) 2024-04-25T01:05:39.515 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:39 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3103263954' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T01:05:40.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T01:05:40.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.226+0000 7fc327600700 1 Processor -- start 2024-04-25T01:05:40.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.227+0000 7fc327600700 1 -- start start 2024-04-25T01:05:40.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.227+0000 7fc327600700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc320105af0 0x7fc320105f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:40.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.227+0000 7fc327600700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc3201064e0 con 0x7fc320105af0 2024-04-25T01:05:40.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.228+0000 7fc324c00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc320105af0 0x7fc320105f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:40.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.228+0000 7fc324c00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc320105af0 0x7fc320105f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58834/0 (socket says 172.21.15.31:58834) 2024-04-25T01:05:40.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.228+0000 7fc324c00700 1 -- 172.21.15.31:0/2144884485 learned_addr learned my addr 172.21.15.31:0/2144884485 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:40.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.228+0000 7fc324c00700 1 -- 172.21.15.31:0/2144884485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc320106660 con 0x7fc320105af0 2024-04-25T01:05:40.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.228+0000 7fc324c00700 1 --2- 172.21.15.31:0/2144884485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc320105af0 0x7fc320105f10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc3140098a0 tx=0x7fc314009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d6b9a19111a203 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:40.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.229+0000 7fc31f400700 1 -- 172.21.15.31:0/2144884485 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc314004160 con 0x7fc320105af0 2024-04-25T01:05:40.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.229+0000 7fc31f400700 1 -- 172.21.15.31:0/2144884485 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc314004300 con 0x7fc320105af0 2024-04-25T01:05:40.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.229+0000 7fc31f400700 1 -- 172.21.15.31:0/2144884485 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc314011b20 con 0x7fc320105af0 2024-04-25T01:05:40.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.230+0000 7fc327600700 1 -- 172.21.15.31:0/2144884485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc320105af0 msgr2=0x7fc320105f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:40.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.230+0000 7fc327600700 1 --2- 172.21.15.31:0/2144884485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc320105af0 0x7fc320105f10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc3140098a0 tx=0x7fc314009bb0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.230+0000 7fc327600700 1 -- 172.21.15.31:0/2144884485 shutdown_connections 2024-04-25T01:05:40.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.230+0000 7fc327600700 1 --2- 172.21.15.31:0/2144884485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc320105af0 0x7fc320105f10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.230+0000 7fc327600700 1 -- 172.21.15.31:0/2144884485 >> 172.21.15.31:0/2144884485 conn(0x7fc3201016a0 msgr2=0x7fc320103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:40.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.230+0000 7fc327600700 1 -- 172.21.15.31:0/2144884485 shutdown_connections 2024-04-25T01:05:40.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.230+0000 7fc327600700 1 -- 172.21.15.31:0/2144884485 wait complete. 2024-04-25T01:05:40.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.231+0000 7fc327600700 1 Processor -- start 2024-04-25T01:05:40.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.231+0000 7fc327600700 1 -- start start 2024-04-25T01:05:40.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.232+0000 7fc327600700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc320105af0 0x7fc320108e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:40.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.232+0000 7fc327600700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc32010c280 con 0x7fc320105af0 2024-04-25T01:05:40.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.232+0000 7fc324c00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc320105af0 0x7fc320108e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:40.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.232+0000 7fc324c00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc320105af0 0x7fc320108e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58838/0 (socket says 172.21.15.31:58838) 2024-04-25T01:05:40.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.232+0000 7fc324c00700 1 -- 172.21.15.31:0/3103263954 learned_addr learned my addr 172.21.15.31:0/3103263954 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:40.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.232+0000 7fc324c00700 1 -- 172.21.15.31:0/3103263954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3201093d0 con 0x7fc320105af0 2024-04-25T01:05:40.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.233+0000 7fc324c00700 1 --2- 172.21.15.31:0/3103263954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc320105af0 0x7fc320108e90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc314011b60 tx=0x7fc314011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:40.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.233+0000 7fc31d600700 1 -- 172.21.15.31:0/3103263954 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc314003b60 con 0x7fc320105af0 2024-04-25T01:05:40.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.233+0000 7fc31d600700 1 -- 172.21.15.31:0/3103263954 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc31401a910 con 0x7fc320105af0 2024-04-25T01:05:40.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.233+0000 7fc31d600700 1 -- 172.21.15.31:0/3103263954 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc314024930 con 0x7fc320105af0 2024-04-25T01:05:40.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.233+0000 7fc327600700 1 -- 172.21.15.31:0/3103263954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3201075c0 con 0x7fc320105af0 2024-04-25T01:05:40.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.233+0000 7fc327600700 1 -- 172.21.15.31:0/3103263954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc320107a80 con 0x7fc320105af0 2024-04-25T01:05:40.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.234+0000 7fc31d600700 1 -- 172.21.15.31:0/3103263954 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fc314022020 con 0x7fc320105af0 2024-04-25T01:05:40.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.234+0000 7fc312a00700 1 -- 172.21.15.31:0/3103263954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2fc005330 con 0x7fc320105af0 2024-04-25T01:05:40.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.235+0000 7fc31d600700 1 --2- 172.21.15.31:0/3103263954 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc308039100 0x7fc30803b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:40.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.235+0000 7fc31d600700 1 -- 172.21.15.31:0/3103263954 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc31404d530 con 0x7fc320105af0 2024-04-25T01:05:40.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.235+0000 7fc31fe00700 1 --2- 172.21.15.31:0/3103263954 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc308039100 0x7fc30803b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:40.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.236+0000 7fc31fe00700 1 --2- 172.21.15.31:0/3103263954 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc308039100 0x7fc30803b5c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc30c006fd0 tx=0x7fc30c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:40.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.241+0000 7fc31d600700 1 -- 172.21.15.31:0/3103263954 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc31402d960 con 0x7fc320105af0 2024-04-25T01:05:40.135 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.257+0000 7fc31d600700 1 -- 172.21.15.31:0/3103263954 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc31402d3f0 con 0x7fc320105af0 2024-04-25T01:05:40.135 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.481+0000 7fc312a00700 1 -- 172.21.15.31:0/3103263954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fc2fc005620 con 0x7fc320105af0 2024-04-25T01:05:40.135 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.482+0000 7fc31d600700 1 -- 172.21.15.31:0/3103263954 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fc31400f2e0 con 0x7fc320105af0 2024-04-25T01:05:40.135 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.485+0000 7fc312a00700 1 -- 172.21.15.31:0/3103263954 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc308039100 msgr2=0x7fc30803b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:40.135 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.485+0000 7fc312a00700 1 --2- 172.21.15.31:0/3103263954 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc308039100 0x7fc30803b5c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc30c006fd0 tx=0x7fc30c006e40 comp rx=0 tx=0).stop 2024-04-25T01:05:40.135 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.485+0000 7fc312a00700 1 -- 172.21.15.31:0/3103263954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc320105af0 msgr2=0x7fc320108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:40.135 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.485+0000 7fc312a00700 1 --2- 172.21.15.31:0/3103263954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc320105af0 0x7fc320108e90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc314011b60 tx=0x7fc314011c40 comp rx=0 tx=0).stop 2024-04-25T01:05:40.135 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.485+0000 7fc312a00700 1 -- 172.21.15.31:0/3103263954 shutdown_connections 2024-04-25T01:05:40.136 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.485+0000 7fc312a00700 1 --2- 172.21.15.31:0/3103263954 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc308039100 0x7fc30803b5c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.136 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.485+0000 7fc312a00700 1 --2- 172.21.15.31:0/3103263954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc320105af0 0x7fc320108e90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.136 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.485+0000 7fc312a00700 1 -- 172.21.15.31:0/3103263954 >> 172.21.15.31:0/3103263954 conn(0x7fc3201016a0 msgr2=0x7fc3201021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:40.136 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.486+0000 7fc312a00700 1 -- 172.21.15.31:0/3103263954 shutdown_connections 2024-04-25T01:05:40.136 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:38.486+0000 7fc312a00700 1 -- 172.21.15.31:0/3103263954 wait complete. 2024-04-25T01:05:40.176 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T01:05:40.177 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-04-25T01:05:40.205 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T01:05:40.205 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-25T01:05:40.205 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-04-25T01:05:40.205 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T01:05:40.211 INFO:teuthology.orchestra.run.smithi031.stdout:Ceph Dashboard is now available at: 2024-04-25T01:05:40.212 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:05:40.212 INFO:teuthology.orchestra.run.smithi031.stdout: URL: https://smithi031.front.sepia.ceph.com:8443/ 2024-04-25T01:05:40.212 INFO:teuthology.orchestra.run.smithi031.stdout: User: admin 2024-04-25T01:05:40.212 INFO:teuthology.orchestra.run.smithi031.stdout: Password: wztpkbxddx 2024-04-25T01:05:40.212 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:05:40.213 INFO:teuthology.orchestra.run.smithi031.stdout:Saving cluster configuration to /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/config directory 2024-04-25T01:05:40.215 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling autotune for osd_memory_target 2024-04-25T01:05:41.775 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.871+0000 7f8150800700 1 Processor -- start 2024-04-25T01:05:41.775 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.871+0000 7f8150800700 1 -- start start 2024-04-25T01:05:41.776 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.871+0000 7f8150800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8148081e60 0x7f8148082280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:41.776 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.871+0000 7f8150800700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8148082850 con 0x7f8148081e60 2024-04-25T01:05:41.776 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.872+0000 7f814de00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8148081e60 0x7f8148082280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:41.776 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.872+0000 7f814de00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8148081e60 0x7f8148082280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58854/0 (socket says 172.21.15.31:58854) 2024-04-25T01:05:41.776 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.872+0000 7f814de00700 1 -- 172.21.15.31:0/2833448355 learned_addr learned my addr 172.21.15.31:0/2833448355 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:41.776 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.872+0000 7f814de00700 1 -- 172.21.15.31:0/2833448355 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814810f660 con 0x7f8148081e60 2024-04-25T01:05:41.776 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.873+0000 7f814de00700 1 --2- 172.21.15.31:0/2833448355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8148081e60 0x7f8148082280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f813c0098a0 tx=0x7f813c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf5509cfd2af8e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:41.776 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.873+0000 7f814ca00700 1 -- 172.21.15.31:0/2833448355 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f813c004160 con 0x7f8148081e60 2024-04-25T01:05:41.776 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.873+0000 7f814ca00700 1 -- 172.21.15.31:0/2833448355 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f813c004300 con 0x7f8148081e60 2024-04-25T01:05:41.776 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.873+0000 7f814ca00700 1 -- 172.21.15.31:0/2833448355 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f813c011b20 con 0x7f8148081e60 2024-04-25T01:05:41.776 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.873+0000 7f8150800700 1 -- 172.21.15.31:0/2833448355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8148081e60 msgr2=0x7f8148082280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:41.776 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.873+0000 7f8150800700 1 --2- 172.21.15.31:0/2833448355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8148081e60 0x7f8148082280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f813c0098a0 tx=0x7f813c009bb0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.776 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.874+0000 7f8150800700 1 -- 172.21.15.31:0/2833448355 shutdown_connections 2024-04-25T01:05:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.874+0000 7f8150800700 1 --2- 172.21.15.31:0/2833448355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8148081e60 0x7f8148082280 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.874+0000 7f8150800700 1 -- 172.21.15.31:0/2833448355 >> 172.21.15.31:0/2833448355 conn(0x7f814807d520 msgr2=0x7f814807f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.874+0000 7f8150800700 1 -- 172.21.15.31:0/2833448355 shutdown_connections 2024-04-25T01:05:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.874+0000 7f8150800700 1 -- 172.21.15.31:0/2833448355 wait complete. 2024-04-25T01:05:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.875+0000 7f8150800700 1 Processor -- start 2024-04-25T01:05:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.875+0000 7f8150800700 1 -- start start 2024-04-25T01:05:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.875+0000 7f8150800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8148081e60 0x7f81481a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.875+0000 7f8150800700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8148112a10 con 0x7f8148081e60 2024-04-25T01:05:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.876+0000 7f814de00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8148081e60 0x7f81481a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.876+0000 7f814de00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8148081e60 0x7f81481a2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58866/0 (socket says 172.21.15.31:58866) 2024-04-25T01:05:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.876+0000 7f814de00700 1 -- 172.21.15.31:0/1388796289 learned_addr learned my addr 172.21.15.31:0/1388796289 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.876+0000 7f814de00700 1 -- 172.21.15.31:0/1388796289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81481a2870 con 0x7f8148081e60 2024-04-25T01:05:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.876+0000 7f814de00700 1 --2- 172.21.15.31:0/1388796289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8148081e60 0x7f81481a2330 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f813c011b60 tx=0x7f813c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.877+0000 7f8142a00700 1 -- 172.21.15.31:0/1388796289 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f813c003b60 con 0x7f8148081e60 2024-04-25T01:05:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.877+0000 7f8142a00700 1 -- 172.21.15.31:0/1388796289 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f813c01a910 con 0x7f8148081e60 2024-04-25T01:05:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.877+0000 7f8142a00700 1 -- 172.21.15.31:0/1388796289 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f813c024930 con 0x7f8148081e60 2024-04-25T01:05:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.877+0000 7f8150800700 1 -- 172.21.15.31:0/1388796289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81481a2b00 con 0x7f8148081e60 2024-04-25T01:05:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.877+0000 7f8150800700 1 -- 172.21.15.31:0/1388796289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81481a2f60 con 0x7f8148081e60 2024-04-25T01:05:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.878+0000 7f8142a00700 1 -- 172.21.15.31:0/1388796289 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f813c022020 con 0x7f8148081e60 2024-04-25T01:05:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.878+0000 7f8150800700 1 -- 172.21.15.31:0/1388796289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8148063890 con 0x7f8148081e60 2024-04-25T01:05:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.879+0000 7f8142a00700 1 --2- 172.21.15.31:0/1388796289 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f81300391a0 0x7f813003b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.879+0000 7f8142a00700 1 -- 172.21.15.31:0/1388796289 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f813c04d420 con 0x7f8148081e60 2024-04-25T01:05:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.879+0000 7f814d400700 1 --2- 172.21.15.31:0/1388796289 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f81300391a0 0x7f813003b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.880+0000 7f814d400700 1 --2- 172.21.15.31:0/1388796289 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f81300391a0 0x7f813003b660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8138006fd0 tx=0x7f8138006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:40.885+0000 7f8142a00700 1 -- 172.21.15.31:0/1388796289 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f813c00b3e0 con 0x7f8148081e60 2024-04-25T01:05:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:41.110+0000 7f8150800700 1 -- 172.21.15.31:0/1388796289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f81481a5930 con 0x7f8148081e60 2024-04-25T01:05:41.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:41.110+0000 7f8142a00700 1 -- 172.21.15.31:0/1388796289 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f813c02de10 con 0x7f8148081e60 2024-04-25T01:05:41.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:41.113+0000 7f8150800700 1 -- 172.21.15.31:0/1388796289 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f81300391a0 msgr2=0x7f813003b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:41.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:41.113+0000 7f8150800700 1 --2- 172.21.15.31:0/1388796289 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f81300391a0 0x7f813003b660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8138006fd0 tx=0x7f8138006e40 comp rx=0 tx=0).stop 2024-04-25T01:05:41.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:41.113+0000 7f8150800700 1 -- 172.21.15.31:0/1388796289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8148081e60 msgr2=0x7f81481a2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:41.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:41.113+0000 7f8150800700 1 --2- 172.21.15.31:0/1388796289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8148081e60 0x7f81481a2330 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f813c011b60 tx=0x7f813c011c40 comp rx=0 tx=0).stop 2024-04-25T01:05:41.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:41.114+0000 7f8150800700 1 -- 172.21.15.31:0/1388796289 shutdown_connections 2024-04-25T01:05:41.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:41.114+0000 7f8150800700 1 --2- 172.21.15.31:0/1388796289 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f81300391a0 0x7f813003b660 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:41.114+0000 7f8150800700 1 --2- 172.21.15.31:0/1388796289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8148081e60 0x7f81481a2330 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:41.114+0000 7f8150800700 1 -- 172.21.15.31:0/1388796289 >> 172.21.15.31:0/1388796289 conn(0x7f814807d520 msgr2=0x7f814807dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:41.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:41.114+0000 7f8150800700 1 -- 172.21.15.31:0/1388796289 shutdown_connections 2024-04-25T01:05:41.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:41.114+0000 7f8150800700 1 -- 172.21.15.31:0/1388796289 wait complete. 2024-04-25T01:05:43.459 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.402+0000 7fb7a8800700 1 Processor -- start 2024-04-25T01:05:43.459 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.402+0000 7fb7a8800700 1 -- start start 2024-04-25T01:05:43.459 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.403+0000 7fb7a8800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7a010a150 0x7fb7a010a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:43.459 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.403+0000 7fb7a8800700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a010ab40 con 0x7fb7a010a150 2024-04-25T01:05:43.459 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.403+0000 7fb7a5e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7a010a150 0x7fb7a010a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:43.460 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.403+0000 7fb7a5e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7a010a150 0x7fb7a010a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58872/0 (socket says 172.21.15.31:58872) 2024-04-25T01:05:43.460 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.403+0000 7fb7a5e00700 1 -- 172.21.15.31:0/1043768878 learned_addr learned my addr 172.21.15.31:0/1043768878 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:43.461 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.403+0000 7fb7a5e00700 1 -- 172.21.15.31:0/1043768878 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a010b220 con 0x7fb7a010a150 2024-04-25T01:05:43.461 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.404+0000 7fb7a5e00700 1 --2- 172.21.15.31:0/1043768878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7a010a150 0x7fb7a010a570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb7940098a0 tx=0x7fb794009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26066e7e486714b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:43.461 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.404+0000 7fb7a4a00700 1 -- 172.21.15.31:0/1043768878 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb794004160 con 0x7fb7a010a150 2024-04-25T01:05:43.461 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.404+0000 7fb7a4a00700 1 -- 172.21.15.31:0/1043768878 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb794004300 con 0x7fb7a010a150 2024-04-25T01:05:43.461 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.404+0000 7fb7a4a00700 1 -- 172.21.15.31:0/1043768878 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb794011b20 con 0x7fb7a010a150 2024-04-25T01:05:43.461 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.405+0000 7fb7a8800700 1 -- 172.21.15.31:0/1043768878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7a010a150 msgr2=0x7fb7a010a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:43.461 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.405+0000 7fb7a8800700 1 --2- 172.21.15.31:0/1043768878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7a010a150 0x7fb7a010a570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb7940098a0 tx=0x7fb794009bb0 comp rx=0 tx=0).stop 2024-04-25T01:05:43.463 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.405+0000 7fb7a8800700 1 -- 172.21.15.31:0/1043768878 shutdown_connections 2024-04-25T01:05:43.463 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.405+0000 7fb7a8800700 1 --2- 172.21.15.31:0/1043768878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7a010a150 0x7fb7a010a570 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:43.463 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.405+0000 7fb7a8800700 1 -- 172.21.15.31:0/1043768878 >> 172.21.15.31:0/1043768878 conn(0x7fb7a007d520 msgr2=0x7fb7a007d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:43.463 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.405+0000 7fb7a8800700 1 -- 172.21.15.31:0/1043768878 shutdown_connections 2024-04-25T01:05:43.463 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.405+0000 7fb7a8800700 1 -- 172.21.15.31:0/1043768878 wait complete. 2024-04-25T01:05:43.463 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.406+0000 7fb7a8800700 1 Processor -- start 2024-04-25T01:05:43.463 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.406+0000 7fb7a8800700 1 -- start start 2024-04-25T01:05:43.463 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.407+0000 7fb7a8800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7a010a150 0x7fb7a019df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:43.463 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.407+0000 7fb7a8800700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a010e5d0 con 0x7fb7a010a150 2024-04-25T01:05:43.463 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.407+0000 7fb7a5e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7a010a150 0x7fb7a019df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:43.463 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.407+0000 7fb7a5e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7a010a150 0x7fb7a019df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58882/0 (socket says 172.21.15.31:58882) 2024-04-25T01:05:43.464 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.407+0000 7fb7a5e00700 1 -- 172.21.15.31:0/2344026438 learned_addr learned my addr 172.21.15.31:0/2344026438 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:43.464 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.407+0000 7fb7a5e00700 1 -- 172.21.15.31:0/2344026438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a019e4d0 con 0x7fb7a010a150 2024-04-25T01:05:43.464 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.408+0000 7fb7a5e00700 1 --2- 172.21.15.31:0/2344026438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7a010a150 0x7fb7a019df90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb794000c00 tx=0x7fb794011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:43.466 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.408+0000 7fb79aa00700 1 -- 172.21.15.31:0/2344026438 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb794003b60 con 0x7fb7a010a150 2024-04-25T01:05:43.466 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.408+0000 7fb79aa00700 1 -- 172.21.15.31:0/2344026438 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb79401a910 con 0x7fb7a010a150 2024-04-25T01:05:43.466 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.408+0000 7fb79aa00700 1 -- 172.21.15.31:0/2344026438 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb794024930 con 0x7fb7a010a150 2024-04-25T01:05:43.466 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.408+0000 7fb7a8800700 1 -- 172.21.15.31:0/2344026438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7a019e760 con 0x7fb7a010a150 2024-04-25T01:05:43.466 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.408+0000 7fb7a8800700 1 -- 172.21.15.31:0/2344026438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7a019ebc0 con 0x7fb7a010a150 2024-04-25T01:05:43.466 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.409+0000 7fb79aa00700 1 -- 172.21.15.31:0/2344026438 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb794022020 con 0x7fb7a010a150 2024-04-25T01:05:43.466 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.409+0000 7fb7a8800700 1 -- 172.21.15.31:0/2344026438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7a0063890 con 0x7fb7a010a150 2024-04-25T01:05:43.466 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.410+0000 7fb79aa00700 1 --2- 172.21.15.31:0/2344026438 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb7880391a0 0x7fb78803b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:43.467 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.410+0000 7fb79aa00700 1 -- 172.21.15.31:0/2344026438 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb79404d4b0 con 0x7fb7a010a150 2024-04-25T01:05:43.467 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.410+0000 7fb7a5400700 1 --2- 172.21.15.31:0/2344026438 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb7880391a0 0x7fb78803b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:43.467 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.411+0000 7fb7a5400700 1 --2- 172.21.15.31:0/2344026438 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb7880391a0 0x7fb78803b660 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb790006fd0 tx=0x7fb790006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:43.467 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.416+0000 7fb79aa00700 1 -- 172.21.15.31:0/2344026438 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb79401ab70 con 0x7fb7a010a150 2024-04-25T01:05:43.467 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.762+0000 7fb7a8800700 1 -- 172.21.15.31:0/2344026438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fb7a01a1590 con 0x7fb7a010a150 2024-04-25T01:05:43.467 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.767+0000 7fb79aa00700 1 -- 172.21.15.31:0/2344026438 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fb79402d3f0 con 0x7fb7a010a150 2024-04-25T01:05:43.467 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.771+0000 7fb7a8800700 1 -- 172.21.15.31:0/2344026438 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb7880391a0 msgr2=0x7fb78803b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:43.467 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.771+0000 7fb7a8800700 1 --2- 172.21.15.31:0/2344026438 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb7880391a0 0x7fb78803b660 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb790006fd0 tx=0x7fb790006e40 comp rx=0 tx=0).stop 2024-04-25T01:05:43.467 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.771+0000 7fb7a8800700 1 -- 172.21.15.31:0/2344026438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7a010a150 msgr2=0x7fb7a019df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:43.467 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.771+0000 7fb7a8800700 1 --2- 172.21.15.31:0/2344026438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7a010a150 0x7fb7a019df90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb794000c00 tx=0x7fb794011c10 comp rx=0 tx=0).stop 2024-04-25T01:05:43.467 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.771+0000 7fb7a8800700 1 -- 172.21.15.31:0/2344026438 shutdown_connections 2024-04-25T01:05:43.467 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.771+0000 7fb7a8800700 1 --2- 172.21.15.31:0/2344026438 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb7880391a0 0x7fb78803b660 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:43.468 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.771+0000 7fb7a8800700 1 --2- 172.21.15.31:0/2344026438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7a010a150 0x7fb7a019df90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:43.469 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.771+0000 7fb7a8800700 1 -- 172.21.15.31:0/2344026438 >> 172.21.15.31:0/2344026438 conn(0x7fb7a007d520 msgr2=0x7fb7a0107d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:43.469 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.771+0000 7fb7a8800700 1 -- 172.21.15.31:0/2344026438 shutdown_connections 2024-04-25T01:05:43.469 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T01:05:42.771+0000 7fb7a8800700 1 -- 172.21.15.31:0/2344026438 wait complete. 2024-04-25T01:05:43.469 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T01:05:43.469 INFO:teuthology.orchestra.run.smithi031.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T01:05:43.469 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:05:43.469 INFO:teuthology.orchestra.run.smithi031.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T01:05:43.469 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:05:43.469 INFO:teuthology.orchestra.run.smithi031.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T01:05:43.469 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:05:43.469 INFO:teuthology.orchestra.run.smithi031.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T01:05:43.469 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:05:43.469 INFO:teuthology.orchestra.run.smithi031.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T01:05:43.469 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:05:43.469 INFO:teuthology.orchestra.run.smithi031.stdout: ceph telemetry on 2024-04-25T01:05:43.470 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:05:43.470 INFO:teuthology.orchestra.run.smithi031.stdout:For more information see: 2024-04-25T01:05:43.470 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:05:43.470 INFO:teuthology.orchestra.run.smithi031.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T01:05:43.470 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:05:43.470 INFO:teuthology.orchestra.run.smithi031.stdout:Bootstrap complete. 2024-04-25T01:05:43.576 INFO:tasks.cephadm:Fetching config... 2024-04-25T01:05:43.577 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T01:05:43.577 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T01:05:43.632 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T01:05:43.632 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T01:05:43.632 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T01:05:43.675 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T01:05:43.675 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T01:05:43.675 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T01:05:43.791 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T01:05:43.791 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T01:05:43.791 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T01:05:43.893 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T01:05:43.893 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQwe7FRAzi79xdQ01F4nxQCtlivRddUKmjPHosaGboRFNlS9nebaurEweJKBtyi5qe/fZR5KWpXGJCpcgNVX+bnzEdMla+AggRzOFUlh9Xcp0GMPSEbkljHUJeoHFjIidYTE7JUtBWpC4tpH/UiyvlYnpQgSZk7R0ud55xE6l3VLS7yV3m4F3mi3uif4YTA5DdUrQzd3qLGpp8oVFJwDtUFI8+MV4yqdX7Hl5eBCLLh63l9LfRkpcvwt6/dz4zj43+KklNSC0t+W47OVLu0Us5dfTCAsfLZR1WrSHtvOchNw9f9daJU8wqSFbE5r1U4B6EpXpYDmIrqU0amsk6yavOJekj6ZjvMmSw4xa2uHKs1d8W6CQc2zNDtW3q9uiwb344On6Ja+Dl2BHGFG9+EAm53KM8HGjJ0/bVitF/hF4wIS1obGiNmXA9AzlK2tj+YlIxe45KrTXcnSfxb4lsmZJmLaxj8CRBxnKXQl4DkZeo1hJ7+0Tvf1AV8T+vMpDsodU= ceph-95849b94-029f-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T01:05:44.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:43 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2344026438' entity='client.admin' 2024-04-25T01:05:44.026 INFO:teuthology.orchestra.run.smithi031.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQwe7FRAzi79xdQ01F4nxQCtlivRddUKmjPHosaGboRFNlS9nebaurEweJKBtyi5qe/fZR5KWpXGJCpcgNVX+bnzEdMla+AggRzOFUlh9Xcp0GMPSEbkljHUJeoHFjIidYTE7JUtBWpC4tpH/UiyvlYnpQgSZk7R0ud55xE6l3VLS7yV3m4F3mi3uif4YTA5DdUrQzd3qLGpp8oVFJwDtUFI8+MV4yqdX7Hl5eBCLLh63l9LfRkpcvwt6/dz4zj43+KklNSC0t+W47OVLu0Us5dfTCAsfLZR1WrSHtvOchNw9f9daJU8wqSFbE5r1U4B6EpXpYDmIrqU0amsk6yavOJekj6ZjvMmSw4xa2uHKs1d8W6CQc2zNDtW3q9uiwb344On6Ja+Dl2BHGFG9+EAm53KM8HGjJ0/bVitF/hF4wIS1obGiNmXA9AzlK2tj+YlIxe45KrTXcnSfxb4lsmZJmLaxj8CRBxnKXQl4DkZeo1hJ7+0Tvf1AV8T+vMpDsodU= ceph-95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:05:44.084 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQwe7FRAzi79xdQ01F4nxQCtlivRddUKmjPHosaGboRFNlS9nebaurEweJKBtyi5qe/fZR5KWpXGJCpcgNVX+bnzEdMla+AggRzOFUlh9Xcp0GMPSEbkljHUJeoHFjIidYTE7JUtBWpC4tpH/UiyvlYnpQgSZk7R0ud55xE6l3VLS7yV3m4F3mi3uif4YTA5DdUrQzd3qLGpp8oVFJwDtUFI8+MV4yqdX7Hl5eBCLLh63l9LfRkpcvwt6/dz4zj43+KklNSC0t+W47OVLu0Us5dfTCAsfLZR1WrSHtvOchNw9f9daJU8wqSFbE5r1U4B6EpXpYDmIrqU0amsk6yavOJekj6ZjvMmSw4xa2uHKs1d8W6CQc2zNDtW3q9uiwb344On6Ja+Dl2BHGFG9+EAm53KM8HGjJ0/bVitF/hF4wIS1obGiNmXA9AzlK2tj+YlIxe45KrTXcnSfxb4lsmZJmLaxj8CRBxnKXQl4DkZeo1hJ7+0Tvf1AV8T+vMpDsodU= ceph-95849b94-029f-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T01:05:44.124 INFO:teuthology.orchestra.run.smithi040.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQwe7FRAzi79xdQ01F4nxQCtlivRddUKmjPHosaGboRFNlS9nebaurEweJKBtyi5qe/fZR5KWpXGJCpcgNVX+bnzEdMla+AggRzOFUlh9Xcp0GMPSEbkljHUJeoHFjIidYTE7JUtBWpC4tpH/UiyvlYnpQgSZk7R0ud55xE6l3VLS7yV3m4F3mi3uif4YTA5DdUrQzd3qLGpp8oVFJwDtUFI8+MV4yqdX7Hl5eBCLLh63l9LfRkpcvwt6/dz4zj43+KklNSC0t+W47OVLu0Us5dfTCAsfLZR1WrSHtvOchNw9f9daJU8wqSFbE5r1U4B6EpXpYDmIrqU0amsk6yavOJekj6ZjvMmSw4xa2uHKs1d8W6CQc2zNDtW3q9uiwb344On6Ja+Dl2BHGFG9+EAm53KM8HGjJ0/bVitF/hF4wIS1obGiNmXA9AzlK2tj+YlIxe45KrTXcnSfxb4lsmZJmLaxj8CRBxnKXQl4DkZeo1hJ7+0Tvf1AV8T+vMpDsodU= ceph-95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:05:44.137 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQwe7FRAzi79xdQ01F4nxQCtlivRddUKmjPHosaGboRFNlS9nebaurEweJKBtyi5qe/fZR5KWpXGJCpcgNVX+bnzEdMla+AggRzOFUlh9Xcp0GMPSEbkljHUJeoHFjIidYTE7JUtBWpC4tpH/UiyvlYnpQgSZk7R0ud55xE6l3VLS7yV3m4F3mi3uif4YTA5DdUrQzd3qLGpp8oVFJwDtUFI8+MV4yqdX7Hl5eBCLLh63l9LfRkpcvwt6/dz4zj43+KklNSC0t+W47OVLu0Us5dfTCAsfLZR1WrSHtvOchNw9f9daJU8wqSFbE5r1U4B6EpXpYDmIrqU0amsk6yavOJekj6ZjvMmSw4xa2uHKs1d8W6CQc2zNDtW3q9uiwb344On6Ja+Dl2BHGFG9+EAm53KM8HGjJ0/bVitF/hF4wIS1obGiNmXA9AzlK2tj+YlIxe45KrTXcnSfxb4lsmZJmLaxj8CRBxnKXQl4DkZeo1hJ7+0Tvf1AV8T+vMpDsodU= ceph-95849b94-029f-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T01:05:44.176 INFO:teuthology.orchestra.run.smithi169.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQwe7FRAzi79xdQ01F4nxQCtlivRddUKmjPHosaGboRFNlS9nebaurEweJKBtyi5qe/fZR5KWpXGJCpcgNVX+bnzEdMla+AggRzOFUlh9Xcp0GMPSEbkljHUJeoHFjIidYTE7JUtBWpC4tpH/UiyvlYnpQgSZk7R0ud55xE6l3VLS7yV3m4F3mi3uif4YTA5DdUrQzd3qLGpp8oVFJwDtUFI8+MV4yqdX7Hl5eBCLLh63l9LfRkpcvwt6/dz4zj43+KklNSC0t+W47OVLu0Us5dfTCAsfLZR1WrSHtvOchNw9f9daJU8wqSFbE5r1U4B6EpXpYDmIrqU0amsk6yavOJekj6ZjvMmSw4xa2uHKs1d8W6CQc2zNDtW3q9uiwb344On6Ja+Dl2BHGFG9+EAm53KM8HGjJ0/bVitF/hF4wIS1obGiNmXA9AzlK2tj+YlIxe45KrTXcnSfxb4lsmZJmLaxj8CRBxnKXQl4DkZeo1hJ7+0Tvf1AV8T+vMpDsodU= ceph-95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:05:44.188 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T01:05:44.626 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:05:47.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.504+0000 7f6780800700 1 -- 172.21.15.31:0/3922635178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6778076260 msgr2=0x7f67780746c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.504+0000 7f6780800700 1 --2- 172.21.15.31:0/3922635178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6778076260 0x7f67780746c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f676c009910 tx=0x7f676c009c20 comp rx=0 tx=0).stop 2024-04-25T01:05:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.505+0000 7f6780800700 1 -- 172.21.15.31:0/3922635178 shutdown_connections 2024-04-25T01:05:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.505+0000 7f6780800700 1 --2- 172.21.15.31:0/3922635178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6778076260 0x7f67780746c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.505+0000 7f6780800700 1 -- 172.21.15.31:0/3922635178 >> 172.21.15.31:0/3922635178 conn(0x7f67780fd230 msgr2=0x7f67780ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.505+0000 7f6780800700 1 -- 172.21.15.31:0/3922635178 shutdown_connections 2024-04-25T01:05:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.505+0000 7f6780800700 1 -- 172.21.15.31:0/3922635178 wait complete. 2024-04-25T01:05:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.506+0000 7f6780800700 1 Processor -- start 2024-04-25T01:05:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.507+0000 7f6780800700 1 -- start start 2024-04-25T01:05:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.507+0000 7f6780800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6778076260 0x7f6778199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.507+0000 7f6780800700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6778109f20 con 0x7f6778076260 2024-04-25T01:05:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.507+0000 7f677de00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6778076260 0x7f6778199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.507+0000 7f677de00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6778076260 0x7f6778199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40380/0 (socket says 172.21.15.31:40380) 2024-04-25T01:05:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.507+0000 7f677de00700 1 -- 172.21.15.31:0/2656489373 learned_addr learned my addr 172.21.15.31:0/2656489373 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:47.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.508+0000 7f677de00700 1 -- 172.21.15.31:0/2656489373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6778199aa0 con 0x7f6778076260 2024-04-25T01:05:47.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.508+0000 7f677de00700 1 --2- 172.21.15.31:0/2656489373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6778076260 0x7f6778199560 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f676c006010 tx=0x7f676c005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:47.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.508+0000 7f6772a00700 1 -- 172.21.15.31:0/2656489373 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f676c027070 con 0x7f6778076260 2024-04-25T01:05:47.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.508+0000 7f6772a00700 1 -- 172.21.15.31:0/2656489373 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f676c00be40 con 0x7f6778076260 2024-04-25T01:05:47.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.508+0000 7f6780800700 1 -- 172.21.15.31:0/2656489373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6778199d30 con 0x7f6778076260 2024-04-25T01:05:47.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.508+0000 7f6772a00700 1 -- 172.21.15.31:0/2656489373 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f676c00f460 con 0x7f6778076260 2024-04-25T01:05:47.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.509+0000 7f6780800700 1 -- 172.21.15.31:0/2656489373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f677819ca20 con 0x7f6778076260 2024-04-25T01:05:47.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.510+0000 7f6780800700 1 -- 172.21.15.31:0/2656489373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f677810a6a0 con 0x7f6778076260 2024-04-25T01:05:47.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.510+0000 7f6772a00700 1 -- 172.21.15.31:0/2656489373 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f676c02a070 con 0x7f6778076260 2024-04-25T01:05:47.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.510+0000 7f6772a00700 1 --2- 172.21.15.31:0/2656489373 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6760039150 0x7f676003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:47.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.510+0000 7f6772a00700 1 -- 172.21.15.31:0/2656489373 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f676c01e030 con 0x7f6778076260 2024-04-25T01:05:47.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.511+0000 7f677d400700 1 --2- 172.21.15.31:0/2656489373 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6760039150 0x7f676003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:47.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.511+0000 7f677d400700 1 --2- 172.21.15.31:0/2656489373 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6760039150 0x7f676003b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6768006fd0 tx=0x7f6768006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:47.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.516+0000 7f6772a00700 1 -- 172.21.15.31:0/2656489373 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f676c013070 con 0x7f6778076260 2024-04-25T01:05:47.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.744+0000 7f6780800700 1 -- 172.21.15.31:0/2656489373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f677802dc40 con 0x7f6778076260 2024-04-25T01:05:47.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.748+0000 7f6772a00700 1 -- 172.21.15.31:0/2656489373 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f676c031020 con 0x7f6778076260 2024-04-25T01:05:47.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.751+0000 7f6780800700 1 -- 172.21.15.31:0/2656489373 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6760039150 msgr2=0x7f676003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:47.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.751+0000 7f6780800700 1 --2- 172.21.15.31:0/2656489373 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6760039150 0x7f676003b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6768006fd0 tx=0x7f6768006e40 comp rx=0 tx=0).stop 2024-04-25T01:05:47.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.751+0000 7f6780800700 1 -- 172.21.15.31:0/2656489373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6778076260 msgr2=0x7f6778199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:47.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.751+0000 7f6780800700 1 --2- 172.21.15.31:0/2656489373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6778076260 0x7f6778199560 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f676c006010 tx=0x7f676c005050 comp rx=0 tx=0).stop 2024-04-25T01:05:47.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.752+0000 7f6780800700 1 -- 172.21.15.31:0/2656489373 shutdown_connections 2024-04-25T01:05:47.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.752+0000 7f6780800700 1 --2- 172.21.15.31:0/2656489373 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6760039150 0x7f676003b610 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:47.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.752+0000 7f6780800700 1 --2- 172.21.15.31:0/2656489373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6778076260 0x7f6778199560 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:47.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.752+0000 7f6780800700 1 -- 172.21.15.31:0/2656489373 >> 172.21.15.31:0/2656489373 conn(0x7f67780fd230 msgr2=0x7f67780fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:47.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.752+0000 7f6780800700 1 -- 172.21.15.31:0/2656489373 shutdown_connections 2024-04-25T01:05:47.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:47.752+0000 7f6780800700 1 -- 172.21.15.31:0/2656489373 wait complete. 2024-04-25T01:05:48.189 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:47 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:05:48.189 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:47 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:05:48.189 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:47 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi031", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:05:48.189 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:47 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:05:48.189 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:47 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:05:48.189 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:47 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:05:48.189 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:47 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2656489373' entity='client.admin' 2024-04-25T01:05:48.189 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:47 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:05:48.189 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:47 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:05:48.596 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T01:05:48.597 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T01:05:49.667 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:05:51.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.694+0000 7f3839000700 1 -- 172.21.15.31:0/3406076535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38340761e0 msgr2=0x7f3834074640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:51.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.694+0000 7f3839000700 1 --2- 172.21.15.31:0/3406076535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38340761e0 0x7f3834074640 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3824009910 tx=0x7f3824009c20 comp rx=0 tx=0).stop 2024-04-25T01:05:51.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.694+0000 7f3839000700 1 -- 172.21.15.31:0/3406076535 shutdown_connections 2024-04-25T01:05:51.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.694+0000 7f3839000700 1 --2- 172.21.15.31:0/3406076535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38340761e0 0x7f3834074640 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:51.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.694+0000 7f3839000700 1 -- 172.21.15.31:0/3406076535 >> 172.21.15.31:0/3406076535 conn(0x7f38340fd0e0 msgr2=0x7f38340ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:51.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.694+0000 7f3839000700 1 -- 172.21.15.31:0/3406076535 shutdown_connections 2024-04-25T01:05:51.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.695+0000 7f3839000700 1 -- 172.21.15.31:0/3406076535 wait complete. 2024-04-25T01:05:51.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.696+0000 7f3839000700 1 Processor -- start 2024-04-25T01:05:51.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.696+0000 7f3839000700 1 -- start start 2024-04-25T01:05:51.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.696+0000 7f3839000700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38340761e0 0x7f38341993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:51.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.696+0000 7f3839000700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3834109d60 con 0x7f38340761e0 2024-04-25T01:05:51.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.696+0000 7f3833400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38340761e0 0x7f38341993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:51.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.696+0000 7f3833400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38340761e0 0x7f38341993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40398/0 (socket says 172.21.15.31:40398) 2024-04-25T01:05:51.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.696+0000 7f3833400700 1 -- 172.21.15.31:0/297226183 learned_addr learned my addr 172.21.15.31:0/297226183 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:51.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.697+0000 7f3833400700 1 -- 172.21.15.31:0/297226183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3834199920 con 0x7f38340761e0 2024-04-25T01:05:51.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.697+0000 7f3833400700 1 --2- 172.21.15.31:0/297226183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38340761e0 0x7f38341993e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3824004fa0 tx=0x7f3824005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:51.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.697+0000 7f382be00700 1 -- 172.21.15.31:0/297226183 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3824027070 con 0x7f38340761e0 2024-04-25T01:05:51.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.698+0000 7f3839000700 1 -- 172.21.15.31:0/297226183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3834199bb0 con 0x7f38340761e0 2024-04-25T01:05:51.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.698+0000 7f382be00700 1 -- 172.21.15.31:0/297226183 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f382400be40 con 0x7f38340761e0 2024-04-25T01:05:51.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.698+0000 7f382be00700 1 -- 172.21.15.31:0/297226183 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f382400f460 con 0x7f38340761e0 2024-04-25T01:05:51.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.698+0000 7f3839000700 1 -- 172.21.15.31:0/297226183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f383419c8a0 con 0x7f38340761e0 2024-04-25T01:05:51.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.699+0000 7f3839000700 1 -- 172.21.15.31:0/297226183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3814005360 con 0x7f38340761e0 2024-04-25T01:05:51.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.699+0000 7f382be00700 1 -- 172.21.15.31:0/297226183 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f382402a070 con 0x7f38340761e0 2024-04-25T01:05:51.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.699+0000 7f382be00700 1 --2- 172.21.15.31:0/297226183 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f38180390f0 0x7f381803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:51.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.699+0000 7f382be00700 1 -- 172.21.15.31:0/297226183 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f382401e030 con 0x7f38340761e0 2024-04-25T01:05:51.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.700+0000 7f3832a00700 1 --2- 172.21.15.31:0/297226183 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f38180390f0 0x7f381803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:51.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.700+0000 7f3832a00700 1 --2- 172.21.15.31:0/297226183 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f38180390f0 0x7f381803b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3820006fd0 tx=0x7f3820006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:51.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.705+0000 7f382be00700 1 -- 172.21.15.31:0/297226183 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3824013070 con 0x7f38340761e0 2024-04-25T01:05:51.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.944+0000 7f3839000700 1 -- 172.21.15.31:0/297226183 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f3814000bf0 con 0x7f38180390f0 2024-04-25T01:05:51.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.951+0000 7f382be00700 1 -- 172.21.15.31:0/297226183 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3814000bf0 con 0x7f38180390f0 2024-04-25T01:05:51.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.954+0000 7f3839000700 1 -- 172.21.15.31:0/297226183 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f38180390f0 msgr2=0x7f381803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:51.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.954+0000 7f3839000700 1 --2- 172.21.15.31:0/297226183 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f38180390f0 0x7f381803b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3820006fd0 tx=0x7f3820006e40 comp rx=0 tx=0).stop 2024-04-25T01:05:51.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.954+0000 7f3839000700 1 -- 172.21.15.31:0/297226183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38340761e0 msgr2=0x7f38341993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:51.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.954+0000 7f3839000700 1 --2- 172.21.15.31:0/297226183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38340761e0 0x7f38341993e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3824004fa0 tx=0x7f3824005080 comp rx=0 tx=0).stop 2024-04-25T01:05:51.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.954+0000 7f3839000700 1 -- 172.21.15.31:0/297226183 shutdown_connections 2024-04-25T01:05:51.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.954+0000 7f3839000700 1 --2- 172.21.15.31:0/297226183 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f38180390f0 0x7f381803b5b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:51.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.954+0000 7f3839000700 1 --2- 172.21.15.31:0/297226183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38340761e0 0x7f38341993e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:51.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.954+0000 7f3839000700 1 -- 172.21.15.31:0/297226183 >> 172.21.15.31:0/297226183 conn(0x7f38340fd0e0 msgr2=0x7f38340fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:51.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.954+0000 7f3839000700 1 -- 172.21.15.31:0/297226183 shutdown_connections 2024-04-25T01:05:51.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:51.954+0000 7f3839000700 1 -- 172.21.15.31:0/297226183 wait complete. 2024-04-25T01:05:52.697 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi040 2024-04-25T01:05:52.698 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T01:05:52.698 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/etc/ceph/ceph.conf 2024-04-25T01:05:52.718 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T01:05:52.718 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:05:52.775 INFO:tasks.cephadm:Adding host smithi040 to orchestrator... 2024-04-25T01:05:52.776 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph orch host add smithi040 2024-04-25T01:05:53.192 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:05:53.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:52 smithi031 ceph-mon[28053]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:05:53.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:52 smithi031 ceph-mon[28053]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:05:53.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:52 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:05:53.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:52 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:05:53.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:52 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:05:53.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:52 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:05:53.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:52 smithi031 ceph-mon[28053]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-25T01:05:53.265 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:52 smithi031 ceph-mon[28053]: Updating smithi031:/var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:05:54.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:53 smithi031 ceph-mon[28053]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:05:54.216 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:53 smithi031 ceph-mon[28053]: Updating smithi031:/var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T01:05:55.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.330+0000 7fb187a00700 1 -- 172.21.15.31:0/3131690589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb180074600 msgr2=0x7fb1800749e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:55.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.330+0000 7fb187a00700 1 --2- 172.21.15.31:0/3131690589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb180074600 0x7fb1800749e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb174009910 tx=0x7fb174009c20 comp rx=0 tx=0).stop 2024-04-25T01:05:55.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.330+0000 7fb187a00700 1 -- 172.21.15.31:0/3131690589 shutdown_connections 2024-04-25T01:05:55.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.330+0000 7fb187a00700 1 --2- 172.21.15.31:0/3131690589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb180074600 0x7fb1800749e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:55.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.330+0000 7fb187a00700 1 -- 172.21.15.31:0/3131690589 >> 172.21.15.31:0/3131690589 conn(0x7fb1800fd770 msgr2=0x7fb1800ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:55.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.330+0000 7fb187a00700 1 -- 172.21.15.31:0/3131690589 shutdown_connections 2024-04-25T01:05:55.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.331+0000 7fb187a00700 1 -- 172.21.15.31:0/3131690589 wait complete. 2024-04-25T01:05:55.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.332+0000 7fb187a00700 1 Processor -- start 2024-04-25T01:05:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.332+0000 7fb187a00700 1 -- start start 2024-04-25T01:05:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.332+0000 7fb187a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb180074600 0x7fb18019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.332+0000 7fb187a00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb18010e4e0 con 0x7fb180074600 2024-04-25T01:05:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.333+0000 7fb185000700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb180074600 0x7fb18019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.333+0000 7fb185000700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb180074600 0x7fb18019f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40414/0 (socket says 172.21.15.31:40414) 2024-04-25T01:05:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.333+0000 7fb185000700 1 -- 172.21.15.31:0/439890204 learned_addr learned my addr 172.21.15.31:0/439890204 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:05:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.333+0000 7fb185000700 1 -- 172.21.15.31:0/439890204 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb18019fb90 con 0x7fb180074600 2024-04-25T01:05:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.333+0000 7fb185000700 1 --2- 172.21.15.31:0/439890204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb180074600 0x7fb18019f650 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb174006010 tx=0x7fb174005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.334+0000 7fb17d600700 1 -- 172.21.15.31:0/439890204 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb174026070 con 0x7fb180074600 2024-04-25T01:05:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.334+0000 7fb17d600700 1 -- 172.21.15.31:0/439890204 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb17400bc30 con 0x7fb180074600 2024-04-25T01:05:55.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.334+0000 7fb187a00700 1 -- 172.21.15.31:0/439890204 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb18019fe20 con 0x7fb180074600 2024-04-25T01:05:55.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.334+0000 7fb17d600700 1 -- 172.21.15.31:0/439890204 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1740039a0 con 0x7fb180074600 2024-04-25T01:05:55.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.334+0000 7fb187a00700 1 -- 172.21.15.31:0/439890204 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1801997d0 con 0x7fb180074600 2024-04-25T01:05:55.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.335+0000 7fb187a00700 1 -- 172.21.15.31:0/439890204 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb180075cb0 con 0x7fb180074600 2024-04-25T01:05:55.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.335+0000 7fb17d600700 1 -- 172.21.15.31:0/439890204 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb174029070 con 0x7fb180074600 2024-04-25T01:05:55.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.336+0000 7fb17d600700 1 --2- 172.21.15.31:0/439890204 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb168039150 0x7fb16803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:55.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.336+0000 7fb17d600700 1 -- 172.21.15.31:0/439890204 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb17401e070 con 0x7fb180074600 2024-04-25T01:05:55.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.336+0000 7fb17fe00700 1 --2- 172.21.15.31:0/439890204 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb168039150 0x7fb16803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:55.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.337+0000 7fb17fe00700 1 --2- 172.21.15.31:0/439890204 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb168039150 0x7fb16803b610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb16c006fd0 tx=0x7fb16c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:55.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.341+0000 7fb17d600700 1 -- 172.21.15.31:0/439890204 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb174013070 con 0x7fb180074600 2024-04-25T01:05:55.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:05:55.578+0000 7fb187a00700 1 -- 172.21.15.31:0/439890204 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi040", "target": ["mon-mgr", ""]}) -- 0x7fb18019ad00 con 0x7fb168039150 2024-04-25T01:05:55.765 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:55 smithi031 ceph-mon[28053]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:05:55.765 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:55 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:05:55.765 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:55 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:05:55.765 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:55 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:05:56.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:56 smithi031 ceph-mon[28053]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi040", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:05:57.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:57 smithi031 ceph-mon[28053]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:05:57.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:57 smithi031 ceph-mon[28053]: Deploying cephadm binary to smithi040 2024-04-25T01:05:59.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:05:59 smithi031 ceph-mon[28053]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:00.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:00.432+0000 7fb17d600700 1 -- 172.21.15.31:0/439890204 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fb18019ad00 con 0x7fb168039150 2024-04-25T01:06:00.429 INFO:teuthology.orchestra.run.smithi031.stdout:Added host 'smithi040' with addr '172.21.15.40' 2024-04-25T01:06:00.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:00.436+0000 7fb187a00700 1 -- 172.21.15.31:0/439890204 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb168039150 msgr2=0x7fb16803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:00.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:00.436+0000 7fb187a00700 1 --2- 172.21.15.31:0/439890204 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb168039150 0x7fb16803b610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb16c006fd0 tx=0x7fb16c006e40 comp rx=0 tx=0).stop 2024-04-25T01:06:00.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:00.436+0000 7fb187a00700 1 -- 172.21.15.31:0/439890204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb180074600 msgr2=0x7fb18019f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:00.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:00.436+0000 7fb187a00700 1 --2- 172.21.15.31:0/439890204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb180074600 0x7fb18019f650 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb174006010 tx=0x7fb174005030 comp rx=0 tx=0).stop 2024-04-25T01:06:00.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:00.436+0000 7fb187a00700 1 -- 172.21.15.31:0/439890204 shutdown_connections 2024-04-25T01:06:00.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:00.436+0000 7fb187a00700 1 --2- 172.21.15.31:0/439890204 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb168039150 0x7fb16803b610 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:00.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:00.436+0000 7fb187a00700 1 --2- 172.21.15.31:0/439890204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb180074600 0x7fb18019f650 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:00.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:00.436+0000 7fb187a00700 1 -- 172.21.15.31:0/439890204 >> 172.21.15.31:0/439890204 conn(0x7fb1800fd770 msgr2=0x7fb180105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:06:00.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:00.437+0000 7fb187a00700 1 -- 172.21.15.31:0/439890204 shutdown_connections 2024-04-25T01:06:00.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:00.437+0000 7fb187a00700 1 -- 172.21.15.31:0/439890204 wait complete. 2024-04-25T01:06:01.207 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T01:06:01.664 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:06:01.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:01 smithi031 ceph-mon[28053]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:01.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:01 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:01.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:01 smithi031 ceph-mon[28053]: Added host smithi040 2024-04-25T01:06:01.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:01 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:06:01.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:01 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:02.829 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:02 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:03.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.691+0000 7f4992200700 1 -- 172.21.15.31:0/2276916765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f498c100bf0 msgr2=0x7f498c100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:03.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.691+0000 7f4992200700 1 --2- 172.21.15.31:0/2276916765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f498c100bf0 0x7f498c100fd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f497c009910 tx=0x7f497c009c20 comp rx=0 tx=0).stop 2024-04-25T01:06:03.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.692+0000 7f4992200700 1 -- 172.21.15.31:0/2276916765 shutdown_connections 2024-04-25T01:06:03.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.692+0000 7f4992200700 1 --2- 172.21.15.31:0/2276916765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f498c100bf0 0x7f498c100fd0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:03.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.692+0000 7f4992200700 1 -- 172.21.15.31:0/2276916765 >> 172.21.15.31:0/2276916765 conn(0x7f498c0fc630 msgr2=0x7f498c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:06:03.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.692+0000 7f4992200700 1 -- 172.21.15.31:0/2276916765 shutdown_connections 2024-04-25T01:06:03.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.692+0000 7f4992200700 1 -- 172.21.15.31:0/2276916765 wait complete. 2024-04-25T01:06:03.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.693+0000 7f4992200700 1 Processor -- start 2024-04-25T01:06:03.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.693+0000 7f4992200700 1 -- start start 2024-04-25T01:06:03.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.693+0000 7f4992200700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f498c100bf0 0x7f498c19de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:03.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.693+0000 7f4992200700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f498c1128d0 con 0x7f498c100bf0 2024-04-25T01:06:03.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.694+0000 7f498b400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f498c100bf0 0x7f498c19de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:03.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.694+0000 7f498b400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f498c100bf0 0x7f498c19de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34984/0 (socket says 172.21.15.31:34984) 2024-04-25T01:06:03.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.694+0000 7f498b400700 1 -- 172.21.15.31:0/2781991677 learned_addr learned my addr 172.21.15.31:0/2781991677 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:06:03.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.694+0000 7f498b400700 1 -- 172.21.15.31:0/2781991677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498c19e380 con 0x7f498c100bf0 2024-04-25T01:06:03.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.694+0000 7f498b400700 1 --2- 172.21.15.31:0/2781991677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f498c100bf0 0x7f498c19de40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f497c006010 tx=0x7f497c005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:06:03.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.695+0000 7f4983e00700 1 -- 172.21.15.31:0/2781991677 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f497c026070 con 0x7f498c100bf0 2024-04-25T01:06:03.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.695+0000 7f4983e00700 1 -- 172.21.15.31:0/2781991677 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f497c00bc30 con 0x7f498c100bf0 2024-04-25T01:06:03.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.695+0000 7f4992200700 1 -- 172.21.15.31:0/2781991677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f498c19e610 con 0x7f498c100bf0 2024-04-25T01:06:03.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.695+0000 7f4983e00700 1 -- 172.21.15.31:0/2781991677 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f497c003ab0 con 0x7f498c100bf0 2024-04-25T01:06:03.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.695+0000 7f4992200700 1 -- 172.21.15.31:0/2781991677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f498c19ea70 con 0x7f498c100bf0 2024-04-25T01:06:03.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.696+0000 7f4983e00700 1 -- 172.21.15.31:0/2781991677 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f497c029070 con 0x7f498c100bf0 2024-04-25T01:06:03.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.696+0000 7f4992200700 1 -- 172.21.15.31:0/2781991677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f498c06a920 con 0x7f498c100bf0 2024-04-25T01:06:03.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.697+0000 7f4983e00700 1 --2- 172.21.15.31:0/2781991677 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f49700390f0 0x7f497003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:03.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.697+0000 7f4983e00700 1 -- 172.21.15.31:0/2781991677 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f497c01e070 con 0x7f498c100bf0 2024-04-25T01:06:03.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.697+0000 7f498aa00700 1 --2- 172.21.15.31:0/2781991677 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f49700390f0 0x7f497003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:03.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.698+0000 7f498aa00700 1 --2- 172.21.15.31:0/2781991677 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f49700390f0 0x7f497003b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4978006fd0 tx=0x7f4978006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:06:03.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.702+0000 7f4983e00700 1 -- 172.21.15.31:0/2781991677 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f497c013070 con 0x7f498c100bf0 2024-04-25T01:06:03.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.943+0000 7f4992200700 1 -- 172.21.15.31:0/2781991677 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f498c19f210 con 0x7f49700390f0 2024-04-25T01:06:03.939 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:03 smithi031 ceph-mon[28053]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:03.939 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:03 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:03.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.945+0000 7f4983e00700 1 -- 172.21.15.31:0/2781991677 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f498c19f210 con 0x7f49700390f0 2024-04-25T01:06:03.941 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:06:03.941 INFO:teuthology.orchestra.run.smithi031.stdout:[{"addr": "172.21.15.31", "hostname": "smithi031", "labels": [], "status": ""}, {"addr": "172.21.15.40", "hostname": "smithi040", "labels": [], "status": ""}] 2024-04-25T01:06:03.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.949+0000 7f4992200700 1 -- 172.21.15.31:0/2781991677 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f49700390f0 msgr2=0x7f497003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:03.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.949+0000 7f4992200700 1 --2- 172.21.15.31:0/2781991677 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f49700390f0 0x7f497003b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4978006fd0 tx=0x7f4978006e40 comp rx=0 tx=0).stop 2024-04-25T01:06:03.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.949+0000 7f4992200700 1 -- 172.21.15.31:0/2781991677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f498c100bf0 msgr2=0x7f498c19de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:03.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.949+0000 7f4992200700 1 --2- 172.21.15.31:0/2781991677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f498c100bf0 0x7f498c19de40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f497c006010 tx=0x7f497c005050 comp rx=0 tx=0).stop 2024-04-25T01:06:03.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.949+0000 7f4992200700 1 -- 172.21.15.31:0/2781991677 shutdown_connections 2024-04-25T01:06:03.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.949+0000 7f4992200700 1 --2- 172.21.15.31:0/2781991677 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f49700390f0 0x7f497003b5b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:03.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.949+0000 7f4992200700 1 --2- 172.21.15.31:0/2781991677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f498c100bf0 0x7f498c19de40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:03.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.949+0000 7f4992200700 1 -- 172.21.15.31:0/2781991677 >> 172.21.15.31:0/2781991677 conn(0x7f498c0fc630 msgr2=0x7f498c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:06:03.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.949+0000 7f4992200700 1 -- 172.21.15.31:0/2781991677 shutdown_connections 2024-04-25T01:06:03.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:03.950+0000 7f4992200700 1 -- 172.21.15.31:0/2781991677 wait complete. 2024-04-25T01:06:04.685 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi169 2024-04-25T01:06:04.685 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T01:06:04.685 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/etc/ceph/ceph.conf 2024-04-25T01:06:04.705 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T01:06:04.705 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:06:04.762 INFO:tasks.cephadm:Adding host smithi169 to orchestrator... 2024-04-25T01:06:04.762 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph orch host add smithi169 2024-04-25T01:06:05.259 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:06:05.684 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:05 smithi031 ceph-mon[28053]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:06:05.684 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:05 smithi031 ceph-mon[28053]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:07.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.190+0000 7ff8ce800700 1 -- 172.21.15.31:0/528690209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c81040c0 msgr2=0x7ff8c81044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:07.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.190+0000 7ff8ce800700 1 --2- 172.21.15.31:0/528690209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c81040c0 0x7ff8c81044a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009910 tx=0x7ff8b8009c20 comp rx=0 tx=0).stop 2024-04-25T01:06:07.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.190+0000 7ff8ce800700 1 -- 172.21.15.31:0/528690209 shutdown_connections 2024-04-25T01:06:07.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.190+0000 7ff8ce800700 1 --2- 172.21.15.31:0/528690209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c81040c0 0x7ff8c81044a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:07.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.190+0000 7ff8ce800700 1 -- 172.21.15.31:0/528690209 >> 172.21.15.31:0/528690209 conn(0x7ff8c80ff950 msgr2=0x7ff8c8101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:06:07.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.190+0000 7ff8ce800700 1 -- 172.21.15.31:0/528690209 shutdown_connections 2024-04-25T01:06:07.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.191+0000 7ff8ce800700 1 -- 172.21.15.31:0/528690209 wait complete. 2024-04-25T01:06:07.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.191+0000 7ff8ce800700 1 Processor -- start 2024-04-25T01:06:07.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.192+0000 7ff8ce800700 1 -- start start 2024-04-25T01:06:07.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.192+0000 7ff8ce800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c81040c0 0x7ff8c819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:07.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.192+0000 7ff8ce800700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c810e490 con 0x7ff8c81040c0 2024-04-25T01:06:07.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.192+0000 7ff8c7400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c81040c0 0x7ff8c819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:07.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.192+0000 7ff8c7400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c81040c0 0x7ff8c819f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52056/0 (socket says 172.21.15.31:52056) 2024-04-25T01:06:07.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.192+0000 7ff8c7400700 1 -- 172.21.15.31:0/1930993550 learned_addr learned my addr 172.21.15.31:0/1930993550 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:06:07.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.193+0000 7ff8c7400700 1 -- 172.21.15.31:0/1930993550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c819fb40 con 0x7ff8c81040c0 2024-04-25T01:06:07.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.193+0000 7ff8c7400700 1 --2- 172.21.15.31:0/1930993550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c81040c0 0x7ff8c819f600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8006010 tx=0x7ff8b8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:06:07.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.193+0000 7ff8bfe00700 1 -- 172.21.15.31:0/1930993550 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8b8026070 con 0x7ff8c81040c0 2024-04-25T01:06:07.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.193+0000 7ff8bfe00700 1 -- 172.21.15.31:0/1930993550 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8b800bbd0 con 0x7ff8c81040c0 2024-04-25T01:06:07.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.193+0000 7ff8ce800700 1 -- 172.21.15.31:0/1930993550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c819fe30 con 0x7ff8c81040c0 2024-04-25T01:06:07.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.193+0000 7ff8bfe00700 1 -- 172.21.15.31:0/1930993550 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8b8003910 con 0x7ff8c81040c0 2024-04-25T01:06:07.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.193+0000 7ff8ce800700 1 -- 172.21.15.31:0/1930993550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c8199780 con 0x7ff8c81040c0 2024-04-25T01:06:07.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.195+0000 7ff8bfe00700 1 -- 172.21.15.31:0/1930993550 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff8b8029070 con 0x7ff8c81040c0 2024-04-25T01:06:07.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.195+0000 7ff8ce800700 1 -- 172.21.15.31:0/1930993550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8c81055a0 con 0x7ff8c81040c0 2024-04-25T01:06:07.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.195+0000 7ff8bfe00700 1 --2- 172.21.15.31:0/1930993550 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff8ac039150 0x7ff8ac03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:07.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.195+0000 7ff8bfe00700 1 -- 172.21.15.31:0/1930993550 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff8b801e070 con 0x7ff8c81040c0 2024-04-25T01:06:07.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.195+0000 7ff8c6a00700 1 --2- 172.21.15.31:0/1930993550 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff8ac039150 0x7ff8ac03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:07.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.196+0000 7ff8c6a00700 1 --2- 172.21.15.31:0/1930993550 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff8ac039150 0x7ff8ac03b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4006fd0 tx=0x7ff8b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:06:07.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.201+0000 7ff8bfe00700 1 -- 172.21.15.31:0/1930993550 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff8b8013070 con 0x7ff8c81040c0 2024-04-25T01:06:07.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:07.434+0000 7ff8ce800700 1 -- 172.21.15.31:0/1930993550 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi169", "target": ["mon-mgr", ""]}) -- 0x7ff8c819acb0 con 0x7ff8ac039150 2024-04-25T01:06:07.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:07 smithi031 ceph-mon[28053]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:07.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:07 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:07.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:07 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:07.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:07 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:07.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:07 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:07.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:07 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi040", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:06:07.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:07 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:07.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:07 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:06:07.765 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:07 smithi031 ceph-mon[28053]: Updating smithi040:/etc/ceph/ceph.conf 2024-04-25T01:06:07.765 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:07 smithi031 ceph-mon[28053]: Updating smithi040:/var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:06:09.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:08 smithi031 ceph-mon[28053]: Updating smithi040:/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:06:09.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:08 smithi031 ceph-mon[28053]: Updating smithi040:/var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T01:06:09.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:08 smithi031 ceph-mon[28053]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi169", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:06:09.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:08 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:09.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:08 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:09.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:08 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:10.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:09 smithi031 ceph-mon[28053]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:10.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:09 smithi031 ceph-mon[28053]: Deploying cephadm binary to smithi169 2024-04-25T01:06:12.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:11 smithi031 ceph-mon[28053]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:12.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:12.377+0000 7ff8bfe00700 1 -- 172.21.15.31:0/1930993550 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ff8c819acb0 con 0x7ff8ac039150 2024-04-25T01:06:12.373 INFO:teuthology.orchestra.run.smithi031.stdout:Added host 'smithi169' with addr '172.21.15.169' 2024-04-25T01:06:12.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:12.380+0000 7ff8ce800700 1 -- 172.21.15.31:0/1930993550 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff8ac039150 msgr2=0x7ff8ac03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:12.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:12.380+0000 7ff8ce800700 1 --2- 172.21.15.31:0/1930993550 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff8ac039150 0x7ff8ac03b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4006fd0 tx=0x7ff8b4006e40 comp rx=0 tx=0).stop 2024-04-25T01:06:12.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:12.381+0000 7ff8ce800700 1 -- 172.21.15.31:0/1930993550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c81040c0 msgr2=0x7ff8c819f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:12.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:12.381+0000 7ff8ce800700 1 --2- 172.21.15.31:0/1930993550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c81040c0 0x7ff8c819f600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8006010 tx=0x7ff8b8005030 comp rx=0 tx=0).stop 2024-04-25T01:06:12.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:12.381+0000 7ff8ce800700 1 -- 172.21.15.31:0/1930993550 shutdown_connections 2024-04-25T01:06:12.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:12.381+0000 7ff8ce800700 1 --2- 172.21.15.31:0/1930993550 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff8ac039150 0x7ff8ac03b610 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:12.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:12.381+0000 7ff8ce800700 1 --2- 172.21.15.31:0/1930993550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c81040c0 0x7ff8c819f600 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:12.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:12.381+0000 7ff8ce800700 1 -- 172.21.15.31:0/1930993550 >> 172.21.15.31:0/1930993550 conn(0x7ff8c80ff950 msgr2=0x7ff8c8105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:06:12.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:12.381+0000 7ff8ce800700 1 -- 172.21.15.31:0/1930993550 shutdown_connections 2024-04-25T01:06:12.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:12.381+0000 7ff8ce800700 1 -- 172.21.15.31:0/1930993550 wait complete. 2024-04-25T01:06:13.216 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T01:06:13.513 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:13 smithi031 ceph-mon[28053]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:13.513 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:13.513 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:13 smithi031 ceph-mon[28053]: Added host smithi169 2024-04-25T01:06:13.514 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:06:13.514 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:13.664 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:06:14.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:14 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:15.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.693+0000 7ffa17200700 1 -- 172.21.15.31:0/2407390518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa100745d0 msgr2=0x7ffa100749b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:15.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.693+0000 7ffa17200700 1 --2- 172.21.15.31:0/2407390518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa100745d0 0x7ffa100749b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009910 tx=0x7ffa04009c20 comp rx=0 tx=0).stop 2024-04-25T01:06:15.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.693+0000 7ffa17200700 1 -- 172.21.15.31:0/2407390518 shutdown_connections 2024-04-25T01:06:15.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.693+0000 7ffa17200700 1 --2- 172.21.15.31:0/2407390518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa100745d0 0x7ffa100749b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:15.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.693+0000 7ffa17200700 1 -- 172.21.15.31:0/2407390518 >> 172.21.15.31:0/2407390518 conn(0x7ffa100fd590 msgr2=0x7ffa100ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:06:15.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.693+0000 7ffa17200700 1 -- 172.21.15.31:0/2407390518 shutdown_connections 2024-04-25T01:06:15.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.693+0000 7ffa17200700 1 -- 172.21.15.31:0/2407390518 wait complete. 2024-04-25T01:06:15.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.694+0000 7ffa17200700 1 Processor -- start 2024-04-25T01:06:15.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.695+0000 7ffa17200700 1 -- start start 2024-04-25T01:06:15.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.695+0000 7ffa17200700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa100745d0 0x7ffa1019dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:15.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.695+0000 7ffa17200700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ffa10112720 con 0x7ffa100745d0 2024-04-25T01:06:15.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.695+0000 7ffa15e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa100745d0 0x7ffa1019dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:15.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.695+0000 7ffa15e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa100745d0 0x7ffa1019dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52072/0 (socket says 172.21.15.31:52072) 2024-04-25T01:06:15.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.695+0000 7ffa15e00700 1 -- 172.21.15.31:0/1007225689 learned_addr learned my addr 172.21.15.31:0/1007225689 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:06:15.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.696+0000 7ffa15e00700 1 -- 172.21.15.31:0/1007225689 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1019e1c0 con 0x7ffa100745d0 2024-04-25T01:06:15.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.696+0000 7ffa15e00700 1 --2- 172.21.15.31:0/1007225689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa100745d0 0x7ffa1019dc80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffa04006010 tx=0x7ffa04005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:06:15.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.696+0000 7ffa0aa00700 1 -- 172.21.15.31:0/1007225689 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa04026070 con 0x7ffa100745d0 2024-04-25T01:06:15.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.696+0000 7ffa0aa00700 1 -- 172.21.15.31:0/1007225689 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa0400bc30 con 0x7ffa100745d0 2024-04-25T01:06:15.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.696+0000 7ffa0aa00700 1 -- 172.21.15.31:0/1007225689 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa040039a0 con 0x7ffa100745d0 2024-04-25T01:06:15.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.696+0000 7ffa17200700 1 -- 172.21.15.31:0/1007225689 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa1019e450 con 0x7ffa100745d0 2024-04-25T01:06:15.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.696+0000 7ffa17200700 1 -- 172.21.15.31:0/1007225689 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa1019e8b0 con 0x7ffa100745d0 2024-04-25T01:06:15.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.697+0000 7ffa0aa00700 1 -- 172.21.15.31:0/1007225689 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ffa04029070 con 0x7ffa100745d0 2024-04-25T01:06:15.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.697+0000 7ffa17200700 1 -- 172.21.15.31:0/1007225689 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa10075d10 con 0x7ffa100745d0 2024-04-25T01:06:15.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.698+0000 7ffa0aa00700 1 --2- 172.21.15.31:0/1007225689 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff9f8039150 0x7ff9f803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:15.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.698+0000 7ffa0aa00700 1 -- 172.21.15.31:0/1007225689 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ffa0401e070 con 0x7ffa100745d0 2024-04-25T01:06:15.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.698+0000 7ffa15400700 1 --2- 172.21.15.31:0/1007225689 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff9f8039150 0x7ff9f803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:15.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.699+0000 7ffa15400700 1 --2- 172.21.15.31:0/1007225689 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff9f8039150 0x7ff9f803b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ffa00006fd0 tx=0x7ffa00006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:06:15.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.704+0000 7ffa0aa00700 1 -- 172.21.15.31:0/1007225689 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffa04013070 con 0x7ffa100745d0 2024-04-25T01:06:15.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:15 smithi031 ceph-mon[28053]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:15.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:15 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:15.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.942+0000 7ffa17200700 1 -- 172.21.15.31:0/1007225689 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffa1019efe0 con 0x7ff9f8039150 2024-04-25T01:06:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.944+0000 7ffa0aa00700 1 -- 172.21.15.31:0/1007225689 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7ffa1019efe0 con 0x7ff9f8039150 2024-04-25T01:06:15.940 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:06:15.940 INFO:teuthology.orchestra.run.smithi031.stdout:[{"addr": "172.21.15.31", "hostname": "smithi031", "labels": [], "status": ""}, {"addr": "172.21.15.40", "hostname": "smithi040", "labels": [], "status": ""}, {"addr": "172.21.15.169", "hostname": "smithi169", "labels": [], "status": ""}] 2024-04-25T01:06:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.947+0000 7ffa17200700 1 -- 172.21.15.31:0/1007225689 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff9f8039150 msgr2=0x7ff9f803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.947+0000 7ffa17200700 1 --2- 172.21.15.31:0/1007225689 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff9f8039150 0x7ff9f803b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ffa00006fd0 tx=0x7ffa00006e40 comp rx=0 tx=0).stop 2024-04-25T01:06:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.947+0000 7ffa17200700 1 -- 172.21.15.31:0/1007225689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa100745d0 msgr2=0x7ffa1019dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:15.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.947+0000 7ffa17200700 1 --2- 172.21.15.31:0/1007225689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa100745d0 0x7ffa1019dc80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffa04006010 tx=0x7ffa04005030 comp rx=0 tx=0).stop 2024-04-25T01:06:15.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.947+0000 7ffa17200700 1 -- 172.21.15.31:0/1007225689 shutdown_connections 2024-04-25T01:06:15.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.947+0000 7ffa17200700 1 --2- 172.21.15.31:0/1007225689 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff9f8039150 0x7ff9f803b610 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:15.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.947+0000 7ffa17200700 1 --2- 172.21.15.31:0/1007225689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa100745d0 0x7ffa1019dc80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:15.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.947+0000 7ffa17200700 1 -- 172.21.15.31:0/1007225689 >> 172.21.15.31:0/1007225689 conn(0x7ffa100fd590 msgr2=0x7ffa100fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:06:15.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.948+0000 7ffa17200700 1 -- 172.21.15.31:0/1007225689 shutdown_connections 2024-04-25T01:06:15.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:15.948+0000 7ffa17200700 1 -- 172.21.15.31:0/1007225689 wait complete. 2024-04-25T01:06:16.701 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T01:06:16.701 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T01:06:17.159 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:06:18.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:17 smithi031 ceph-mon[28053]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:06:18.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:17 smithi031 ceph-mon[28053]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:18.872 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:18.872 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:18.872 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:18.872 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:18.872 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi169", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:06:18.872 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:18.873 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:06:18.873 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:18 smithi031 ceph-mon[28053]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T01:06:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.453+0000 7f93f6400700 1 -- 172.21.15.31:0/3570787284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93f0103f00 msgr2=0x7f93f01042e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.453+0000 7f93f6400700 1 --2- 172.21.15.31:0/3570787284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93f0103f00 0x7f93f01042e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009910 tx=0x7f93e4009c20 comp rx=0 tx=0).stop 2024-04-25T01:06:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.453+0000 7f93f6400700 1 -- 172.21.15.31:0/3570787284 shutdown_connections 2024-04-25T01:06:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.453+0000 7f93f6400700 1 --2- 172.21.15.31:0/3570787284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93f0103f00 0x7f93f01042e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:19.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.453+0000 7f93f6400700 1 -- 172.21.15.31:0/3570787284 >> 172.21.15.31:0/3570787284 conn(0x7f93f00ff7b0 msgr2=0x7f93f0101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:06:19.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.453+0000 7f93f6400700 1 -- 172.21.15.31:0/3570787284 shutdown_connections 2024-04-25T01:06:19.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.453+0000 7f93f6400700 1 -- 172.21.15.31:0/3570787284 wait complete. 2024-04-25T01:06:19.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.455+0000 7f93f6400700 1 Processor -- start 2024-04-25T01:06:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.455+0000 7f93f6400700 1 -- start start 2024-04-25T01:06:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.455+0000 7f93f6400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93f0103f00 0x7f93f019f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.455+0000 7f93f6400700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f93f010e2d0 con 0x7f93f0103f00 2024-04-25T01:06:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.456+0000 7f93f5000700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93f0103f00 0x7f93f019f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.456+0000 7f93f5000700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93f0103f00 0x7f93f019f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57756/0 (socket says 172.21.15.31:57756) 2024-04-25T01:06:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.456+0000 7f93f5000700 1 -- 172.21.15.31:0/3632843023 learned_addr learned my addr 172.21.15.31:0/3632843023 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:06:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.456+0000 7f93f5000700 1 -- 172.21.15.31:0/3632843023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f019f940 con 0x7f93f0103f00 2024-04-25T01:06:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.457+0000 7f93f5000700 1 --2- 172.21.15.31:0/3632843023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93f0103f00 0x7f93f019f400 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f93e4006010 tx=0x7f93e4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:06:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.457+0000 7f93ed600700 1 -- 172.21.15.31:0/3632843023 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93e4026070 con 0x7f93f0103f00 2024-04-25T01:06:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.457+0000 7f93ed600700 1 -- 172.21.15.31:0/3632843023 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93e400bbd0 con 0x7f93f0103f00 2024-04-25T01:06:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.457+0000 7f93ed600700 1 -- 172.21.15.31:0/3632843023 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93e4003910 con 0x7f93f0103f00 2024-04-25T01:06:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.457+0000 7f93f6400700 1 -- 172.21.15.31:0/3632843023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f019fc30 con 0x7f93f0103f00 2024-04-25T01:06:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.457+0000 7f93f6400700 1 -- 172.21.15.31:0/3632843023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f01995c0 con 0x7f93f0103f00 2024-04-25T01:06:19.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.458+0000 7f93ed600700 1 -- 172.21.15.31:0/3632843023 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f93e4029070 con 0x7f93f0103f00 2024-04-25T01:06:19.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.458+0000 7f93f6400700 1 -- 172.21.15.31:0/3632843023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93f01053e0 con 0x7f93f0103f00 2024-04-25T01:06:19.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.459+0000 7f93ed600700 1 --2- 172.21.15.31:0/3632843023 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f93d8039150 0x7f93d803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:19.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.459+0000 7f93ed600700 1 -- 172.21.15.31:0/3632843023 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f93e401e070 con 0x7f93f0103f00 2024-04-25T01:06:19.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.459+0000 7f93efe00700 1 --2- 172.21.15.31:0/3632843023 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f93d8039150 0x7f93d803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:19.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.460+0000 7f93efe00700 1 --2- 172.21.15.31:0/3632843023 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f93d8039150 0x7f93d803b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f93e0006fd0 tx=0x7f93e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:06:19.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.464+0000 7f93ed600700 1 -- 172.21.15.31:0/3632843023 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f93e4013070 con 0x7f93f0103f00 2024-04-25T01:06:19.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:19.697+0000 7f93f6400700 1 -- 172.21.15.31:0/3632843023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f93f0063890 con 0x7f93f0103f00 2024-04-25T01:06:20.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:19 smithi031 ceph-mon[28053]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:20.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:19 smithi031 ceph-mon[28053]: Updating smithi169:/var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:06:20.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:19 smithi031 ceph-mon[28053]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:06:20.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:19 smithi031 ceph-mon[28053]: Updating smithi169:/var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T01:06:20.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:19 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:20.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:19 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:20.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:19 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:20.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:19 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3632843023' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T01:06:20.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:20.111+0000 7f93ed600700 1 -- 172.21.15.31:0/3632843023 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f93e4030090 con 0x7f93f0103f00 2024-04-25T01:06:20.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:20.115+0000 7f93f6400700 1 -- 172.21.15.31:0/3632843023 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f93d8039150 msgr2=0x7f93d803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:20.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:20.115+0000 7f93f6400700 1 --2- 172.21.15.31:0/3632843023 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f93d8039150 0x7f93d803b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f93e0006fd0 tx=0x7f93e0006e40 comp rx=0 tx=0).stop 2024-04-25T01:06:20.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:20.115+0000 7f93f6400700 1 -- 172.21.15.31:0/3632843023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93f0103f00 msgr2=0x7f93f019f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:20.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:20.115+0000 7f93f6400700 1 --2- 172.21.15.31:0/3632843023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93f0103f00 0x7f93f019f400 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f93e4006010 tx=0x7f93e4005030 comp rx=0 tx=0).stop 2024-04-25T01:06:20.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:20.116+0000 7f93f6400700 1 -- 172.21.15.31:0/3632843023 shutdown_connections 2024-04-25T01:06:20.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:20.116+0000 7f93f6400700 1 --2- 172.21.15.31:0/3632843023 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f93d8039150 0x7f93d803b610 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:20.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:20.116+0000 7f93f6400700 1 --2- 172.21.15.31:0/3632843023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93f0103f00 0x7f93f019f400 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:20.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:20.116+0000 7f93f6400700 1 -- 172.21.15.31:0/3632843023 >> 172.21.15.31:0/3632843023 conn(0x7f93f00ff7b0 msgr2=0x7f93f0106c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:06:20.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:20.116+0000 7f93f6400700 1 -- 172.21.15.31:0/3632843023 shutdown_connections 2024-04-25T01:06:20.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:20.116+0000 7f93f6400700 1 -- 172.21.15.31:0/3632843023 wait complete. 2024-04-25T01:06:20.113 INFO:teuthology.orchestra.run.smithi031.stderr:adjusted tunables profile to default 2024-04-25T01:06:20.784 INFO:tasks.cephadm:Adding mon.a on smithi031 2024-04-25T01:06:20.784 INFO:tasks.cephadm:Adding mon.b on smithi040 2024-04-25T01:06:20.785 INFO:tasks.cephadm:Adding mon.c on smithi169 2024-04-25T01:06:20.785 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi031:172.21.15.31=a;smithi040:172.21.15.40=b;smithi169:172.21.15.169=c' 2024-04-25T01:06:21.040 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:06:21.513 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:21 smithi031 ceph-mon[28053]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:21.513 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:21 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3632843023' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T01:06:21.514 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:21 smithi031 ceph-mon[28053]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T01:06:22.506 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.511+0000 7f781fd64700 1 -- 172.21.15.169:0/1334852912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7818101ea0 msgr2=0x7f7818102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:22.506 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.511+0000 7f781fd64700 1 --2- 172.21.15.169:0/1334852912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7818101ea0 0x7f7818102280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7808009910 tx=0x7f7808009c20 comp rx=0 tx=0).stop 2024-04-25T01:06:22.506 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.511+0000 7f781fd64700 1 -- 172.21.15.169:0/1334852912 shutdown_connections 2024-04-25T01:06:22.506 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.511+0000 7f781fd64700 1 --2- 172.21.15.169:0/1334852912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7818101ea0 0x7f7818102280 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:22.506 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.511+0000 7f781fd64700 1 -- 172.21.15.169:0/1334852912 >> 172.21.15.169:0/1334852912 conn(0x7f78180fd770 msgr2=0x7f78180ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:06:22.507 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.511+0000 7f781fd64700 1 -- 172.21.15.169:0/1334852912 shutdown_connections 2024-04-25T01:06:22.507 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.511+0000 7f781fd64700 1 -- 172.21.15.169:0/1334852912 wait complete. 2024-04-25T01:06:22.507 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.512+0000 7f781fd64700 1 Processor -- start 2024-04-25T01:06:22.507 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.512+0000 7f781fd64700 1 -- start start 2024-04-25T01:06:22.507 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.512+0000 7f781fd64700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7818101ea0 0x7f781819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:22.507 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.512+0000 7f781fd64700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f781810c270 con 0x7f7818101ea0 2024-04-25T01:06:22.508 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.512+0000 7f781db00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7818101ea0 0x7f781819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:22.508 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.512+0000 7f781db00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7818101ea0 0x7f781819d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.169:52604/0 (socket says 172.21.15.169:52604) 2024-04-25T01:06:22.508 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.512+0000 7f781db00700 1 -- 172.21.15.169:0/2455707019 learned_addr learned my addr 172.21.15.169:0/2455707019 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T01:06:22.508 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.513+0000 7f781db00700 1 -- 172.21.15.169:0/2455707019 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781819d990 con 0x7f7818101ea0 2024-04-25T01:06:22.508 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.513+0000 7f781db00700 1 --2- 172.21.15.169:0/2455707019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7818101ea0 0x7f781819d450 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7808005850 tx=0x7f7808005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:06:22.508 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.514+0000 7f780effd700 1 -- 172.21.15.169:0/2455707019 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7808026070 con 0x7f7818101ea0 2024-04-25T01:06:22.508 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.514+0000 7f781fd64700 1 -- 172.21.15.169:0/2455707019 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f781819dc80 con 0x7f7818101ea0 2024-04-25T01:06:22.508 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.514+0000 7f781fd64700 1 -- 172.21.15.169:0/2455707019 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7818197610 con 0x7f7818101ea0 2024-04-25T01:06:22.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.514+0000 7f780effd700 1 -- 172.21.15.169:0/2455707019 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f780800bbd0 con 0x7f7818101ea0 2024-04-25T01:06:22.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.514+0000 7f780effd700 1 -- 172.21.15.169:0/2455707019 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78080047e0 con 0x7f7818101ea0 2024-04-25T01:06:22.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.515+0000 7f781fd64700 1 -- 172.21.15.169:0/2455707019 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77fc005330 con 0x7f7818101ea0 2024-04-25T01:06:22.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.515+0000 7f780effd700 1 -- 172.21.15.169:0/2455707019 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7808029070 con 0x7f7818101ea0 2024-04-25T01:06:22.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.515+0000 7f780effd700 1 --2- 172.21.15.169:0/2455707019 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f78040390f0 0x7f780403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:22.510 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.515+0000 7f780effd700 1 -- 172.21.15.169:0/2455707019 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f780801e070 con 0x7f7818101ea0 2024-04-25T01:06:22.510 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.516+0000 7f781d2ff700 1 --2- 172.21.15.169:0/2455707019 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f78040390f0 0x7f780403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:22.511 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.517+0000 7f781d2ff700 1 --2- 172.21.15.169:0/2455707019 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f78040390f0 0x7f780403b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7814006fd0 tx=0x7f7814006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:06:22.525 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.531+0000 7f780effd700 1 -- 172.21.15.169:0/2455707019 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7808013070 con 0x7f7818101ea0 2024-04-25T01:06:22.732 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.738+0000 7f781fd64700 1 -- 172.21.15.169:0/2455707019 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi040:172.21.15.40=b;smithi169:172.21.15.169=c", "target": ["mon-mgr", ""]}) -- 0x7f77fc000c80 con 0x7f78040390f0 2024-04-25T01:06:22.747 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.753+0000 7f780effd700 1 -- 172.21.15.169:0/2455707019 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f77fc000c80 con 0x7f78040390f0 2024-04-25T01:06:22.747 INFO:teuthology.orchestra.run.smithi169.stdout:Scheduled mon update... 2024-04-25T01:06:22.749 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.755+0000 7f781fd64700 1 -- 172.21.15.169:0/2455707019 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f78040390f0 msgr2=0x7f780403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:22.750 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.755+0000 7f781fd64700 1 --2- 172.21.15.169:0/2455707019 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f78040390f0 0x7f780403b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7814006fd0 tx=0x7f7814006e40 comp rx=0 tx=0).stop 2024-04-25T01:06:22.750 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.755+0000 7f781fd64700 1 -- 172.21.15.169:0/2455707019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7818101ea0 msgr2=0x7f781819d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:22.750 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.755+0000 7f781fd64700 1 --2- 172.21.15.169:0/2455707019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7818101ea0 0x7f781819d450 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7808005850 tx=0x7f7808005030 comp rx=0 tx=0).stop 2024-04-25T01:06:22.750 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.755+0000 7f781fd64700 1 -- 172.21.15.169:0/2455707019 shutdown_connections 2024-04-25T01:06:22.750 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.755+0000 7f781fd64700 1 --2- 172.21.15.169:0/2455707019 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f78040390f0 0x7f780403b5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:22.750 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.755+0000 7f781fd64700 1 --2- 172.21.15.169:0/2455707019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7818101ea0 0x7f781819d450 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:22.750 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.755+0000 7f781fd64700 1 -- 172.21.15.169:0/2455707019 >> 172.21.15.169:0/2455707019 conn(0x7f78180fd770 msgr2=0x7f7818104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:06:22.750 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.755+0000 7f781fd64700 1 -- 172.21.15.169:0/2455707019 shutdown_connections 2024-04-25T01:06:22.750 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:22.755+0000 7f781fd64700 1 -- 172.21.15.169:0/2455707019 wait complete. 2024-04-25T01:06:23.403 DEBUG:teuthology.orchestra.run.smithi040:mon.b> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T01:06:23.406 DEBUG:teuthology.orchestra.run.smithi169:mon.c> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T01:06:23.408 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T01:06:23.408 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T01:06:24.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:23 smithi031 ceph-mon[28053]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:24.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:23 smithi031 ceph-mon[28053]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi040:172.21.15.40=b;smithi169:172.21.15.169=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:06:24.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:23 smithi031 ceph-mon[28053]: Saving service mon spec with placement smithi031:172.21.15.31=a;smithi040:172.21.15.40=b;smithi169:172.21.15.169=c;count:3 2024-04-25T01:06:24.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:23 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:24.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:23 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:06:24.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:23 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:24.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:23 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:06:24.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:23 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:24.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:23 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:06:24.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:23 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:24.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:23 smithi031 ceph-mon[28053]: Deploying daemon mon.c on smithi169 2024-04-25T01:06:25.400 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:06:26.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:25 smithi031 ceph-mon[28053]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:26.668 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:26 smithi169 systemd[1]: Starting Ceph mon.c for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:06:27.216 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.221+0000 7fa6fc06d700 1 -- 172.21.15.169:0/2424348358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6f4101fa0 msgr2=0x7fa6f4102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:27.216 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.221+0000 7fa6fc06d700 1 --2- 172.21.15.169:0/2424348358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6f4101fa0 0x7fa6f4102380 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0009910 tx=0x7fa6f0009c20 comp rx=0 tx=0).stop 2024-04-25T01:06:27.216 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.221+0000 7fa6fc06d700 1 -- 172.21.15.169:0/2424348358 shutdown_connections 2024-04-25T01:06:27.216 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.221+0000 7fa6fc06d700 1 --2- 172.21.15.169:0/2424348358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6f4101fa0 0x7fa6f4102380 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:27.216 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.221+0000 7fa6fc06d700 1 -- 172.21.15.169:0/2424348358 >> 172.21.15.169:0/2424348358 conn(0x7fa6f40fd830 msgr2=0x7fa6f40ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:06:27.217 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.221+0000 7fa6fc06d700 1 -- 172.21.15.169:0/2424348358 shutdown_connections 2024-04-25T01:06:27.217 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.221+0000 7fa6fc06d700 1 -- 172.21.15.169:0/2424348358 wait complete. 2024-04-25T01:06:27.217 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.222+0000 7fa6fc06d700 1 Processor -- start 2024-04-25T01:06:27.217 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.222+0000 7fa6fc06d700 1 -- start start 2024-04-25T01:06:27.217 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.222+0000 7fa6fc06d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6f4101fa0 0x7fa6f4074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:27.217 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.222+0000 7fa6fc06d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f410c370 con 0x7fa6f4101fa0 2024-04-25T01:06:27.218 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.223+0000 7fa6f9e09700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6f4101fa0 0x7fa6f4074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:27.218 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.223+0000 7fa6f9e09700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6f4101fa0 0x7fa6f4074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.169:52622/0 (socket says 172.21.15.169:52622) 2024-04-25T01:06:27.218 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.223+0000 7fa6f9e09700 1 -- 172.21.15.169:0/4064700772 learned_addr learned my addr 172.21.15.169:0/4064700772 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T01:06:27.218 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.223+0000 7fa6f9e09700 1 -- 172.21.15.169:0/4064700772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f4076420 con 0x7fa6f4101fa0 2024-04-25T01:06:27.218 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.224+0000 7fa6f9e09700 1 --2- 172.21.15.169:0/4064700772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6f4101fa0 0x7fa6f4074530 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0005850 tx=0x7fa6f0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:06:27.219 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.224+0000 7fa6eaffd700 1 -- 172.21.15.169:0/4064700772 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6f0026070 con 0x7fa6f4101fa0 2024-04-25T01:06:27.219 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.224+0000 7fa6fc06d700 1 -- 172.21.15.169:0/4064700772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f4074ad0 con 0x7fa6f4101fa0 2024-04-25T01:06:27.219 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.224+0000 7fa6fc06d700 1 -- 172.21.15.169:0/4064700772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f4074f90 con 0x7fa6f4101fa0 2024-04-25T01:06:27.219 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.224+0000 7fa6eaffd700 1 -- 172.21.15.169:0/4064700772 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6f000b6f0 con 0x7fa6f4101fa0 2024-04-25T01:06:27.219 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.224+0000 7fa6eaffd700 1 -- 172.21.15.169:0/4064700772 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6f0030060 con 0x7fa6f4101fa0 2024-04-25T01:06:27.219 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.225+0000 7fa6fc06d700 1 -- 172.21.15.169:0/4064700772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6d8005330 con 0x7fa6f4101fa0 2024-04-25T01:06:27.220 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.226+0000 7fa6eaffd700 1 -- 172.21.15.169:0/4064700772 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa6f0029070 con 0x7fa6f4101fa0 2024-04-25T01:06:27.220 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.226+0000 7fa6eaffd700 1 --2- 172.21.15.169:0/4064700772 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa6e00390f0 0x7fa6e003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:27.220 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.226+0000 7fa6eaffd700 1 -- 172.21.15.169:0/4064700772 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa6f001e070 con 0x7fa6f4101fa0 2024-04-25T01:06:27.220 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.226+0000 7fa6f9608700 1 --2- 172.21.15.169:0/4064700772 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa6e00390f0 0x7fa6e003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:27.221 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.227+0000 7fa6f9608700 1 --2- 172.21.15.169:0/4064700772 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa6e00390f0 0x7fa6e003b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4006fd0 tx=0x7fa6e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:06:27.236 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.242+0000 7fa6eaffd700 1 -- 172.21.15.169:0/4064700772 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa6f0013070 con 0x7fa6f4101fa0 2024-04-25T01:06:27.521 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.526+0000 7fa6fc06d700 1 -- 172.21.15.169:0/4064700772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa6d80058f0 con 0x7fa6f4101fa0 2024-04-25T01:06:27.521 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.528+0000 7fa6eaffd700 1 -- 172.21.15.169:0/4064700772 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fa6f004c010 con 0x7fa6f4101fa0 2024-04-25T01:06:27.522 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T01:06:27.522 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"95849b94-029f-11ef-bc93-c7b262605968","modified":"2024-04-25T01:04:18.937945Z","created":"2024-04-25T01:04:18.937945Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T01:06:27.524 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.530+0000 7fa6fc06d700 1 -- 172.21.15.169:0/4064700772 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa6e00390f0 msgr2=0x7fa6e003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:27.524 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.530+0000 7fa6fc06d700 1 --2- 172.21.15.169:0/4064700772 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa6e00390f0 0x7fa6e003b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4006fd0 tx=0x7fa6e4006e40 comp rx=0 tx=0).stop 2024-04-25T01:06:27.524 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.530+0000 7fa6fc06d700 1 -- 172.21.15.169:0/4064700772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6f4101fa0 msgr2=0x7fa6f4074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:27.524 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.530+0000 7fa6fc06d700 1 --2- 172.21.15.169:0/4064700772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6f4101fa0 0x7fa6f4074530 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0005850 tx=0x7fa6f0005030 comp rx=0 tx=0).stop 2024-04-25T01:06:27.524 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.530+0000 7fa6fc06d700 1 -- 172.21.15.169:0/4064700772 shutdown_connections 2024-04-25T01:06:27.524 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.530+0000 7fa6fc06d700 1 --2- 172.21.15.169:0/4064700772 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa6e00390f0 0x7fa6e003b5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:27.524 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.530+0000 7fa6fc06d700 1 --2- 172.21.15.169:0/4064700772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6f4101fa0 0x7fa6f4074530 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:27.524 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.530+0000 7fa6fc06d700 1 -- 172.21.15.169:0/4064700772 >> 172.21.15.169:0/4064700772 conn(0x7fa6f40fd830 msgr2=0x7fa6f4103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:06:27.524 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.530+0000 7fa6fc06d700 1 -- 172.21.15.169:0/4064700772 shutdown_connections 2024-04-25T01:06:27.525 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:27.530+0000 7fa6fc06d700 1 -- 172.21.15.169:0/4064700772 wait complete. 2024-04-25T01:06:27.525 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-04-25T01:06:27.595 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 podman[32705]: 2024-04-25 01:06:27.328188014 +0000 UTC m=+0.292263177 container create 1dac5730d76111be6cdc562dc47962f948ce3421a07a19c27d4df2ad293be3fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T01:06:27.596 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 podman[32705]: 2024-04-25 01:06:27.234771097 +0000 UTC m=+0.198846276 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:06:27.596 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 podman[32705]: 2024-04-25 01:06:27.574939598 +0000 UTC m=+0.539014763 container init 1dac5730d76111be6cdc562dc47962f948ce3421a07a19c27d4df2ad293be3fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T01:06:27.596 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 podman[32705]: 2024-04-25 01:06:27.578269455 +0000 UTC m=+0.542344622 container start 1dac5730d76111be6cdc562dc47962f948ce3421a07a19c27d4df2ad293be3fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-c, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T01:06:27.596 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T01:06:27.954 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-25T01:06:27.954 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: pidfile_write: ignore empty --pid-file 2024-04-25T01:06:27.954 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: load: jerasure load: lrc 2024-04-25T01:06:27.954 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: RocksDB version: 7.9.2 2024-04-25T01:06:27.954 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Git sha 0 2024-04-25T01:06:27.954 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-25T01:06:27.954 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: DB SUMMARY 2024-04-25T01:06:27.954 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: DB Session ID: M4S917D7D6ETG2ZLGAAV 2024-04-25T01:06:27.954 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: CURRENT file: CURRENT 2024-04-25T01:06:27.954 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T01:06:27.954 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T01:06:27.955 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-25T01:06:27.955 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-25T01:06:27.955 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.error_if_exists: 0 2024-04-25T01:06:27.955 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.create_if_missing: 0 2024-04-25T01:06:27.955 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.paranoid_checks: 1 2024-04-25T01:06:27.955 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T01:06:27.955 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T01:06:27.955 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T01:06:27.955 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.env: 0x561f39563c60 2024-04-25T01:06:27.955 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T01:06:27.955 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.info_log: 0x561f3af7daa0 2024-04-25T01:06:27.955 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T01:06:27.955 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.statistics: (nil) 2024-04-25T01:06:27.956 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.use_fsync: 0 2024-04-25T01:06:27.956 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_log_file_size: 0 2024-04-25T01:06:27.956 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T01:06:27.956 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T01:06:27.956 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T01:06:27.956 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T01:06:27.956 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.allow_fallocate: 1 2024-04-25T01:06:27.956 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T01:06:27.956 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T01:06:27.956 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.use_direct_reads: 0 2024-04-25T01:06:27.956 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T01:06:27.956 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T01:06:27.956 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.db_log_dir: 2024-04-25T01:06:27.957 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.wal_dir: 2024-04-25T01:06:27.957 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T01:06:27.957 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T01:06:27.957 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T01:06:27.957 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T01:06:27.957 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T01:06:27.957 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T01:06:27.957 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T01:06:27.957 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T01:06:27.957 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.write_buffer_manager: 0x561f3af501e0 2024-04-25T01:06:27.957 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T01:06:27.957 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T01:06:27.958 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T01:06:27.958 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T01:06:27.958 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T01:06:27.958 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T01:06:27.958 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T01:06:27.958 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T01:06:27.958 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.unordered_write: 0 2024-04-25T01:06:27.958 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T01:06:27.958 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T01:06:27.958 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T01:06:27.958 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T01:06:27.958 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.row_cache: None 2024-04-25T01:06:27.958 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.wal_filter: None 2024-04-25T01:06:27.958 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T01:06:27.959 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T01:06:27.959 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.two_write_queues: 0 2024-04-25T01:06:27.959 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T01:06:27.959 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.wal_compression: 0 2024-04-25T01:06:27.959 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.atomic_flush: 0 2024-04-25T01:06:27.959 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T01:06:27.959 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T01:06:27.959 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T01:06:27.959 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.log_readahead_size: 0 2024-04-25T01:06:27.959 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T01:06:27.959 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T01:06:27.959 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T01:06:27.959 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T01:06:27.959 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T01:06:27.960 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T01:06:27.960 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T01:06:27.960 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_background_jobs: 2 2024-04-25T01:06:27.961 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_background_compactions: -1 2024-04-25T01:06:27.961 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_subcompactions: 1 2024-04-25T01:06:27.961 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T01:06:27.962 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T01:06:27.962 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T01:06:27.962 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T01:06:27.962 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T01:06:27.962 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T01:06:27.962 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T01:06:27.962 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T01:06:27.962 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_open_files: -1 2024-04-25T01:06:27.962 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T01:06:27.962 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T01:06:27.962 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T01:06:27.962 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T01:06:27.962 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_background_flushes: -1 2024-04-25T01:06:27.962 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Compression algorithms supported: 2024-04-25T01:06:27.963 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T01:06:27.963 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: kZSTD supported: 0 2024-04-25T01:06:27.963 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: kXpressCompression supported: 0 2024-04-25T01:06:27.963 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T01:06:27.963 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: kZlibCompression supported: 1 2024-04-25T01:06:27.963 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: kSnappyCompression supported: 1 2024-04-25T01:06:27.963 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: kLZ4Compression supported: 1 2024-04-25T01:06:27.963 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: kBZip2Compression supported: 0 2024-04-25T01:06:27.963 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T01:06:27.963 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T01:06:27.963 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-25T01:06:27.963 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T01:06:27.963 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T01:06:27.963 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.merge_operator: 2024-04-25T01:06:27.964 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.compaction_filter: None 2024-04-25T01:06:27.964 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T01:06:27.964 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T01:06:27.964 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T01:06:27.964 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T01:06:27.964 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561f3af7dba0) 2024-04-25T01:06:27.964 INFO:journalctl@ceph.mon.c.smithi169.stdout: cache_index_and_filter_blocks: 1 2024-04-25T01:06:27.964 INFO:journalctl@ceph.mon.c.smithi169.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T01:06:27.964 INFO:journalctl@ceph.mon.c.smithi169.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T01:06:27.964 INFO:journalctl@ceph.mon.c.smithi169.stdout: pin_top_level_index_and_filter: 1 2024-04-25T01:06:27.964 INFO:journalctl@ceph.mon.c.smithi169.stdout: index_type: 0 2024-04-25T01:06:27.964 INFO:journalctl@ceph.mon.c.smithi169.stdout: data_block_index_type: 0 2024-04-25T01:06:27.964 INFO:journalctl@ceph.mon.c.smithi169.stdout: index_shortening: 1 2024-04-25T01:06:27.965 INFO:journalctl@ceph.mon.c.smithi169.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T01:06:27.965 INFO:journalctl@ceph.mon.c.smithi169.stdout: checksum: 4 2024-04-25T01:06:27.965 INFO:journalctl@ceph.mon.c.smithi169.stdout: no_block_cache: 0 2024-04-25T01:06:27.965 INFO:journalctl@ceph.mon.c.smithi169.stdout: block_cache: 0x561f3af81090 2024-04-25T01:06:27.965 INFO:journalctl@ceph.mon.c.smithi169.stdout: block_cache_name: BinnedLRUCache 2024-04-25T01:06:27.965 INFO:journalctl@ceph.mon.c.smithi169.stdout: block_cache_options: 2024-04-25T01:06:27.965 INFO:journalctl@ceph.mon.c.smithi169.stdout: capacity : 536870912 2024-04-25T01:06:27.965 INFO:journalctl@ceph.mon.c.smithi169.stdout: num_shard_bits : 4 2024-04-25T01:06:27.965 INFO:journalctl@ceph.mon.c.smithi169.stdout: strict_capacity_limit : 0 2024-04-25T01:06:27.965 INFO:journalctl@ceph.mon.c.smithi169.stdout: high_pri_pool_ratio: 0.000 2024-04-25T01:06:27.965 INFO:journalctl@ceph.mon.c.smithi169.stdout: block_cache_compressed: (nil) 2024-04-25T01:06:27.965 INFO:journalctl@ceph.mon.c.smithi169.stdout: persistent_cache: (nil) 2024-04-25T01:06:27.965 INFO:journalctl@ceph.mon.c.smithi169.stdout: block_size: 4096 2024-04-25T01:06:27.965 INFO:journalctl@ceph.mon.c.smithi169.stdout: block_size_deviation: 10 2024-04-25T01:06:27.966 INFO:journalctl@ceph.mon.c.smithi169.stdout: block_restart_interval: 16 2024-04-25T01:06:27.966 INFO:journalctl@ceph.mon.c.smithi169.stdout: index_block_restart_interval: 1 2024-04-25T01:06:27.966 INFO:journalctl@ceph.mon.c.smithi169.stdout: metadata_block_size: 4096 2024-04-25T01:06:27.966 INFO:journalctl@ceph.mon.c.smithi169.stdout: partition_filters: 0 2024-04-25T01:06:27.966 INFO:journalctl@ceph.mon.c.smithi169.stdout: use_delta_encoding: 1 2024-04-25T01:06:27.966 INFO:journalctl@ceph.mon.c.smithi169.stdout: filter_policy: bloomfilter 2024-04-25T01:06:27.966 INFO:journalctl@ceph.mon.c.smithi169.stdout: whole_key_filtering: 1 2024-04-25T01:06:27.966 INFO:journalctl@ceph.mon.c.smithi169.stdout: verify_compression: 0 2024-04-25T01:06:27.966 INFO:journalctl@ceph.mon.c.smithi169.stdout: read_amp_bytes_per_bit: 0 2024-04-25T01:06:27.966 INFO:journalctl@ceph.mon.c.smithi169.stdout: format_version: 5 2024-04-25T01:06:27.966 INFO:journalctl@ceph.mon.c.smithi169.stdout: enable_index_compression: 1 2024-04-25T01:06:27.966 INFO:journalctl@ceph.mon.c.smithi169.stdout: block_align: 0 2024-04-25T01:06:27.966 INFO:journalctl@ceph.mon.c.smithi169.stdout: max_auto_readahead_size: 262144 2024-04-25T01:06:27.966 INFO:journalctl@ceph.mon.c.smithi169.stdout: prepopulate_block_cache: 0 2024-04-25T01:06:27.966 INFO:journalctl@ceph.mon.c.smithi169.stdout: initial_auto_readahead_size: 8192 2024-04-25T01:06:27.967 INFO:journalctl@ceph.mon.c.smithi169.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T01:06:27.967 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T01:06:27.967 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T01:06:27.967 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.compression: NoCompression 2024-04-25T01:06:27.967 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T01:06:27.967 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T01:06:27.967 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T01:06:27.967 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.num_levels: 7 2024-04-25T01:06:27.967 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T01:06:27.967 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T01:06:27.969 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T01:06:27.969 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T01:06:27.969 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T01:06:27.969 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T01:06:27.970 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T01:06:27.970 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T01:06:27.970 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T01:06:27.970 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T01:06:27.970 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T01:06:27.970 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T01:06:27.970 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T01:06:27.970 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T01:06:27.970 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T01:06:27.970 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T01:06:27.970 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T01:06:27.970 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T01:06:27.971 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T01:06:27.971 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T01:06:27.971 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T01:06:27.971 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T01:06:27.971 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T01:06:27.971 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T01:06:27.971 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T01:06:27.971 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T01:06:27.971 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T01:06:27.971 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T01:06:27.971 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T01:06:27.971 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T01:06:27.971 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T01:06:27.971 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T01:06:27.971 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T01:06:27.972 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T01:06:27.972 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T01:06:27.972 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T01:06:27.972 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T01:06:27.972 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T01:06:27.972 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T01:06:27.972 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T01:06:27.972 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T01:06:27.972 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T01:06:27.972 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T01:06:27.972 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T01:06:27.972 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T01:06:27.972 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T01:06:27.972 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T01:06:27.973 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T01:06:27.973 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T01:06:27.973 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T01:06:27.973 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T01:06:27.973 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T01:06:27.973 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T01:06:27.973 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.table_properties_collectors: 2024-04-25T01:06:27.973 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.inplace_update_support: 0 2024-04-25T01:06:27.973 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T01:06:27.973 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T01:06:27.973 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T01:06:27.973 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T01:06:27.973 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.bloom_locality: 0 2024-04-25T01:06:27.973 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.max_successive_merges: 0 2024-04-25T01:06:27.973 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T01:06:27.974 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T01:06:27.974 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T01:06:27.974 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T01:06:27.974 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.ttl: 2592000 2024-04-25T01:06:27.974 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T01:06:27.974 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T01:06:27.974 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T01:06:27.974 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.enable_blob_files: false 2024-04-25T01:06:27.974 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.min_blob_size: 0 2024-04-25T01:06:27.974 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T01:06:27.974 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T01:06:27.974 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T01:06:27.974 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T01:06:27.974 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T01:06:27.975 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T01:06:27.975 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T01:06:27.975 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T01:06:27.975 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T01:06:27.975 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T01:06:27.975 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a1bdfaec-7cb5-4e0e-b68e-68f421afe223 2024-04-25T01:06:27.975 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714007187608706, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T01:06:27.975 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T01:06:27.975 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714007187609269, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714007187, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a1bdfaec-7cb5-4e0e-b68e-68f421afe223", "db_session_id": "M4S917D7D6ETG2ZLGAAV", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T01:06:27.975 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714007187609373, "job": 1, "event": "recovery_finished"} 2024-04-25T01:06:27.975 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T01:06:27.975 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T01:06:27.975 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561f3b072000 2024-04-25T01:06:27.976 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: DB pointer 0x561f3b05c000 2024-04-25T01:06:27.976 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T01:06:27.976 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T01:06:27.976 INFO:journalctl@ceph.mon.c.smithi169.stdout: ** DB Stats ** 2024-04-25T01:06:27.976 INFO:journalctl@ceph.mon.c.smithi169.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T01:06:27.976 INFO:journalctl@ceph.mon.c.smithi169.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T01:06:27.976 INFO:journalctl@ceph.mon.c.smithi169.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T01:06:27.976 INFO:journalctl@ceph.mon.c.smithi169.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T01:06:27.976 INFO:journalctl@ceph.mon.c.smithi169.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T01:06:27.976 INFO:journalctl@ceph.mon.c.smithi169.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T01:06:27.976 INFO:journalctl@ceph.mon.c.smithi169.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T01:06:27.976 INFO:journalctl@ceph.mon.c.smithi169.stdout: 2024-04-25T01:06:27.976 INFO:journalctl@ceph.mon.c.smithi169.stdout: ** Compaction Stats [default] ** 2024-04-25T01:06:27.977 INFO:journalctl@ceph.mon.c.smithi169.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T01:06:27.977 INFO:journalctl@ceph.mon.c.smithi169.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T01:06:27.977 INFO:journalctl@ceph.mon.c.smithi169.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T01:06:27.977 INFO:journalctl@ceph.mon.c.smithi169.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T01:06:27.977 INFO:journalctl@ceph.mon.c.smithi169.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T01:06:27.977 INFO:journalctl@ceph.mon.c.smithi169.stdout: 2024-04-25T01:06:27.977 INFO:journalctl@ceph.mon.c.smithi169.stdout: ** Compaction Stats [default] ** 2024-04-25T01:06:27.977 INFO:journalctl@ceph.mon.c.smithi169.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T01:06:27.977 INFO:journalctl@ceph.mon.c.smithi169.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T01:06:27.977 INFO:journalctl@ceph.mon.c.smithi169.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T01:06:27.977 INFO:journalctl@ceph.mon.c.smithi169.stdout: 2024-04-25T01:06:27.977 INFO:journalctl@ceph.mon.c.smithi169.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T01:06:27.977 INFO:journalctl@ceph.mon.c.smithi169.stdout: 2024-04-25T01:06:27.977 INFO:journalctl@ceph.mon.c.smithi169.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T01:06:27.977 INFO:journalctl@ceph.mon.c.smithi169.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T01:06:27.978 INFO:journalctl@ceph.mon.c.smithi169.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T01:06:27.978 INFO:journalctl@ceph.mon.c.smithi169.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T01:06:27.978 INFO:journalctl@ceph.mon.c.smithi169.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T01:06:27.978 INFO:journalctl@ceph.mon.c.smithi169.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T01:06:27.978 INFO:journalctl@ceph.mon.c.smithi169.stdout: Cumulative compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T01:06:27.978 INFO:journalctl@ceph.mon.c.smithi169.stdout: Interval compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T01:06:27.978 INFO:journalctl@ceph.mon.c.smithi169.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T01:06:27.978 INFO:journalctl@ceph.mon.c.smithi169.stdout: Block cache BinnedLRUCache@0x561f3af81090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-25T01:06:27.978 INFO:journalctl@ceph.mon.c.smithi169.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T01:06:27.978 INFO:journalctl@ceph.mon.c.smithi169.stdout: 2024-04-25T01:06:27.978 INFO:journalctl@ceph.mon.c.smithi169.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T01:06:27.978 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-25T01:06:27.978 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: using public_addr v2:172.21.15.169:0/0 -> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] 2024-04-25T01:06:27.978 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: starting mon.c rank -1 at public addrs [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] at bind addrs [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:06:27.979 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: mon.c@-1(???) e0 preinit fsid 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:06:27.979 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: mon.c@-1(synchronizing).mds e1 new map 2024-04-25T01:06:27.979 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-25T01:06:27.979 INFO:journalctl@ceph.mon.c.smithi169.stdout: e1 2024-04-25T01:06:27.979 INFO:journalctl@ceph.mon.c.smithi169.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:06:27.979 INFO:journalctl@ceph.mon.c.smithi169.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T01:06:27.979 INFO:journalctl@ceph.mon.c.smithi169.stdout: legacy client fscid: -1 2024-04-25T01:06:27.979 INFO:journalctl@ceph.mon.c.smithi169.stdout: 2024-04-25T01:06:27.979 INFO:journalctl@ceph.mon.c.smithi169.stdout: No filesystems configured 2024-04-25T01:06:27.979 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T01:06:27.979 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T01:06:27.979 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T01:06:27.979 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T01:06:27.979 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T01:06:27.980 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T01:06:27.980 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T01:06:27.980 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T01:06:27.980 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T01:06:27.980 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T01:06:27.980 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:27.980 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:27.980 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: Added host smithi040 2024-04-25T01:06:27.980 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:06:27.980 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:27.980 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:27.980 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:27.980 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:27.980 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:06:27.981 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:27.981 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:27.981 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:27.981 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:27.981 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:27.981 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:27.981 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi040", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:06:27.981 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:27.981 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:06:27.981 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: Updating smithi040:/etc/ceph/ceph.conf 2024-04-25T01:06:27.981 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: Updating smithi040:/var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:06:27.981 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: Updating smithi040:/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:06:27.981 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: Updating smithi040:/var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T01:06:27.981 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi169", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:06:27.981 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:27.982 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:27.982 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:27.982 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:27.982 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: Deploying cephadm binary to smithi169 2024-04-25T01:06:27.982 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:27.982 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:27.982 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:27.982 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: Added host smithi169 2024-04-25T01:06:27.982 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:06:27.982 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:27.982 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:27.982 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:27.982 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:27.983 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:06:27.983 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:27.983 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:27.983 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:27.983 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:27.983 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:27.983 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi169", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:06:27.983 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:27.983 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:06:27.983 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T01:06:27.983 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:27.983 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: Updating smithi169:/var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:06:27.983 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:06:27.983 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: Updating smithi169:/var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T01:06:27.983 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:27.983 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:27.984 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:27.984 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3632843023' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T01:06:27.984 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:27.984 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3632843023' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T01:06:27.984 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T01:06:27.984 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:27.984 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi040:172.21.15.40=b;smithi169:172.21.15.169=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:06:27.984 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: Saving service mon spec with placement smithi031:172.21.15.31=a;smithi040:172.21.15.40=b;smithi169:172.21.15.169=c;count:3 2024-04-25T01:06:27.984 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:27.984 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:06:27.984 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:27.984 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:06:27.984 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:27.984 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:06:27.984 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:27.985 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: Deploying daemon mon.c on smithi169 2024-04-25T01:06:27.985 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:27.985 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:27 smithi169 ceph-mon[32759]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T01:06:28.703 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:28 smithi169 bash[32705]: 1dac5730d76111be6cdc562dc47962f948ce3421a07a19c27d4df2ad293be3fb 2024-04-25T01:06:28.703 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:28 smithi169 systemd[1]: Started Ceph mon.c for 95849b94-029f-11ef-bc93-c7b262605968. 2024-04-25T01:06:29.209 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T01:06:29.210 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T01:06:29.456 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:06:33.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:06:33.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:06:33.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: mon.a calling monitor election 2024-04-25T01:06:33.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:06:33.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:33.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:06:33.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:06:33.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: mon.c calling monitor election 2024-04-25T01:06:33.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:33.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:06:33.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:06:33.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:06:33.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T01:06:33.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: monmap epoch 2 2024-04-25T01:06:33.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: fsid 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:06:33.016 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: last_changed 2024-04-25T01:06:27.643092+0000 2024-04-25T01:06:33.016 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: created 2024-04-25T01:04:18.937945+0000 2024-04-25T01:06:33.016 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: min_mon_release 19 (squid) 2024-04-25T01:06:33.016 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: election_strategy: 1 2024-04-25T01:06:33.016 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T01:06:33.016 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T01:06:33.016 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: fsmap 2024-04-25T01:06:33.016 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T01:06:33.016 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: mgrmap e13: x(active, since 60s) 2024-04-25T01:06:33.016 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: overall HEALTH_OK 2024-04-25T01:06:33.016 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:33.016 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:33.016 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:33.017 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:32 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:06:33.203 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:06:33.203 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:06:33.203 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: mon.a calling monitor election 2024-04-25T01:06:33.203 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:06:33.203 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:33.203 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:06:33.203 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:06:33.204 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: mon.c calling monitor election 2024-04-25T01:06:33.204 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:33.204 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:06:33.204 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:06:33.204 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:06:33.204 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T01:06:33.204 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: monmap epoch 2 2024-04-25T01:06:33.204 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: fsid 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:06:33.204 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: last_changed 2024-04-25T01:06:27.643092+0000 2024-04-25T01:06:33.204 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: created 2024-04-25T01:04:18.937945+0000 2024-04-25T01:06:33.204 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: min_mon_release 19 (squid) 2024-04-25T01:06:33.204 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: election_strategy: 1 2024-04-25T01:06:33.204 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T01:06:33.204 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T01:06:33.205 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: fsmap 2024-04-25T01:06:33.205 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T01:06:33.205 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: mgrmap e13: x(active, since 60s) 2024-04-25T01:06:33.205 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: overall HEALTH_OK 2024-04-25T01:06:33.205 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:33.205 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:33.205 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:33.205 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:32 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:06:33.919 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.925+0000 7f7898486700 1 -- 172.21.15.169:0/2122825664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f787c0035c0 msgr2=0x7f787c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:33.920 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.925+0000 7f7898486700 1 --2- 172.21.15.169:0/2122825664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f787c0035c0 0x7f787c005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7890100b50 tx=0x7f7888009e80 comp rx=0 tx=0).stop 2024-04-25T01:06:33.920 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.925+0000 7f7898486700 1 -- 172.21.15.169:0/2122825664 shutdown_connections 2024-04-25T01:06:33.920 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.925+0000 7f7898486700 1 --2- 172.21.15.169:0/2122825664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f787c0035c0 0x7f787c005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:33.920 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.925+0000 7f7898486700 1 -- 172.21.15.169:0/2122825664 >> 172.21.15.169:0/2122825664 conn(0x7f78900fc710 msgr2=0x7f78900feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:06:33.920 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.925+0000 7f7898486700 1 -- 172.21.15.169:0/2122825664 shutdown_connections 2024-04-25T01:06:33.920 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.925+0000 7f7898486700 1 -- 172.21.15.169:0/2122825664 wait complete. 2024-04-25T01:06:33.922 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.926+0000 7f7898486700 1 Processor -- start 2024-04-25T01:06:33.922 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.926+0000 7f7898486700 1 -- start start 2024-04-25T01:06:33.922 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.926+0000 7f7898486700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f787c0035c0 0x7f7890102920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:33.922 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.926+0000 7f7898486700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7890102e60 0x7f78901032e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:33.922 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.926+0000 7f7898486700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7890108cb0 con 0x7f7890102e60 2024-04-25T01:06:33.922 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.926+0000 7f7898486700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7890108b30 con 0x7f787c0035c0 2024-04-25T01:06:33.922 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.926+0000 7f7896222700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f787c0035c0 0x7f7890102920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:33.922 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.926+0000 7f7896222700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f787c0035c0 0x7f7890102920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54852/0 (socket says 172.21.15.169:54852) 2024-04-25T01:06:33.922 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.926+0000 7f7896222700 1 -- 172.21.15.169:0/3315262458 learned_addr learned my addr 172.21.15.169:0/3315262458 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T01:06:33.922 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.927+0000 7f7896222700 1 -- 172.21.15.169:0/3315262458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f787c0035c0 msgr2=0x7f7890102920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T01:06:33.922 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.927+0000 7f7896222700 1 -- 172.21.15.169:0/3315262458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f787c0035c0 msgr2=0x7f7890102920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:06:33.922 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.927+0000 7f7896222700 1 --2- 172.21.15.169:0/3315262458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f787c0035c0 0x7f7890102920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:06:33.923 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.927+0000 7f7896222700 1 --2- 172.21.15.169:0/3315262458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f787c0035c0 0x7f7890102920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:06:33.923 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.927+0000 7f7895a21700 1 --2- 172.21.15.169:0/3315262458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7890102e60 0x7f78901032e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:33.923 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.927+0000 7f7895a21700 1 -- 172.21.15.169:0/3315262458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f787c0035c0 msgr2=0x7f7890102920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:06:33.923 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.927+0000 7f7895a21700 1 --2- 172.21.15.169:0/3315262458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f787c0035c0 0x7f7890102920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:33.923 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.927+0000 7f7895a21700 1 -- 172.21.15.169:0/3315262458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7890104840 con 0x7f7890102e60 2024-04-25T01:06:33.923 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.928+0000 7f7895a21700 1 --2- 172.21.15.169:0/3315262458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7890102e60 0x7f78901032e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f788000b4d0 tx=0x7f788000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:06:33.924 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.928+0000 7f78877fe700 1 -- 172.21.15.169:0/3315262458 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f7880011630 con 0x7f7890102e60 2024-04-25T01:06:33.924 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.928+0000 7f7898486700 1 -- 172.21.15.169:0/3315262458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78901a3bc0 con 0x7f7890102e60 2024-04-25T01:06:33.924 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.929+0000 7f78877fe700 1 -- 172.21.15.169:0/3315262458 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7880011cb0 con 0x7f7890102e60 2024-04-25T01:06:33.924 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.929+0000 7f78877fe700 1 -- 172.21.15.169:0/3315262458 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f7880010e60 con 0x7f7890102e60 2024-04-25T01:06:33.924 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.929+0000 7f7898486700 1 -- 172.21.15.169:0/3315262458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78901a4100 con 0x7f7890102e60 2024-04-25T01:06:33.924 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.930+0000 7f7898486700 1 -- 172.21.15.169:0/3315262458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7890102360 con 0x7f7890102e60 2024-04-25T01:06:33.925 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.930+0000 7f78877fe700 1 -- 172.21.15.169:0/3315262458 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7880018020 con 0x7f7890102e60 2024-04-25T01:06:33.925 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.930+0000 7f78877fe700 1 --2- 172.21.15.169:0/3315262458 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7874038e30 0x7f787403b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:33.925 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.930+0000 7f78877fe700 1 -- 172.21.15.169:0/3315262458 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f788004bc40 con 0x7f7890102e60 2024-04-25T01:06:33.925 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.931+0000 7f7896222700 1 --2- 172.21.15.169:0/3315262458 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7874038e30 0x7f787403b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:33.926 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.932+0000 7f7896222700 1 --2- 172.21.15.169:0/3315262458 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7874038e30 0x7f787403b2f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7888019000 tx=0x7f7888005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:06:33.940 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:33.946+0000 7f78877fe700 1 -- 172.21.15.169:0/3315262458 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f788001f020 con 0x7f7890102e60 2024-04-25T01:06:34.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:33 smithi031 ceph-mon[28053]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:34.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:33 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:34.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:33 smithi031 ceph-mon[28053]: Deploying daemon mon.b on smithi040 2024-04-25T01:06:34.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:33 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:06:34.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:33 smithi169 ceph-mon[32759]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:34.203 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:33 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:34.203 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:33 smithi169 ceph-mon[32759]: Deploying daemon mon.b on smithi040 2024-04-25T01:06:34.203 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:33 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:06:34.220 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:34.226+0000 7f7898486700 1 -- 172.21.15.169:0/3315262458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f789002db30 con 0x7f7890102e60 2024-04-25T01:06:34.221 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:34.227+0000 7f78877fe700 1 -- 172.21.15.169:0/3315262458 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f78800193b0 con 0x7f7890102e60 2024-04-25T01:06:34.221 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T01:06:34.221 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":2,"fsid":"95849b94-029f-11ef-bc93-c7b262605968","modified":"2024-04-25T01:06:27.643092Z","created":"2024-04-25T01:04:18.937945Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:3300","nonce":0},{"type":"v1","addr":"172.21.15.169:6789","nonce":0}]},"addr":"172.21.15.169:6789/0","public_addr":"172.21.15.169:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T01:06:34.224 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:34.230+0000 7f7898486700 1 -- 172.21.15.169:0/3315262458 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7874038e30 msgr2=0x7f787403b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:34.224 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:34.230+0000 7f7898486700 1 --2- 172.21.15.169:0/3315262458 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7874038e30 0x7f787403b2f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7888019000 tx=0x7f7888005040 comp rx=0 tx=0).stop 2024-04-25T01:06:34.224 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:34.230+0000 7f7898486700 1 -- 172.21.15.169:0/3315262458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7890102e60 msgr2=0x7f78901032e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:34.224 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:34.230+0000 7f7898486700 1 --2- 172.21.15.169:0/3315262458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7890102e60 0x7f78901032e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f788000b4d0 tx=0x7f788000b890 comp rx=0 tx=0).stop 2024-04-25T01:06:34.224 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:34.230+0000 7f7898486700 1 -- 172.21.15.169:0/3315262458 shutdown_connections 2024-04-25T01:06:34.224 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:34.230+0000 7f7898486700 1 --2- 172.21.15.169:0/3315262458 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7874038e30 0x7f787403b2f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:34.224 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:34.230+0000 7f7898486700 1 --2- 172.21.15.169:0/3315262458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f787c0035c0 0x7f7890102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:34.224 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:34.230+0000 7f7898486700 1 --2- 172.21.15.169:0/3315262458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7890102e60 0x7f78901032e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:34.224 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:34.230+0000 7f7898486700 1 -- 172.21.15.169:0/3315262458 >> 172.21.15.169:0/3315262458 conn(0x7f78900fc710 msgr2=0x7f789006a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:06:34.224 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:34.230+0000 7f7898486700 1 -- 172.21.15.169:0/3315262458 shutdown_connections 2024-04-25T01:06:34.224 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:34.230+0000 7f7898486700 1 -- 172.21.15.169:0/3315262458 wait complete. 2024-04-25T01:06:34.225 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 2 2024-04-25T01:06:34.820 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:34 smithi169 ceph-mon[32759]: from='client.? 172.21.15.169:0/3315262458' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T01:06:35.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:34 smithi031 ceph-mon[28053]: from='client.? 172.21.15.169:0/3315262458' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T01:06:35.848 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T01:06:35.848 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T01:06:36.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:35 smithi031 ceph-mon[28053]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:36.095 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:06:36.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:35 smithi169 ceph-mon[32759]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:36.706 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:36 smithi040 systemd[1]: Started Ceph mon.b for 95849b94-029f-11ef-bc93-c7b262605968. 2024-04-25T01:06:41.927 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.933+0000 7f046d896700 1 -- 172.21.15.169:0/48007713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04500035c0 msgr2=0x7f0450005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:41.927 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.933+0000 7f046d896700 1 --2- 172.21.15.169:0/48007713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04500035c0 0x7f0450005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0468100b70 tx=0x7f045c00fe60 comp rx=0 tx=0).stop 2024-04-25T01:06:41.928 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.933+0000 7f046d896700 1 -- 172.21.15.169:0/48007713 shutdown_connections 2024-04-25T01:06:41.928 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.933+0000 7f046d896700 1 --2- 172.21.15.169:0/48007713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04500035c0 0x7f0450005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:41.928 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.933+0000 7f046d896700 1 -- 172.21.15.169:0/48007713 >> 172.21.15.169:0/48007713 conn(0x7f04680fc770 msgr2=0x7f04680feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:06:41.928 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.933+0000 7f046d896700 1 -- 172.21.15.169:0/48007713 shutdown_connections 2024-04-25T01:06:41.928 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.933+0000 7f046d896700 1 -- 172.21.15.169:0/48007713 wait complete. 2024-04-25T01:06:41.930 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.934+0000 7f046d896700 1 Processor -- start 2024-04-25T01:06:41.930 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.934+0000 7f046d896700 1 -- start start 2024-04-25T01:06:41.930 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.935+0000 7f046d896700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04500035c0 0x7f046819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:41.930 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.935+0000 7f046d896700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f046819dbb0 0x7f04681977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:41.930 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.935+0000 7f046d896700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0468197d30 0x7f04681981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:41.930 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.935+0000 7f046d896700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0468108c80 con 0x7f0468197d30 2024-04-25T01:06:41.930 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.935+0000 7f046d896700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0468108b00 con 0x7f04500035c0 2024-04-25T01:06:41.930 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.935+0000 7f046d896700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0468108e00 con 0x7f046819dbb0 2024-04-25T01:06:41.930 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.935+0000 7f04667fc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f046819dbb0 0x7f04681977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:41.930 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.935+0000 7f04667fc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f046819dbb0 0x7f04681977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54868/0 (socket says 172.21.15.169:54868) 2024-04-25T01:06:41.930 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.935+0000 7f04667fc700 1 -- 172.21.15.169:0/1663860829 learned_addr learned my addr 172.21.15.169:0/1663860829 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T01:06:41.931 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.935+0000 7f04667fc700 1 -- 172.21.15.169:0/1663860829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f046819dbb0 msgr2=0x7f04681977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T01:06:41.931 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.935+0000 7f04667fc700 1 -- 172.21.15.169:0/1663860829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f046819dbb0 msgr2=0x7f04681977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:06:41.931 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.935+0000 7f04667fc700 1 --2- 172.21.15.169:0/1663860829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f046819dbb0 0x7f04681977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:06:41.931 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.935+0000 7f0466ffd700 1 --2- 172.21.15.169:0/1663860829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04500035c0 0x7f046819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:41.931 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.935+0000 7f04667fc700 1 --2- 172.21.15.169:0/1663860829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f046819dbb0 0x7f04681977f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:06:41.931 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.935+0000 7f04677fe700 1 --2- 172.21.15.169:0/1663860829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0468197d30 0x7f04681981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:41.931 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.935+0000 7f0466ffd700 1 -- 172.21.15.169:0/1663860829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04500035c0 msgr2=0x7f046819d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T01:06:41.931 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.935+0000 7f0466ffd700 1 -- 172.21.15.169:0/1663860829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04500035c0 msgr2=0x7f046819d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:06:41.931 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.935+0000 7f0466ffd700 1 --2- 172.21.15.169:0/1663860829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04500035c0 0x7f046819d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:06:41.931 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.935+0000 7f0466ffd700 1 --2- 172.21.15.169:0/1663860829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04500035c0 0x7f046819d670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:06:41.931 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.936+0000 7f04677fe700 1 -- 172.21.15.169:0/1663860829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f046819dbb0 msgr2=0x7f04681977f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:06:41.931 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.936+0000 7f04677fe700 1 --2- 172.21.15.169:0/1663860829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f046819dbb0 0x7f04681977f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:41.931 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.936+0000 7f04677fe700 1 -- 172.21.15.169:0/1663860829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04500035c0 msgr2=0x7f046819d670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:06:41.932 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.936+0000 7f04677fe700 1 --2- 172.21.15.169:0/1663860829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04500035c0 0x7f046819d670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:41.932 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.936+0000 7f04677fe700 1 -- 172.21.15.169:0/1663860829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04681a44d0 con 0x7f0468197d30 2024-04-25T01:06:41.933 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.936+0000 7f04677fe700 1 --2- 172.21.15.169:0/1663860829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0468197d30 0x7f04681981e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f046000bd50 tx=0x7f046000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:06:41.933 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.937+0000 7f046c894700 1 -- 172.21.15.169:0/1663860829 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460019070 con 0x7f0468197d30 2024-04-25T01:06:41.933 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.937+0000 7f046d896700 1 -- 172.21.15.169:0/1663860829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04681a4760 con 0x7f0468197d30 2024-04-25T01:06:41.933 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.937+0000 7f046c894700 1 -- 172.21.15.169:0/1663860829 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f046000cd10 con 0x7f0468197d30 2024-04-25T01:06:41.933 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.937+0000 7f046c894700 1 -- 172.21.15.169:0/1663860829 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04600127e0 con 0x7f0468197d30 2024-04-25T01:06:41.933 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.937+0000 7f046d896700 1 -- 172.21.15.169:0/1663860829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04681a4cc0 con 0x7f0468197d30 2024-04-25T01:06:41.933 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.938+0000 7f046d896700 1 -- 172.21.15.169:0/1663860829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0468102380 con 0x7f0468197d30 2024-04-25T01:06:41.933 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.939+0000 7f046c894700 1 -- 172.21.15.169:0/1663860829 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f046001c070 con 0x7f0468197d30 2024-04-25T01:06:41.934 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.939+0000 7f046c894700 1 --2- 172.21.15.169:0/1663860829 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0454039580 0x7f045403ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:41.934 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.939+0000 7f046c894700 1 -- 172.21.15.169:0/1663860829 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f046004bae0 con 0x7f0468197d30 2024-04-25T01:06:41.934 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.940+0000 7f0466ffd700 1 --2- 172.21.15.169:0/1663860829 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0454039580 0x7f045403ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:41.934 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.941+0000 7f0466ffd700 1 --2- 172.21.15.169:0/1663860829 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0454039580 0x7f045403ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f045c020040 tx=0x7f045c001e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:06:41.948 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:41.954+0000 7f046c894700 1 -- 172.21.15.169:0/1663860829 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f046001d3b0 con 0x7f0468197d30 2024-04-25T01:06:41.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:41.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: mon.c calling monitor election 2024-04-25T01:06:41.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:06:41.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: mon.a calling monitor election 2024-04-25T01:06:41.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:06:41.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:06:41.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:06:41.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:41.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:06:41.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:06:41.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:41.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:06:41.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:06:41.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T01:06:41.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: monmap epoch 3 2024-04-25T01:06:41.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: fsid 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:06:41.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: last_changed 2024-04-25T01:06:36.621868+0000 2024-04-25T01:06:41.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: created 2024-04-25T01:04:18.937945+0000 2024-04-25T01:06:41.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: min_mon_release 19 (squid) 2024-04-25T01:06:41.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: election_strategy: 1 2024-04-25T01:06:41.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T01:06:41.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T01:06:41.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: 2: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.b 2024-04-25T01:06:41.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: fsmap 2024-04-25T01:06:41.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T01:06:41.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: mgrmap e13: x(active, since 69s) 2024-04-25T01:06:41.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: overall HEALTH_OK 2024-04-25T01:06:41.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:41.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:41.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:41.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:42.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:42.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: mon.c calling monitor election 2024-04-25T01:06:42.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:06:42.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: mon.a calling monitor election 2024-04-25T01:06:42.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:06:42.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:06:42.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:06:42.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:42.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:06:42.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:06:42.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:42.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:06:42.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:06:42.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T01:06:42.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: monmap epoch 3 2024-04-25T01:06:42.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: fsid 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:06:42.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: last_changed 2024-04-25T01:06:36.621868+0000 2024-04-25T01:06:42.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: created 2024-04-25T01:04:18.937945+0000 2024-04-25T01:06:42.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: min_mon_release 19 (squid) 2024-04-25T01:06:42.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: election_strategy: 1 2024-04-25T01:06:42.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T01:06:42.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T01:06:42.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: 2: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.b 2024-04-25T01:06:42.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: fsmap 2024-04-25T01:06:42.016 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T01:06:42.016 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: mgrmap e13: x(active, since 69s) 2024-04-25T01:06:42.016 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: overall HEALTH_OK 2024-04-25T01:06:42.016 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:42.016 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:42.016 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:42.016 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:42.237 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:42.243+0000 7f046d896700 1 -- 172.21.15.169:0/1663860829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f04681a4fb0 con 0x7f0468197d30 2024-04-25T01:06:42.237 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:42.244+0000 7f046c894700 1 -- 172.21.15.169:0/1663860829 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f0460014ab0 con 0x7f0468197d30 2024-04-25T01:06:42.238 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T01:06:42.238 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":3,"fsid":"95849b94-029f-11ef-bc93-c7b262605968","modified":"2024-04-25T01:06:36.621868Z","created":"2024-04-25T01:04:18.937945Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:3300","nonce":0},{"type":"v1","addr":"172.21.15.169:6789","nonce":0}]},"addr":"172.21.15.169:6789/0","public_addr":"172.21.15.169:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T01:06:42.240 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:42.246+0000 7f046d896700 1 -- 172.21.15.169:0/1663860829 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0454039580 msgr2=0x7f045403ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:42.240 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:42.246+0000 7f046d896700 1 --2- 172.21.15.169:0/1663860829 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0454039580 0x7f045403ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f045c020040 tx=0x7f045c001e70 comp rx=0 tx=0).stop 2024-04-25T01:06:42.240 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:42.246+0000 7f046d896700 1 -- 172.21.15.169:0/1663860829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0468197d30 msgr2=0x7f04681981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:42.240 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:42.246+0000 7f046d896700 1 --2- 172.21.15.169:0/1663860829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0468197d30 0x7f04681981e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f046000bd50 tx=0x7f046000be30 comp rx=0 tx=0).stop 2024-04-25T01:06:42.240 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:42.246+0000 7f046d896700 1 -- 172.21.15.169:0/1663860829 shutdown_connections 2024-04-25T01:06:42.240 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:42.246+0000 7f046d896700 1 --2- 172.21.15.169:0/1663860829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0468197d30 0x7f04681981e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:42.240 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:42.246+0000 7f046d896700 1 --2- 172.21.15.169:0/1663860829 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0454039580 0x7f045403ba40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:42.240 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:42.246+0000 7f046d896700 1 --2- 172.21.15.169:0/1663860829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04500035c0 0x7f046819d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:42.240 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:42.246+0000 7f046d896700 1 --2- 172.21.15.169:0/1663860829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f046819dbb0 0x7f04681977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:42.240 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:42.246+0000 7f046d896700 1 -- 172.21.15.169:0/1663860829 >> 172.21.15.169:0/1663860829 conn(0x7f04680fc770 msgr2=0x7f0468104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:06:42.240 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:42.246+0000 7f046d896700 1 -- 172.21.15.169:0/1663860829 shutdown_connections 2024-04-25T01:06:42.241 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:42.246+0000 7f046d896700 1 -- 172.21.15.169:0/1663860829 wait complete. 2024-04-25T01:06:42.241 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 3 2024-04-25T01:06:42.752 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T01:06:42.752 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T01:06:43.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:42 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:06:43.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:42 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:43.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:42 smithi031 ceph-mon[28053]: from='client.? 172.21.15.169:0/1663860829' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T01:06:43.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:42 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:06:43.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:42 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:06:43.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:42 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:43.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:42 smithi169 ceph-mon[32759]: from='client.? 172.21.15.169:0/1663860829' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T01:06:43.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:42 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:06:43.221 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:06:43.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:43.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: mon.c calling monitor election 2024-04-25T01:06:43.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:06:43.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: mon.a calling monitor election 2024-04-25T01:06:43.957 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:06:43.957 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:06:43.957 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:06:43.957 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:43.957 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:06:43.957 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:06:43.957 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:43.957 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:06:43.957 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:06:43.957 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T01:06:43.957 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: monmap epoch 3 2024-04-25T01:06:43.957 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: fsid 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:06:43.958 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: last_changed 2024-04-25T01:06:36.621868+0000 2024-04-25T01:06:43.958 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: created 2024-04-25T01:04:18.937945+0000 2024-04-25T01:06:43.958 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: min_mon_release 19 (squid) 2024-04-25T01:06:43.958 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: election_strategy: 1 2024-04-25T01:06:43.958 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T01:06:43.958 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T01:06:43.958 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: 2: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.b 2024-04-25T01:06:43.958 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: fsmap 2024-04-25T01:06:43.958 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T01:06:43.958 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: mgrmap e13: x(active, since 69s) 2024-04-25T01:06:43.958 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: overall HEALTH_OK 2024-04-25T01:06:43.958 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:43.959 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:43.959 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:43.959 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:43.959 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:06:43.959 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:43.959 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: from='client.? 172.21.15.169:0/1663860829' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T01:06:43.959 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:43 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:06:44.830 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: mon.b calling monitor election 2024-04-25T01:06:44.830 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: mon.b calling monitor election 2024-04-25T01:06:44.830 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: mon.c calling monitor election 2024-04-25T01:06:44.830 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: mon.a calling monitor election 2024-04-25T01:06:44.830 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T01:06:44.830 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:06:44.830 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: monmap epoch 3 2024-04-25T01:06:44.830 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: fsid 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:06:44.830 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: last_changed 2024-04-25T01:06:36.621868+0000 2024-04-25T01:06:44.830 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: created 2024-04-25T01:04:18.937945+0000 2024-04-25T01:06:44.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: min_mon_release 19 (squid) 2024-04-25T01:06:44.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: election_strategy: 1 2024-04-25T01:06:44.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T01:06:44.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T01:06:44.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: 2: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.b 2024-04-25T01:06:44.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: fsmap 2024-04-25T01:06:44.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T01:06:44.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: mgrmap e13: x(active, since 71s) 2024-04-25T01:06:44.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: overall HEALTH_OK 2024-04-25T01:06:44.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-25T01:06:44.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: Updating smithi040:/etc/ceph/ceph.conf 2024-04-25T01:06:44.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T01:06:44.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:44.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:44.832 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:44.832 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:44 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:44.953 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: mon.b calling monitor election 2024-04-25T01:06:44.953 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: mon.b calling monitor election 2024-04-25T01:06:44.953 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: mon.c calling monitor election 2024-04-25T01:06:44.953 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: mon.a calling monitor election 2024-04-25T01:06:44.953 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T01:06:44.953 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:06:44.954 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: monmap epoch 3 2024-04-25T01:06:44.954 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: fsid 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:06:44.954 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: last_changed 2024-04-25T01:06:36.621868+0000 2024-04-25T01:06:44.954 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: created 2024-04-25T01:04:18.937945+0000 2024-04-25T01:06:44.954 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: min_mon_release 19 (squid) 2024-04-25T01:06:44.954 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: election_strategy: 1 2024-04-25T01:06:44.954 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T01:06:44.954 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T01:06:44.954 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: 2: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.b 2024-04-25T01:06:44.954 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: fsmap 2024-04-25T01:06:44.954 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T01:06:44.954 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: mgrmap e13: x(active, since 71s) 2024-04-25T01:06:44.954 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: overall HEALTH_OK 2024-04-25T01:06:44.955 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-25T01:06:44.955 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: Updating smithi040:/etc/ceph/ceph.conf 2024-04-25T01:06:44.955 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T01:06:44.955 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:44.955 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:44.955 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:44.955 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:44 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:44.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: mon.b calling monitor election 2024-04-25T01:06:44.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: mon.b calling monitor election 2024-04-25T01:06:44.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: mon.c calling monitor election 2024-04-25T01:06:44.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: mon.a calling monitor election 2024-04-25T01:06:44.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T01:06:44.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:06:44.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: monmap epoch 3 2024-04-25T01:06:44.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: fsid 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:06:44.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: last_changed 2024-04-25T01:06:36.621868+0000 2024-04-25T01:06:44.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: created 2024-04-25T01:04:18.937945+0000 2024-04-25T01:06:44.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: min_mon_release 19 (squid) 2024-04-25T01:06:44.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: election_strategy: 1 2024-04-25T01:06:44.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T01:06:44.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T01:06:44.957 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: 2: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.b 2024-04-25T01:06:44.957 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: fsmap 2024-04-25T01:06:44.957 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T01:06:44.957 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: mgrmap e13: x(active, since 71s) 2024-04-25T01:06:44.957 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: overall HEALTH_OK 2024-04-25T01:06:44.957 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-25T01:06:44.957 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: Updating smithi040:/etc/ceph/ceph.conf 2024-04-25T01:06:44.957 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T01:06:44.957 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:44.957 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:44.957 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:44.957 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:44 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:45.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.246+0000 7f8f0d000700 1 -- 172.21.15.31:0/4229778061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f08101ea0 msgr2=0x7f8f08102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:45.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.246+0000 7f8f0d000700 1 --2- 172.21.15.31:0/4229778061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f08101ea0 0x7f8f08102280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009910 tx=0x7f8ef8009c20 comp rx=0 tx=0).stop 2024-04-25T01:06:45.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.247+0000 7f8f0d000700 1 -- 172.21.15.31:0/4229778061 shutdown_connections 2024-04-25T01:06:45.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.247+0000 7f8f0d000700 1 --2- 172.21.15.31:0/4229778061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f08101ea0 0x7f8f08102280 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:45.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.247+0000 7f8f0d000700 1 -- 172.21.15.31:0/4229778061 >> 172.21.15.31:0/4229778061 conn(0x7f8f080fd770 msgr2=0x7f8f080ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:06:45.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.247+0000 7f8f0d000700 1 -- 172.21.15.31:0/4229778061 shutdown_connections 2024-04-25T01:06:45.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.247+0000 7f8f0d000700 1 -- 172.21.15.31:0/4229778061 wait complete. 2024-04-25T01:06:45.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.249+0000 7f8f0d000700 1 Processor -- start 2024-04-25T01:06:45.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.249+0000 7f8f0d000700 1 -- start start 2024-04-25T01:06:45.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.249+0000 7f8f0d000700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f08101ea0 0x7f8f08074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:45.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.249+0000 7f8f0d000700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f08074a70 0x7f8f08074ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:45.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.249+0000 7f8f0d000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f081a3bb0 0x7f8f081a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:45.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.249+0000 7f8f0d000700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0810c270 con 0x7f8f08101ea0 2024-04-25T01:06:45.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.249+0000 7f8f0d000700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0810c0f0 con 0x7f8f08074a70 2024-04-25T01:06:45.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.249+0000 7f8f0d000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0810c3f0 con 0x7f8f081a3bb0 2024-04-25T01:06:45.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.250+0000 7f8f05400700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f08074a70 0x7f8f08074ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:45.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.250+0000 7f8f05e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f08101ea0 0x7f8f08074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:45.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.250+0000 7f8f05400700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f08074a70 0x7f8f08074ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:34564/0 (socket says 172.21.15.31:34564) 2024-04-25T01:06:45.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.250+0000 7f8f05400700 1 -- 172.21.15.31:0/3329770751 learned_addr learned my addr 172.21.15.31:0/3329770751 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:06:45.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.250+0000 7f8f06800700 1 --2- 172.21.15.31:0/3329770751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f081a3bb0 0x7f8f081a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:45.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.250+0000 7f8f05400700 1 -- 172.21.15.31:0/3329770751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f081a3bb0 msgr2=0x7f8f081a5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:45.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.250+0000 7f8f05400700 1 --2- 172.21.15.31:0/3329770751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f081a3bb0 0x7f8f081a5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:45.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.250+0000 7f8f05400700 1 -- 172.21.15.31:0/3329770751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f08101ea0 msgr2=0x7f8f08074530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:45.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.250+0000 7f8f05400700 1 --2- 172.21.15.31:0/3329770751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f08101ea0 0x7f8f08074530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:45.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.250+0000 7f8f05400700 1 -- 172.21.15.31:0/3329770751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f081a66d0 con 0x7f8f08074a70 2024-04-25T01:06:45.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.250+0000 7f8f05e00700 1 --2- 172.21.15.31:0/3329770751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f08101ea0 0x7f8f08074530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:06:45.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.251+0000 7f8f05400700 1 --2- 172.21.15.31:0/3329770751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f08074a70 0x7f8f08074ef0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8ef400b780 tx=0x7f8ef400ba90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:06:45.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.251+0000 7f8efea00700 1 -- 172.21.15.31:0/3329770751 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4016070 con 0x7f8f08074a70 2024-04-25T01:06:45.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.251+0000 7f8f0d000700 1 -- 172.21.15.31:0/3329770751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f081a69c0 con 0x7f8f08074a70 2024-04-25T01:06:45.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.251+0000 7f8efea00700 1 -- 172.21.15.31:0/3329770751 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ef40036a0 con 0x7f8f08074a70 2024-04-25T01:06:45.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.251+0000 7f8efea00700 1 -- 172.21.15.31:0/3329770751 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4011af0 con 0x7f8f08074a70 2024-04-25T01:06:45.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.251+0000 7f8f0d000700 1 -- 172.21.15.31:0/3329770751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f081a6f50 con 0x7f8f08074a70 2024-04-25T01:06:45.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.252+0000 7f8f0d000700 1 -- 172.21.15.31:0/3329770751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f08063890 con 0x7f8f08074a70 2024-04-25T01:06:45.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.253+0000 7f8efea00700 1 -- 172.21.15.31:0/3329770751 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f8ef4019070 con 0x7f8f08074a70 2024-04-25T01:06:45.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.253+0000 7f8efea00700 1 --2- 172.21.15.31:0/3329770751 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f8eec039580 0x7f8eec03ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:45.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.253+0000 7f8efea00700 1 -- 172.21.15.31:0/3329770751 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8ef404ae30 con 0x7f8f08074a70 2024-04-25T01:06:45.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.254+0000 7f8f05e00700 1 --2- 172.21.15.31:0/3329770751 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f8eec039580 0x7f8eec03ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:45.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.254+0000 7f8f05e00700 1 --2- 172.21.15.31:0/3329770751 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f8eec039580 0x7f8eec03ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8006010 tx=0x7f8ef80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:06:45.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.259+0000 7f8efea00700 1 -- 172.21.15.31:0/3329770751 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8ef4019970 con 0x7f8f08074a70 2024-04-25T01:06:45.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.494+0000 7f8f0d000700 1 -- 172.21.15.31:0/3329770751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f8f081a7280 con 0x7f8f08074a70 2024-04-25T01:06:45.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.494+0000 7f8efea00700 1 -- 172.21.15.31:0/3329770751 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f8ef4019bd0 con 0x7f8f08074a70 2024-04-25T01:06:45.489 INFO:teuthology.orchestra.run.smithi031.stdout:# minimal ceph.conf for 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:06:45.489 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2024-04-25T01:06:45.489 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:06:45.489 INFO:teuthology.orchestra.run.smithi031.stdout: mon_host = [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] 2024-04-25T01:06:45.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.497+0000 7f8f0d000700 1 -- 172.21.15.31:0/3329770751 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f8eec039580 msgr2=0x7f8eec03ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:45.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.497+0000 7f8f0d000700 1 --2- 172.21.15.31:0/3329770751 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f8eec039580 0x7f8eec03ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8006010 tx=0x7f8ef80058e0 comp rx=0 tx=0).stop 2024-04-25T01:06:45.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.497+0000 7f8f0d000700 1 -- 172.21.15.31:0/3329770751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f08074a70 msgr2=0x7f8f08074ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:45.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.497+0000 7f8f0d000700 1 --2- 172.21.15.31:0/3329770751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f08074a70 0x7f8f08074ef0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8ef400b780 tx=0x7f8ef400ba90 comp rx=0 tx=0).stop 2024-04-25T01:06:45.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.498+0000 7f8f0d000700 1 -- 172.21.15.31:0/3329770751 shutdown_connections 2024-04-25T01:06:45.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.498+0000 7f8f0d000700 1 --2- 172.21.15.31:0/3329770751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f08101ea0 0x7f8f08074530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:45.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.498+0000 7f8f0d000700 1 --2- 172.21.15.31:0/3329770751 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f8eec039580 0x7f8eec03ba40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:45.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.498+0000 7f8f0d000700 1 --2- 172.21.15.31:0/3329770751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f081a3bb0 0x7f8f081a5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:45.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.498+0000 7f8f0d000700 1 --2- 172.21.15.31:0/3329770751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f08074a70 0x7f8f08074ef0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:45.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.498+0000 7f8f0d000700 1 -- 172.21.15.31:0/3329770751 >> 172.21.15.31:0/3329770751 conn(0x7f8f080fd770 msgr2=0x7f8f08103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:06:45.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.498+0000 7f8f0d000700 1 -- 172.21.15.31:0/3329770751 shutdown_connections 2024-04-25T01:06:45.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:45.498+0000 7f8f0d000700 1 -- 172.21.15.31:0/3329770751 wait complete. 2024-04-25T01:06:45.685 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:45 smithi031 ceph-mon[28053]: Updating smithi169:/var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:06:45.685 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:45 smithi031 ceph-mon[28053]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:45.685 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:45 smithi031 ceph-mon[28053]: Updating smithi040:/var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:06:45.685 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:45 smithi031 ceph-mon[28053]: Updating smithi031:/var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:06:45.685 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:45 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:06:45.685 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:45 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:45.685 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:45 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:45.685 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:45 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:45.686 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:45 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:45.686 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:45 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:45.686 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:45 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:45.686 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:45 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:45.686 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:45 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:06:45.686 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:45 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:06:45.686 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:45 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:45.686 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:45 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3329770751' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:45.952 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:45 smithi169 ceph-mon[32759]: Updating smithi169:/var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:06:45.952 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:45 smithi169 ceph-mon[32759]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:45.952 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:45 smithi169 ceph-mon[32759]: Updating smithi040:/var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:06:45.953 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:45 smithi169 ceph-mon[32759]: Updating smithi031:/var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:06:45.953 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:45 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:06:45.953 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:45 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:45.953 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:45 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:45.953 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:45 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:45.953 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:45 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:45.953 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:45 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:45.953 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:45 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:45.953 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:45 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:45.953 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:45 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:06:45.953 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:45 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:06:45.953 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:45 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:45.953 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:45 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3329770751' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:45.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:45 smithi040 ceph-mon[32518]: Updating smithi169:/var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:06:45.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:45 smithi040 ceph-mon[32518]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:45.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:45 smithi040 ceph-mon[32518]: Updating smithi040:/var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:06:45.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:45 smithi040 ceph-mon[32518]: Updating smithi031:/var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:06:45.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:45 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:06:45.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:45 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:45.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:45 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:45.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:45 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:45.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:45 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:45.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:45 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:45.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:45 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:45.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:45 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:45.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:45 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:06:45.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:45 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:06:45.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:45 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:45.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:45 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3329770751' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:46.229 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T01:06:46.229 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T01:06:46.229 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T01:06:46.310 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T01:06:46.310 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:06:46.420 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T01:06:46.420 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T01:06:46.453 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T01:06:46.453 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:06:46.521 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T01:06:46.521 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T01:06:46.548 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T01:06:46.548 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:06:46.613 INFO:tasks.cephadm:Adding mgr.x on smithi031 2024-04-25T01:06:46.613 INFO:tasks.cephadm:Adding mgr.y on smithi040 2024-04-25T01:06:46.613 INFO:tasks.cephadm:Adding mgr.z on smithi169 2024-04-25T01:06:46.613 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi031=x;smithi040=y;smithi169=z' 2024-04-25T01:06:46.705 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:46 smithi040 ceph-mon[32518]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T01:06:46.706 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:46 smithi040 ceph-mon[32518]: Reconfiguring daemon mon.a on smithi031 2024-04-25T01:06:46.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:46 smithi031 ceph-mon[28053]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T01:06:46.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:46 smithi031 ceph-mon[28053]: Reconfiguring daemon mon.a on smithi031 2024-04-25T01:06:46.881 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:06:46.952 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:46 smithi169 ceph-mon[32759]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T01:06:46.952 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:46 smithi169 ceph-mon[32759]: Reconfiguring daemon mon.a on smithi031 2024-04-25T01:06:47.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:47 smithi031 ceph-mon[28053]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:47.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:47 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:47.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:47 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:47.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:47 smithi169 ceph-mon[32759]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:47.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:47 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:47.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:47 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:47.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:47 smithi040 ceph-mon[32518]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:47.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:47 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:47.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:47 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:48.257 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.263+0000 7f025c903700 1 -- 172.21.15.169:0/658274440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0258100c90 msgr2=0x7f02581010b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:48.257 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.263+0000 7f025c903700 1 --2- 172.21.15.169:0/658274440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0258100c90 0x7f02581010b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0248009910 tx=0x7f0248009c20 comp rx=0 tx=0).stop 2024-04-25T01:06:48.257 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.263+0000 7f025c903700 1 -- 172.21.15.169:0/658274440 shutdown_connections 2024-04-25T01:06:48.258 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.263+0000 7f025c903700 1 --2- 172.21.15.169:0/658274440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0258100c90 0x7f02581010b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:48.258 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.263+0000 7f025c903700 1 -- 172.21.15.169:0/658274440 >> 172.21.15.169:0/658274440 conn(0x7f02580fc230 msgr2=0x7f02580fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:06:48.258 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.263+0000 7f025c903700 1 -- 172.21.15.169:0/658274440 shutdown_connections 2024-04-25T01:06:48.258 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.263+0000 7f025c903700 1 -- 172.21.15.169:0/658274440 wait complete. 2024-04-25T01:06:48.260 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.264+0000 7f025c903700 1 Processor -- start 2024-04-25T01:06:48.260 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.264+0000 7f025c903700 1 -- start start 2024-04-25T01:06:48.260 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.264+0000 7f025c903700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0258100c90 0x7f02581975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:48.261 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.264+0000 7f025c903700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0258197af0 0x7f025819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:48.261 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.264+0000 7f025c903700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0258197f70 0x7f025819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:48.261 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.264+0000 7f025c903700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0258106cd0 con 0x7f0258197f70 2024-04-25T01:06:48.261 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.264+0000 7f025c903700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0258106b50 con 0x7f0258100c90 2024-04-25T01:06:48.261 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.264+0000 7f025c903700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0258106e50 con 0x7f0258197af0 2024-04-25T01:06:48.261 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.264+0000 7f024ffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0258197af0 0x7f025819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:48.261 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.264+0000 7f024ffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0258197af0 0x7f025819cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:50050/0 (socket says 172.21.15.169:50050) 2024-04-25T01:06:48.261 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.264+0000 7f024ffff700 1 -- 172.21.15.169:0/3282444152 learned_addr learned my addr 172.21.15.169:0/3282444152 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T01:06:48.261 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.264+0000 7f025659c700 1 --2- 172.21.15.169:0/3282444152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0258100c90 0x7f02581975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:48.261 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.265+0000 7f024ffff700 1 -- 172.21.15.169:0/3282444152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0258100c90 msgr2=0x7f02581975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:48.261 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.265+0000 7f024ffff700 1 --2- 172.21.15.169:0/3282444152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0258100c90 0x7f02581975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:48.262 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.265+0000 7f024ffff700 1 -- 172.21.15.169:0/3282444152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0258197f70 msgr2=0x7f025819f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:48.262 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.265+0000 7f0256d9d700 1 --2- 172.21.15.169:0/3282444152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0258197f70 0x7f025819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:48.262 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.265+0000 7f024ffff700 1 --2- 172.21.15.169:0/3282444152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0258197f70 0x7f025819f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:48.262 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.265+0000 7f024ffff700 1 -- 172.21.15.169:0/3282444152 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025819f9c0 con 0x7f0258197af0 2024-04-25T01:06:48.262 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.265+0000 7f024ffff700 1 --2- 172.21.15.169:0/3282444152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0258197af0 0x7f025819cb60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f024000b780 tx=0x7f024000ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:06:48.262 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.265+0000 7f024f7fe700 1 -- 172.21.15.169:0/3282444152 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240004230 con 0x7f0258197af0 2024-04-25T01:06:48.262 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.265+0000 7f024f7fe700 1 -- 172.21.15.169:0/3282444152 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02400043d0 con 0x7f0258197af0 2024-04-25T01:06:48.262 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.265+0000 7f025c903700 1 -- 172.21.15.169:0/3282444152 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025819fcb0 con 0x7f0258197af0 2024-04-25T01:06:48.262 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.265+0000 7f024f7fe700 1 -- 172.21.15.169:0/3282444152 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240011c90 con 0x7f0258197af0 2024-04-25T01:06:48.262 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.265+0000 7f025c903700 1 -- 172.21.15.169:0/3282444152 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02581a0220 con 0x7f0258197af0 2024-04-25T01:06:48.262 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.266+0000 7f024f7fe700 1 -- 172.21.15.169:0/3282444152 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f024001a400 con 0x7f0258197af0 2024-04-25T01:06:48.262 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.266+0000 7f025c903700 1 -- 172.21.15.169:0/3282444152 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0258063890 con 0x7f0258197af0 2024-04-25T01:06:48.262 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.266+0000 7f024f7fe700 1 --2- 172.21.15.169:0/3282444152 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0244039580 0x7f024403ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:48.262 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.266+0000 7f024f7fe700 1 -- 172.21.15.169:0/3282444152 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f024004beb0 con 0x7f0258197af0 2024-04-25T01:06:48.263 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.267+0000 7f025659c700 1 --2- 172.21.15.169:0/3282444152 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0244039580 0x7f024403ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:48.263 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.268+0000 7f025659c700 1 --2- 172.21.15.169:0/3282444152 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0244039580 0x7f024403ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0248005200 tx=0x7f024801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:06:48.265 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.272+0000 7f024f7fe700 1 -- 172.21.15.169:0/3282444152 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f024001a840 con 0x7f0258197af0 2024-04-25T01:06:48.471 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.477+0000 7f025c903700 1 -- 172.21.15.169:0/3282444152 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi040=y;smithi169=z", "target": ["mon-mgr", ""]}) -- 0x7f0258062670 con 0x7f0244039580 2024-04-25T01:06:48.486 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.492+0000 7f024f7fe700 1 -- 172.21.15.169:0/3282444152 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0258062670 con 0x7f0244039580 2024-04-25T01:06:48.486 INFO:teuthology.orchestra.run.smithi169.stdout:Scheduled mgr update... 2024-04-25T01:06:48.488 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.494+0000 7f025c903700 1 -- 172.21.15.169:0/3282444152 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0244039580 msgr2=0x7f024403ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:48.488 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.494+0000 7f025c903700 1 --2- 172.21.15.169:0/3282444152 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0244039580 0x7f024403ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0248005200 tx=0x7f024801a040 comp rx=0 tx=0).stop 2024-04-25T01:06:48.488 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.494+0000 7f025c903700 1 -- 172.21.15.169:0/3282444152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0258197af0 msgr2=0x7f025819cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:48.488 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.494+0000 7f025c903700 1 --2- 172.21.15.169:0/3282444152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0258197af0 0x7f025819cb60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f024000b780 tx=0x7f024000ba90 comp rx=0 tx=0).stop 2024-04-25T01:06:48.489 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.494+0000 7f025c903700 1 -- 172.21.15.169:0/3282444152 shutdown_connections 2024-04-25T01:06:48.489 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.494+0000 7f025c903700 1 --2- 172.21.15.169:0/3282444152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0258197f70 0x7f025819f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:48.489 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.494+0000 7f025c903700 1 --2- 172.21.15.169:0/3282444152 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0244039580 0x7f024403ba40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:48.489 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.494+0000 7f025c903700 1 --2- 172.21.15.169:0/3282444152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0258100c90 0x7f02581975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:48.489 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.494+0000 7f025c903700 1 --2- 172.21.15.169:0/3282444152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0258197af0 0x7f025819cb60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:48.489 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.494+0000 7f025c903700 1 -- 172.21.15.169:0/3282444152 >> 172.21.15.169:0/3282444152 conn(0x7f02580fc230 msgr2=0x7f025806a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:06:48.489 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.495+0000 7f025c903700 1 -- 172.21.15.169:0/3282444152 shutdown_connections 2024-04-25T01:06:48.489 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:06:48.495+0000 7f025c903700 1 -- 172.21.15.169:0/3282444152 wait complete. 2024-04-25T01:06:48.772 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:48 smithi169 ceph-mon[32759]: Reconfiguring mon.b (monmap changed)... 2024-04-25T01:06:48.772 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:48 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:06:48.772 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:48 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:06:48.773 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:48 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:48.773 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:48 smithi169 ceph-mon[32759]: Reconfiguring daemon mon.b on smithi040 2024-04-25T01:06:48.773 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:48 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:48.869 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:48 smithi040 ceph-mon[32518]: Reconfiguring mon.b (monmap changed)... 2024-04-25T01:06:48.869 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:48 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:06:48.869 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:48 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:06:48.869 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:48 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:48.869 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:48 smithi040 ceph-mon[32518]: Reconfiguring daemon mon.b on smithi040 2024-04-25T01:06:48.870 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:48 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:49.004 DEBUG:teuthology.orchestra.run.smithi040:mgr.y> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T01:06:49.006 DEBUG:teuthology.orchestra.run.smithi169:mgr.z> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T01:06:49.009 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T01:06:49.009 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T01:06:49.009 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T01:06:49.017 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:48 smithi031 ceph-mon[28053]: Reconfiguring mon.b (monmap changed)... 2024-04-25T01:06:49.017 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:48 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:06:49.017 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:48 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:06:49.017 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:48 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:49.017 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:48 smithi031 ceph-mon[28053]: Reconfiguring daemon mon.b on smithi040 2024-04-25T01:06:49.017 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:48 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:49.070 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T01:06:49.070 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2024-04-25T01:06:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T01:06:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:06:49.127 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 6h/6d Inode: 639 Links: 1 2024-04-25T01:06:49.127 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:06:49.127 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:06:49.127 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-04-25 01:05:48.448956187 +0000 2024-04-25T01:06:49.127 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-04-25 01:00:31.250726608 +0000 2024-04-25T01:06:49.127 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-04-25 01:00:31.250726608 +0000 2024-04-25T01:06:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-04-25 01:00:31.250726608 +0000 2024-04-25T01:06:49.128 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T01:06:49.221 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-04-25T01:06:49.221 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-04-25T01:06:49.221 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000394327 s, 1.3 MB/s 2024-04-25T01:06:49.225 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T01:06:49.285 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2024-04-25T01:06:49.334 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T01:06:49.335 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:06:49.335 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 6h/6d Inode: 659 Links: 1 2024-04-25T01:06:49.335 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:06:49.335 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:06:49.335 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-04-25 01:05:48.525954952 +0000 2024-04-25T01:06:49.335 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-04-25 01:00:31.865716703 +0000 2024-04-25T01:06:49.335 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-04-25 01:00:31.865716703 +0000 2024-04-25T01:06:49.335 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-04-25 01:00:31.865716703 +0000 2024-04-25T01:06:49.335 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T01:06:49.417 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-04-25T01:06:49.417 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-04-25T01:06:49.417 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000356195 s, 1.4 MB/s 2024-04-25T01:06:49.421 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T01:06:49.480 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2024-04-25T01:06:49.534 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T01:06:49.535 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:06:49.535 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 6h/6d Inode: 677 Links: 1 2024-04-25T01:06:49.535 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:06:49.535 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:06:49.535 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-04-25 01:05:48.603953700 +0000 2024-04-25T01:06:49.535 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-04-25 01:00:32.498706506 +0000 2024-04-25T01:06:49.535 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-04-25 01:00:32.498706506 +0000 2024-04-25T01:06:49.535 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-04-25 01:00:32.498706506 +0000 2024-04-25T01:06:49.536 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T01:06:49.617 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-04-25T01:06:49.618 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-04-25T01:06:49.618 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.00039233 s, 1.3 MB/s 2024-04-25T01:06:49.622 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T01:06:49.682 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2024-04-25T01:06:49.738 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T01:06:49.738 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:06:49.738 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 6h/6d Inode: 697 Links: 1 2024-04-25T01:06:49.738 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:06:49.738 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:06:49.738 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-04-25 01:05:48.661952770 +0000 2024-04-25T01:06:49.739 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-04-25 01:00:33.215694958 +0000 2024-04-25T01:06:49.739 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-04-25 01:00:33.215694958 +0000 2024-04-25T01:06:49.739 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-04-25 01:00:33.215694958 +0000 2024-04-25T01:06:49.739 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T01:06:49.823 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-04-25T01:06:49.823 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-04-25T01:06:49.823 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000384894 s, 1.3 MB/s 2024-04-25T01:06:49.827 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T01:06:49.887 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-25T01:06:49.887 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T01:06:49.906 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T01:06:49.906 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_1 2024-04-25T01:06:49.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:49 smithi169 ceph-mon[32759]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:49.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:49 smithi169 ceph-mon[32759]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi040=y;smithi169=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:06:49.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:49 smithi169 ceph-mon[32759]: Saving service mgr spec with placement smithi031=x;smithi040=y;smithi169=z;count:3 2024-04-25T01:06:49.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:49 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:49.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:49 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:49.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:49 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:06:49.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:49 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:06:49.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:49 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:49.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:49 smithi040 ceph-mon[32518]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:49.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:49 smithi040 ceph-mon[32518]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi040=y;smithi169=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:06:49.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:49 smithi040 ceph-mon[32518]: Saving service mgr spec with placement smithi031=x;smithi040=y;smithi169=z;count:3 2024-04-25T01:06:49.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:49 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:49.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:49 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:49.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:49 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:06:49.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:49 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:06:49.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:49 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:49.965 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T01:06:49.965 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:06:49.965 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T01:06:49.965 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:06:49.965 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:06:49.965 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-25 01:06:36.034515525 +0000 2024-04-25T01:06:49.965 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-25 00:59:40.407736666 +0000 2024-04-25T01:06:49.965 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-25 00:59:40.407736666 +0000 2024-04-25T01:06:49.965 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-25 00:59:40.407736666 +0000 2024-04-25T01:06:49.965 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T01:06:50.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:49 smithi031 ceph-mon[28053]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:50.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:49 smithi031 ceph-mon[28053]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi040=y;smithi169=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:06:50.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:49 smithi031 ceph-mon[28053]: Saving service mgr spec with placement smithi031=x;smithi040=y;smithi169=z;count:3 2024-04-25T01:06:50.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:49 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:50.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:49 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:50.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:49 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:06:50.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:49 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:06:50.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:49 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:50.033 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-25T01:06:50.033 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-25T01:06:50.033 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000287562 s, 1.8 MB/s 2024-04-25T01:06:50.035 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T01:06:50.093 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_2 2024-04-25T01:06:50.152 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T01:06:50.152 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:06:50.152 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T01:06:50.152 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:06:50.152 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:06:50.152 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-25 01:06:36.035515514 +0000 2024-04-25T01:06:50.152 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-25 00:59:40.760732579 +0000 2024-04-25T01:06:50.152 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-25 00:59:40.760732579 +0000 2024-04-25T01:06:50.152 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-25 00:59:40.760732579 +0000 2024-04-25T01:06:50.153 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T01:06:50.220 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-25T01:06:50.220 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-25T01:06:50.221 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000198578 s, 2.6 MB/s 2024-04-25T01:06:50.222 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T01:06:50.281 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_3 2024-04-25T01:06:50.339 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T01:06:50.339 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:06:50.339 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T01:06:50.340 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:06:50.340 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:06:50.340 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-25 01:06:36.035515514 +0000 2024-04-25T01:06:50.340 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-25 00:59:41.109728538 +0000 2024-04-25T01:06:50.340 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-25 00:59:41.109728538 +0000 2024-04-25T01:06:50.340 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-25 00:59:41.109728538 +0000 2024-04-25T01:06:50.340 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T01:06:50.406 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-25T01:06:50.406 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-25T01:06:50.406 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.00018753 s, 2.7 MB/s 2024-04-25T01:06:50.408 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T01:06:50.466 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_4 2024-04-25T01:06:50.524 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T01:06:50.524 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:06:50.524 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T01:06:50.524 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:06:50.524 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:06:50.525 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-25 01:06:36.036515503 +0000 2024-04-25T01:06:50.525 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-25 00:59:41.446724636 +0000 2024-04-25T01:06:50.525 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-25 00:59:41.446724636 +0000 2024-04-25T01:06:50.525 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-25 00:59:41.446724636 +0000 2024-04-25T01:06:50.525 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T01:06:50.591 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-25T01:06:50.591 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-25T01:06:50.591 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000208965 s, 2.5 MB/s 2024-04-25T01:06:50.592 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T01:06:50.651 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T01:06:50.651 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T01:06:50.666 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T01:06:50.666 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_1 2024-04-25T01:06:50.721 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T01:06:50.721 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:06:50.721 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-25T01:06:50.721 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:06:50.721 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:06:50.721 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-25 01:06:26.434571322 +0000 2024-04-25T01:06:50.721 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-25 00:59:19.300848815 +0000 2024-04-25T01:06:50.721 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-25 00:59:19.300848815 +0000 2024-04-25T01:06:50.721 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-25 00:59:19.300848815 +0000 2024-04-25T01:06:50.722 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T01:06:50.783 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-25T01:06:50.783 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-25T01:06:50.783 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000197572 s, 2.6 MB/s 2024-04-25T01:06:50.784 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T01:06:50.840 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_2 2024-04-25T01:06:50.895 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T01:06:50.895 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:06:50.895 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T01:06:50.895 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:06:50.895 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:06:50.895 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-25 01:06:26.434571322 +0000 2024-04-25T01:06:50.896 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-25 00:59:19.646841500 +0000 2024-04-25T01:06:50.896 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-25 00:59:19.646841500 +0000 2024-04-25T01:06:50.896 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-25 00:59:19.646841500 +0000 2024-04-25T01:06:50.896 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T01:06:50.952 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:50 smithi169 ceph-mon[32759]: Reconfiguring mon.c (monmap changed)... 2024-04-25T01:06:50.952 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:50 smithi169 ceph-mon[32759]: Reconfiguring daemon mon.c on smithi169 2024-04-25T01:06:50.952 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:50 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:50.952 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:50 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:50.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:50 smithi040 ceph-mon[32518]: Reconfiguring mon.c (monmap changed)... 2024-04-25T01:06:50.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:50 smithi040 ceph-mon[32518]: Reconfiguring daemon mon.c on smithi169 2024-04-25T01:06:50.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:50 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:50.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:50 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:50.958 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-25T01:06:50.958 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-25T01:06:50.958 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000176087 s, 2.9 MB/s 2024-04-25T01:06:50.959 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T01:06:51.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:50 smithi031 ceph-mon[28053]: Reconfiguring mon.c (monmap changed)... 2024-04-25T01:06:51.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:50 smithi031 ceph-mon[28053]: Reconfiguring daemon mon.c on smithi169 2024-04-25T01:06:51.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:50 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:51.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:50 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:51.015 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_3 2024-04-25T01:06:51.070 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T01:06:51.070 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:06:51.070 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T01:06:51.070 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:06:51.070 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:06:51.070 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-25 01:06:26.435571302 +0000 2024-04-25T01:06:51.070 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-25 00:59:19.997834080 +0000 2024-04-25T01:06:51.070 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-25 00:59:19.997834080 +0000 2024-04-25T01:06:51.070 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-25 00:59:19.997834080 +0000 2024-04-25T01:06:51.071 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T01:06:51.132 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-25T01:06:51.132 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-25T01:06:51.132 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.00021433 s, 2.4 MB/s 2024-04-25T01:06:51.134 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T01:06:51.190 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_4 2024-04-25T01:06:51.245 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T01:06:51.245 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:06:51.246 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T01:06:51.246 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:06:51.246 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:06:51.246 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-25 01:06:26.435571302 +0000 2024-04-25T01:06:51.246 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-25 00:59:20.347826681 +0000 2024-04-25T01:06:51.246 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-25 00:59:20.347826681 +0000 2024-04-25T01:06:51.246 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-25 00:59:20.347826681 +0000 2024-04-25T01:06:51.246 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T01:06:51.308 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-25T01:06:51.308 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-25T01:06:51.308 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.00024381 s, 2.1 MB/s 2024-04-25T01:06:51.309 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T01:06:51.365 INFO:tasks.cephadm:Deploying osd.0 on smithi031 with /dev/vg_nvme/lv_4... 2024-04-25T01:06:51.365 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T01:06:51.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:51 smithi031 ceph-mon[28053]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:51.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:51 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:06:51.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:51 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:51.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:51 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:06:51.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:51 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:51.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:51 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:06:51.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:51 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T01:06:51.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:51 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:06:51.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:51 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:51.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:51 smithi031 ceph-mon[28053]: Deploying daemon mgr.z on smithi169 2024-04-25T01:06:51.815 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:06:51.910 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:51 smithi169 ceph-mon[32759]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:51.910 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:51 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:06:51.910 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:51 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:51.910 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:51 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:06:51.910 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:51 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:51.910 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:51 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:06:51.910 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:51 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T01:06:51.910 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:51 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:06:51.910 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:51 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:51.910 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:51 smithi169 ceph-mon[32759]: Deploying daemon mgr.z on smithi169 2024-04-25T01:06:51.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:51 smithi040 ceph-mon[32518]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:51.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:51 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:06:51.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:51 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:51.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:51 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:06:51.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:51 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:51.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:51 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:06:51.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:51 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T01:06:51.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:51 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:06:51.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:51 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:51.956 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:51 smithi040 ceph-mon[32518]: Deploying daemon mgr.z on smithi169 2024-04-25T01:06:52.846 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:52 smithi169 systemd[1]: Starting Ceph mgr.z for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:06:53.123 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:52 smithi169 podman[34611]: 2024-04-25 01:06:52.852975941 +0000 UTC m=+0.099051530 container create 4a052505ea91141e28ded6f9007d8529579c45606fe1028a0f3f2367c6699892 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:06:53.123 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:52 smithi169 podman[34611]: 2024-04-25 01:06:52.772113492 +0000 UTC m=+0.018189095 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:06:53.123 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:53 smithi169 podman[34611]: 2024-04-25 01:06:53.047104763 +0000 UTC m=+0.293180354 container init 4a052505ea91141e28ded6f9007d8529579c45606fe1028a0f3f2367c6699892 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:06:53.123 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:53 smithi169 podman[34611]: 2024-04-25 01:06:53.049750554 +0000 UTC m=+0.295826147 container start 4a052505ea91141e28ded6f9007d8529579c45606fe1028a0f3f2367c6699892 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T01:06:53.123 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:53 smithi169 ceph-mgr[34630]: -- 172.21.15.169:0/1839912572 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5569ab6aa5a0 con 0x5569ab67e800 2024-04-25T01:06:53.124 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:53 smithi169 bash[34611]: 4a052505ea91141e28ded6f9007d8529579c45606fe1028a0f3f2367c6699892 2024-04-25T01:06:53.391 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:53 smithi169 systemd[1]: Started Ceph mgr.z for 95849b94-029f-11ef-bc93-c7b262605968. 2024-04-25T01:06:53.391 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:53 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z[34626]: 2024-04-25T01:06:53.211+0000 7f3fecdb2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T01:06:53.391 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:53 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z[34626]: 2024-04-25T01:06:53.397+0000 7f3fecdb2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:06:53.660 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:53 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z[34626]: 2024-04-25T01:06:53.498+0000 7f3fecdb2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T01:06:53.952 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:53 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z[34626]: 2024-04-25T01:06:53.806+0000 7f3fecdb2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T01:06:53.952 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:53 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z[34626]: 2024-04-25T01:06:53.890+0000 7f3fecdb2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T01:06:53.952 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:53 smithi169 ceph-mon[32759]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:53.952 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:53 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:53.952 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:53 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:53.952 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:53 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:53.953 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:53 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:06:53.953 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:53 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T01:06:53.953 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:53 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:06:53.953 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:53 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:53.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:53 smithi040 ceph-mon[32518]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:53.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:53 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:53.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:53 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:53.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:53 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:53.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:53 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:06:53.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:53 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T01:06:53.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:53 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:06:53.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:53 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:53.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:53 smithi031 ceph-mon[28053]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:53.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:53 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:53.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:53 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:53.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:53 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:53.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:53 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:06:53.971 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:53 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T01:06:53.971 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:53 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:06:53.971 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:53 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:54.913 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T01:06:54.913 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T01:06:54.913 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2024-04-25T01:06:54.913 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2024-04-25T01:06:54.914 INFO:teuthology.orchestra.run.smithi031.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0359269 s, 292 MB/s 2024-04-25T01:06:54.914 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2024-04-25T01:06:54.952 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:54 smithi169 ceph-mon[32759]: Deploying daemon mgr.y on smithi040 2024-04-25T01:06:54.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:54 smithi040 ceph-mon[32518]: Deploying daemon mgr.y on smithi040 2024-04-25T01:06:54.959 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:54 smithi031 ceph-mon[28053]: Deploying daemon mgr.y on smithi040 2024-04-25T01:06:55.424 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:55 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z[34626]: 2024-04-25T01:06:55.091+0000 7f3fecdb2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T01:06:55.425 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:55 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z[34626]: 2024-04-25T01:06:55.259+0000 7f3fecdb2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T01:06:55.425 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:55 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z[34626]: 2024-04-25T01:06:55.430+0000 7f3fecdb2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T01:06:55.455 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:06:55 smithi040 systemd[1]: Starting Ceph mgr.y for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:06:55.685 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_4 2024-04-25T01:06:55.702 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:55 smithi169 ceph-mon[32759]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:55.744 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:55 smithi040 ceph-mon[32518]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:55.744 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:06:55 smithi040 podman[33906]: 2024-04-25 01:06:55.496360527 +0000 UTC m=+0.114284828 container create e8346f709fab7619eb6505e9debb4390d9d6eeddab0d5ac2fbf865bcce720251 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:06:55.744 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:06:55 smithi040 podman[33906]: 2024-04-25 01:06:55.399965672 +0000 UTC m=+0.017889987 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:06:55.744 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:06:55 smithi040 podman[33906]: 2024-04-25 01:06:55.723683486 +0000 UTC m=+0.341607789 container init e8346f709fab7619eb6505e9debb4390d9d6eeddab0d5ac2fbf865bcce720251 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T01:06:55.744 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:06:55 smithi040 podman[33906]: 2024-04-25 01:06:55.726469827 +0000 UTC m=+0.344394139 container start e8346f709fab7619eb6505e9debb4390d9d6eeddab0d5ac2fbf865bcce720251 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T01:06:55.745 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:06:55 smithi040 ceph-mgr[33926]: -- 172.21.15.40:0/1705179227 <== mon.2 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5595ed2145a0 con 0x5595ed1e8800 2024-04-25T01:06:56.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:55 smithi031 ceph-mon[28053]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:56.013 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:55 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z[34626]: 2024-04-25T01:06:55.717+0000 7f3fecdb2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T01:06:56.013 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:55 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z[34626]: 2024-04-25T01:06:55.809+0000 7f3fecdb2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T01:06:56.013 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:56 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z[34626]: 2024-04-25T01:06:56.018+0000 7f3fecdb2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T01:06:56.087 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:06:55 smithi040 bash[33906]: e8346f709fab7619eb6505e9debb4390d9d6eeddab0d5ac2fbf865bcce720251 2024-04-25T01:06:56.087 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:06:55 smithi040 systemd[1]: Started Ceph mgr.y for 95849b94-029f-11ef-bc93-c7b262605968. 2024-04-25T01:06:56.087 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:06:55 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y[33922]: 2024-04-25T01:06:55.902+0000 7f17aee28200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T01:06:56.088 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:06:56 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y[33922]: 2024-04-25T01:06:56.093+0000 7f17aee28200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:06:56.133 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:06:56.452 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:56 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z[34626]: 2024-04-25T01:06:56.215+0000 7f3fecdb2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:06:56.455 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:06:56 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y[33922]: 2024-04-25T01:06:56.199+0000 7f17aee28200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T01:06:56.872 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:06:56 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y[33922]: 2024-04-25T01:06:56.529+0000 7f17aee28200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T01:06:56.873 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:06:56 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y[33922]: 2024-04-25T01:06:56.618+0000 7f17aee28200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T01:06:57.138 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:56 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:57.138 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:56 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:57.138 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:56 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:57.138 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:56 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:57.138 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:56 smithi040 ceph-mon[32518]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:57.138 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:56 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:06:57.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:56 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:57.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:56 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:57.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:56 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:57.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:56 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:57.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:56 smithi169 ceph-mon[32759]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:57.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:56 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:06:57.203 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:56 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z[34626]: 2024-04-25T01:06:56.852+0000 7f3fecdb2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T01:06:57.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:56 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:57.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:56 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:57.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:56 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:57.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:56 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:57.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:56 smithi031 ceph-mon[28053]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:57.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:56 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:06:57.590 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:57 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z[34626]: 2024-04-25T01:06:57.288+0000 7f3fecdb2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T01:06:57.590 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:57 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z[34626]: 2024-04-25T01:06:57.390+0000 7f3fecdb2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T01:06:57.590 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:57 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z[34626]: 2024-04-25T01:06:57.595+0000 7f3fecdb2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T01:06:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.145+0000 7f7ed8600700 1 -- 172.21.15.31:0/513951548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ed00782c0 msgr2=0x7f7ed00786e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.145+0000 7f7ed8600700 1 --2- 172.21.15.31:0/513951548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ed00782c0 0x7f7ed00786e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4009990 tx=0x7f7ec4009ca0 comp rx=0 tx=0).stop 2024-04-25T01:06:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.145+0000 7f7ed8600700 1 -- 172.21.15.31:0/513951548 shutdown_connections 2024-04-25T01:06:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.145+0000 7f7ed8600700 1 --2- 172.21.15.31:0/513951548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ed0077210 0x7f7ed0079070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.145+0000 7f7ed8600700 1 --2- 172.21.15.31:0/513951548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ed00782c0 0x7f7ed00786e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.145+0000 7f7ed8600700 1 --2- 172.21.15.31:0/513951548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ed0076850 0x7f7ed0076cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:58.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.145+0000 7f7ed8600700 1 -- 172.21.15.31:0/513951548 >> 172.21.15.31:0/513951548 conn(0x7f7ed00ffa10 msgr2=0x7f7ed0101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:06:58.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.145+0000 7f7ed8600700 1 -- 172.21.15.31:0/513951548 shutdown_connections 2024-04-25T01:06:58.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.145+0000 7f7ed8600700 1 -- 172.21.15.31:0/513951548 wait complete. 2024-04-25T01:06:58.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.147+0000 7f7ed8600700 1 Processor -- start 2024-04-25T01:06:58.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.147+0000 7f7ed8600700 1 -- start start 2024-04-25T01:06:58.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.147+0000 7f7ed8600700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ed0076850 0x7f7ed019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:58.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.147+0000 7f7ed8600700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ed0077210 0x7f7ed019f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.147+0000 7f7ed8600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ed00782c0 0x7f7ed01a61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.147+0000 7f7ed8600700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed01145e0 con 0x7f7ed0076850 2024-04-25T01:06:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.147+0000 7f7ed8600700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed0114460 con 0x7f7ed0077210 2024-04-25T01:06:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.147+0000 7f7ed8600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed0114760 con 0x7f7ed00782c0 2024-04-25T01:06:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.147+0000 7f7ed5200700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ed0077210 0x7f7ed019f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.148+0000 7f7ed5c00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ed0076850 0x7f7ed019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.148+0000 7f7ed6600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ed00782c0 0x7f7ed01a61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.148+0000 7f7ed6600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ed00782c0 0x7f7ed01a61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:58012/0 (socket says 172.21.15.31:58012) 2024-04-25T01:06:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.148+0000 7f7ed6600700 1 -- 172.21.15.31:0/3238331140 learned_addr learned my addr 172.21.15.31:0/3238331140 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:06:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.148+0000 7f7ed5200700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ed0077210 0x7f7ed019f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:58760/0 (socket says 172.21.15.31:58760) 2024-04-25T01:06:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.148+0000 7f7ed5c00700 1 -- 172.21.15.31:0/3238331140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ed00782c0 msgr2=0x7f7ed01a61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.148+0000 7f7ed5c00700 1 --2- 172.21.15.31:0/3238331140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ed00782c0 0x7f7ed01a61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.148+0000 7f7ed5c00700 1 -- 172.21.15.31:0/3238331140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ed0077210 msgr2=0x7f7ed019f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:06:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.148+0000 7f7ed5c00700 1 --2- 172.21.15.31:0/3238331140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ed0077210 0x7f7ed019f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:06:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.148+0000 7f7ed5c00700 1 -- 172.21.15.31:0/3238331140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed01a6840 con 0x7f7ed0076850 2024-04-25T01:06:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.148+0000 7f7ed5c00700 1 --2- 172.21.15.31:0/3238331140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ed0076850 0x7f7ed019ebe0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7ec40098e0 tx=0x7f7ec400bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:06:58.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.149+0000 7f7ece000700 1 -- 172.21.15.31:0/3238331140 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4026070 con 0x7f7ed0076850 2024-04-25T01:06:58.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.149+0000 7f7ece000700 1 -- 172.21.15.31:0/3238331140 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ec400f460 con 0x7f7ed0076850 2024-04-25T01:06:58.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.149+0000 7f7ed8600700 1 -- 172.21.15.31:0/3238331140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed01a6ad0 con 0x7f7ed0076850 2024-04-25T01:06:58.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.149+0000 7f7ece000700 1 -- 172.21.15.31:0/3238331140 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4021670 con 0x7f7ed0076850 2024-04-25T01:06:58.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.149+0000 7f7ed8600700 1 -- 172.21.15.31:0/3238331140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed01a6f80 con 0x7f7ed0076850 2024-04-25T01:06:58.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.150+0000 7f7ece000700 1 -- 172.21.15.31:0/3238331140 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7ec4029070 con 0x7f7ed0076850 2024-04-25T01:06:58.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.150+0000 7f7ed8600700 1 -- 172.21.15.31:0/3238331140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed0068f20 con 0x7f7ed0076850 2024-04-25T01:06:58.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.151+0000 7f7ece000700 1 --2- 172.21.15.31:0/3238331140 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7eb8039330 0x7f7eb803b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:06:58.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.151+0000 7f7ece000700 1 -- 172.21.15.31:0/3238331140 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7ec404d0a0 con 0x7f7ed0076850 2024-04-25T01:06:58.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.151+0000 7f7ed5200700 1 --2- 172.21.15.31:0/3238331140 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7eb8039330 0x7f7eb803b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:06:58.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.152+0000 7f7ed5200700 1 --2- 172.21.15.31:0/3238331140 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7eb8039330 0x7f7eb803b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009c60 tx=0x7f7ebc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:06:58.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.157+0000 7f7ece000700 1 -- 172.21.15.31:0/3238331140 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7ec4013070 con 0x7f7ed0076850 2024-04-25T01:06:58.205 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:06:57 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y[33922]: 2024-04-25T01:06:57.905+0000 7f17aee28200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T01:06:58.205 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:06:58 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y[33922]: 2024-04-25T01:06:58.080+0000 7f17aee28200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T01:06:58.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:58 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:58.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:58 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:58.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:58 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:58.206 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:58 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:58.206 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:58 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:58.206 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:58 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:58.206 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:58 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:06:58.206 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:58 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:58.206 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:58 smithi040 ceph-mon[32518]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T01:06:58.206 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:58 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:06:58.206 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:58 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:06:58.206 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:58 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:58.206 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:58 smithi040 ceph-mon[32518]: Reconfiguring daemon mgr.x on smithi031 2024-04-25T01:06:58.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:58 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:58.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:58 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:58.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:58 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:58.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:58 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:58.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:58 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:58.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:58 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:58.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:58 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:06:58.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:58 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:58.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:58 smithi031 ceph-mon[28053]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T01:06:58.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:58 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:06:58.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:58 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:06:58.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:58 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:58.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:58 smithi031 ceph-mon[28053]: Reconfiguring daemon mgr.x on smithi031 2024-04-25T01:06:58.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:06:58.400+0000 7f7ed8600700 1 -- 172.21.15.31:0/3238331140 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7ed0062670 con 0x7f7eb8039330 2024-04-25T01:06:58.452 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:58 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z[34626]: 2024-04-25T01:06:58.125+0000 7f3fecdb2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T01:06:58.452 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:58 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z[34626]: 2024-04-25T01:06:58.217+0000 7f3fecdb2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T01:06:58.453 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:58 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:58.453 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:58 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:58.453 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:58 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:58.453 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:58 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:58.453 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:58 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:58.453 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:58 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:58.453 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:58 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:06:58.453 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:58 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:06:58.453 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:58 smithi169 ceph-mon[32759]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T01:06:58.453 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:58 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:06:58.453 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:58 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:06:58.453 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:58 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:58.453 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:58 smithi169 ceph-mon[32759]: Reconfiguring daemon mgr.x on smithi031 2024-04-25T01:06:58.545 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:06:58 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y[33922]: 2024-04-25T01:06:58.255+0000 7f17aee28200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T01:06:58.545 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:06:58 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y[33922]: 2024-04-25T01:06:58.551+0000 7f17aee28200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T01:06:58.796 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:58 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z[34626]: 2024-04-25T01:06:58.459+0000 7f3fecdb2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T01:06:58.797 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:58 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z[34626]: 2024-04-25T01:06:58.593+0000 7f3fecdb2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T01:06:58.797 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:58 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z[34626]: 2024-04-25T01:06:58.687+0000 7f3fecdb2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T01:06:58.797 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:58 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z[34626]: 2024-04-25T01:06:58.802+0000 7f3fecdb2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T01:06:58.849 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:06:58 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y[33922]: 2024-04-25T01:06:58.646+0000 7f17aee28200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T01:06:58.849 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:06:58 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y[33922]: 2024-04-25T01:06:58.855+0000 7f17aee28200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T01:06:59.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:59 smithi169 ceph-mon[32759]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:59.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:59 smithi169 ceph-mon[32759]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:06:59.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:59 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:06:59.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:59 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:06:59.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:06:59 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:59.202 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:58 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z[34626]: 2024-04-25T01:06:58.893+0000 7f3fecdb2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T01:06:59.205 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:06:59 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y[33922]: 2024-04-25T01:06:59.052+0000 7f17aee28200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:06:59.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:59 smithi040 ceph-mon[32518]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:59.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:59 smithi040 ceph-mon[32518]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:06:59.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:59 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:06:59.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:59 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:06:59.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:06:59 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:59.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:59 smithi031 ceph-mon[28053]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:06:59.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:59 smithi031 ceph-mon[28053]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:06:59.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:59 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:06:59.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:59 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:06:59.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:06:59 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:06:59.955 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:06:59 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y[33922]: 2024-04-25T01:06:59.674+0000 7f17aee28200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T01:07:00.202 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:59 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z[34626]: 2024-04-25T01:06:59.711+0000 7f3fecdb2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T01:07:00.202 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:06:59 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z[34626]: 2024-04-25T01:06:59.925+0000 7f3fecdb2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T01:07:00.406 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:07:00 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y[33922]: 2024-04-25T01:07:00.109+0000 7f17aee28200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T01:07:00.406 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:07:00 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y[33922]: 2024-04-25T01:07:00.208+0000 7f17aee28200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T01:07:00.406 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:07:00 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y[33922]: 2024-04-25T01:07:00.412+0000 7f17aee28200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T01:07:00.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:07:00.937+0000 7f7ece000700 1 -- 172.21.15.31:0/3238331140 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f7ec4009d10 con 0x7f7ed0076850 2024-04-25T01:07:01.120 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:00 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:01.120 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:00 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:01.120 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:00 smithi031 ceph-mon[28053]: Standby manager daemon z started 2024-04-25T01:07:01.120 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:00 smithi031 ceph-mon[28053]: from='mgr.? 172.21.15.169:0/2621431256' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T01:07:01.120 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:00 smithi031 ceph-mon[28053]: from='mgr.? 172.21.15.169:0/2621431256' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T01:07:01.120 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:00 smithi031 ceph-mon[28053]: from='mgr.? 172.21.15.169:0/2621431256' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T01:07:01.120 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:00 smithi031 ceph-mon[28053]: from='mgr.? 172.21.15.169:0/2621431256' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T01:07:01.120 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:00 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:07:01.120 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:00 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:01.120 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:00 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:07:01.120 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:00 smithi031 ceph-mon[28053]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:01.121 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:00 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:01.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:00 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:01.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:00 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:01.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:00 smithi169 ceph-mon[32759]: Standby manager daemon z started 2024-04-25T01:07:01.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:00 smithi169 ceph-mon[32759]: from='mgr.? 172.21.15.169:0/2621431256' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T01:07:01.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:00 smithi169 ceph-mon[32759]: from='mgr.? 172.21.15.169:0/2621431256' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T01:07:01.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:00 smithi169 ceph-mon[32759]: from='mgr.? 172.21.15.169:0/2621431256' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T01:07:01.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:00 smithi169 ceph-mon[32759]: from='mgr.? 172.21.15.169:0/2621431256' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T01:07:01.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:00 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:07:01.203 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:00 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:01.203 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:00 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:07:01.203 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:00 smithi169 ceph-mon[32759]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:01.203 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:00 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:01.205 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:07:00 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y[33922]: 2024-04-25T01:07:00.955+0000 7f17aee28200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T01:07:01.205 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:07:01 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y[33922]: 2024-04-25T01:07:01.048+0000 7f17aee28200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T01:07:01.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:00 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:01.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:00 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:01.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:00 smithi040 ceph-mon[32518]: Standby manager daemon z started 2024-04-25T01:07:01.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:00 smithi040 ceph-mon[32518]: from='mgr.? 172.21.15.169:0/2621431256' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T01:07:01.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:00 smithi040 ceph-mon[32518]: from='mgr.? 172.21.15.169:0/2621431256' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T01:07:01.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:00 smithi040 ceph-mon[32518]: from='mgr.? 172.21.15.169:0/2621431256' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T01:07:01.206 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:00 smithi040 ceph-mon[32518]: from='mgr.? 172.21.15.169:0/2621431256' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T01:07:01.206 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:00 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:07:01.206 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:00 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:01.206 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:00 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:07:01.206 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:00 smithi040 ceph-mon[32518]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:01.206 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:00 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:01.641 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:07:01 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y[33922]: 2024-04-25T01:07:01.296+0000 7f17aee28200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T01:07:01.641 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:07:01 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y[33922]: 2024-04-25T01:07:01.432+0000 7f17aee28200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T01:07:01.641 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:07:01 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y[33922]: 2024-04-25T01:07:01.529+0000 7f17aee28200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T01:07:01.641 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:07:01 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y[33922]: 2024-04-25T01:07:01.647+0000 7f17aee28200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T01:07:01.924 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:07:01 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y[33922]: 2024-04-25T01:07:01.741+0000 7f17aee28200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T01:07:02.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:01 smithi031 ceph-mon[28053]: mgrmap e14: x(active, since 89s), standbys: z 2024-04-25T01:07:02.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:01 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T01:07:02.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:01 smithi169 ceph-mon[32759]: mgrmap e14: x(active, since 89s), standbys: z 2024-04-25T01:07:02.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:01 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T01:07:02.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:01 smithi040 ceph-mon[32518]: mgrmap e14: x(active, since 89s), standbys: z 2024-04-25T01:07:02.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:01 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T01:07:02.927 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:07:02 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y[33922]: 2024-04-25T01:07:02.577+0000 7f17aee28200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T01:07:02.927 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:07:02 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y[33922]: 2024-04-25T01:07:02.797+0000 7f17aee28200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T01:07:02.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:07:02.956+0000 7f7ece000700 1 -- 172.21.15.31:0/3238331140 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f7ec40163a0 con 0x7f7ed0076850 2024-04-25T01:07:03.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:02 smithi169 ceph-mon[32759]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:03.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:02 smithi169 ceph-mon[32759]: from='mgr.? 172.21.15.40:0/1945521753' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T01:07:03.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:02 smithi169 ceph-mon[32759]: Standby manager daemon y started 2024-04-25T01:07:03.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:02 smithi169 ceph-mon[32759]: from='mgr.? 172.21.15.40:0/1945521753' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T01:07:03.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:02 smithi169 ceph-mon[32759]: from='mgr.? 172.21.15.40:0/1945521753' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T01:07:03.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:02 smithi169 ceph-mon[32759]: from='mgr.? 172.21.15.40:0/1945521753' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T01:07:03.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:02 smithi040 ceph-mon[32518]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:03.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:02 smithi040 ceph-mon[32518]: from='mgr.? 172.21.15.40:0/1945521753' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T01:07:03.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:02 smithi040 ceph-mon[32518]: Standby manager daemon y started 2024-04-25T01:07:03.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:02 smithi040 ceph-mon[32518]: from='mgr.? 172.21.15.40:0/1945521753' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T01:07:03.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:02 smithi040 ceph-mon[32518]: from='mgr.? 172.21.15.40:0/1945521753' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T01:07:03.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:02 smithi040 ceph-mon[32518]: from='mgr.? 172.21.15.40:0/1945521753' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T01:07:03.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:02 smithi031 ceph-mon[28053]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:03.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:02 smithi031 ceph-mon[28053]: from='mgr.? 172.21.15.40:0/1945521753' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T01:07:03.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:02 smithi031 ceph-mon[28053]: Standby manager daemon y started 2024-04-25T01:07:03.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:02 smithi031 ceph-mon[28053]: from='mgr.? 172.21.15.40:0/1945521753' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T01:07:03.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:02 smithi031 ceph-mon[28053]: from='mgr.? 172.21.15.40:0/1945521753' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T01:07:03.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:02 smithi031 ceph-mon[28053]: from='mgr.? 172.21.15.40:0/1945521753' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T01:07:04.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:03 smithi169 ceph-mon[32759]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-04-25T01:07:04.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:03 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T01:07:04.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:03 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3961909922' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caa08e4c-fe6b-4116-bdf5-1adf5b465d43"}]: dispatch 2024-04-25T01:07:04.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:03 smithi169 ceph-mon[32759]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caa08e4c-fe6b-4116-bdf5-1adf5b465d43"}]: dispatch 2024-04-25T01:07:04.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:03 smithi169 ceph-mon[32759]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "caa08e4c-fe6b-4116-bdf5-1adf5b465d43"}]': finished 2024-04-25T01:07:04.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:03 smithi169 ceph-mon[32759]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T01:07:04.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:03 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:07:04.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:03 smithi040 ceph-mon[32518]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-04-25T01:07:04.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:03 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T01:07:04.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:03 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3961909922' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caa08e4c-fe6b-4116-bdf5-1adf5b465d43"}]: dispatch 2024-04-25T01:07:04.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:03 smithi040 ceph-mon[32518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caa08e4c-fe6b-4116-bdf5-1adf5b465d43"}]: dispatch 2024-04-25T01:07:04.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:03 smithi040 ceph-mon[32518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "caa08e4c-fe6b-4116-bdf5-1adf5b465d43"}]': finished 2024-04-25T01:07:04.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:03 smithi040 ceph-mon[32518]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T01:07:04.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:03 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:07:04.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:03 smithi031 ceph-mon[28053]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-04-25T01:07:04.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:03 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T01:07:04.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:03 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3961909922' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caa08e4c-fe6b-4116-bdf5-1adf5b465d43"}]: dispatch 2024-04-25T01:07:04.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:03 smithi031 ceph-mon[28053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caa08e4c-fe6b-4116-bdf5-1adf5b465d43"}]: dispatch 2024-04-25T01:07:04.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:03 smithi031 ceph-mon[28053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "caa08e4c-fe6b-4116-bdf5-1adf5b465d43"}]': finished 2024-04-25T01:07:04.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:03 smithi031 ceph-mon[28053]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T01:07:04.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:03 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:07:05.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:04 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1668345038' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:07:05.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:04 smithi169 ceph-mon[32759]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:05.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:04 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1668345038' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:07:05.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:04 smithi040 ceph-mon[32518]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:05.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:04 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1668345038' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:07:05.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:04 smithi031 ceph-mon[28053]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:07.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:07 smithi031 ceph-mon[28053]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:07.453 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:07 smithi169 ceph-mon[32759]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:07.455 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:07 smithi040 ceph-mon[32518]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:09.399 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:09 smithi031 ceph-mon[28053]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:09.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:09 smithi169 ceph-mon[32759]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:09.455 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:09 smithi040 ceph-mon[32518]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:11.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:11 smithi031 ceph-mon[28053]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:11.452 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:11 smithi169 ceph-mon[32759]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:11.454 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:11 smithi040 ceph-mon[32518]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:13.451 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:13 smithi031 ceph-mon[28053]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:13.451 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T01:07:13.451 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:13.452 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:13 smithi031 ceph-mon[28053]: Deploying daemon osd.0 on smithi031 2024-04-25T01:07:13.452 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:13 smithi169 ceph-mon[32759]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:13.452 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T01:07:13.452 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:13.452 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:13 smithi169 ceph-mon[32759]: Deploying daemon osd.0 on smithi031 2024-04-25T01:07:13.454 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:13 smithi040 ceph-mon[32518]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:13.454 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T01:07:13.455 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:13.455 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:13 smithi040 ceph-mon[32518]: Deploying daemon osd.0 on smithi031 2024-04-25T01:07:15.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:15 smithi031 ceph-mon[28053]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:15.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:15 smithi169 ceph-mon[32759]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:15.454 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:15 smithi040 ceph-mon[32518]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:17.420 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:17 smithi031 ceph-mon[28053]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:17.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:17 smithi169 ceph-mon[32759]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:17.454 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:17 smithi040 ceph-mon[32518]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:19.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:19 smithi031 ceph-mon[28053]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:19.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:19 smithi169 ceph-mon[32759]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:19.454 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:19 smithi040 ceph-mon[32518]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:21.798 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:21 smithi031 ceph-mon[28053]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:21.798 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:21 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:21.798 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:21 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:07:21.798 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:21 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:21.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:21 smithi169 ceph-mon[32759]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:21.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:21 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:21.952 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:21 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:07:21.952 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:21 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:21.954 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:21 smithi040 ceph-mon[32518]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:21.954 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:21 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:21.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:21 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:07:21.955 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:21 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:23.707 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:23 smithi031 ceph-mon[28053]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:23.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:23 smithi169 ceph-mon[32759]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:23.954 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:23 smithi040 ceph-mon[32518]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:25.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:24 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:25.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:24 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:25.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:24 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:25.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:24 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:07:25.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:24 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:25.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:24 smithi169 ceph-mon[32759]: from='osd.0 [v2:172.21.15.31:6802/788174091,v1:172.21.15.31:6803/788174091]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T01:07:25.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:24 smithi169 ceph-mon[32759]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T01:07:25.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:24 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:25.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:24 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:25.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:24 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:25.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:24 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:07:25.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:24 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:25.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:24 smithi040 ceph-mon[32518]: from='osd.0 [v2:172.21.15.31:6802/788174091,v1:172.21.15.31:6803/788174091]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T01:07:25.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:24 smithi040 ceph-mon[32518]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T01:07:25.217 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:24 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:25.217 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:24 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:25.217 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:24 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:25.217 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:24 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:07:25.217 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:24 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:25.217 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:24 smithi031 ceph-mon[28053]: from='osd.0 [v2:172.21.15.31:6802/788174091,v1:172.21.15.31:6803/788174091]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T01:07:25.217 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:24 smithi031 ceph-mon[28053]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T01:07:25.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:07:25.350+0000 7f7ece000700 1 -- 172.21.15.31:0/3238331140 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7ed0062670 con 0x7f7eb8039330 2024-04-25T01:07:25.344 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 0 on host 'smithi031' 2024-04-25T01:07:25.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:07:25.354+0000 7f7ed8600700 1 -- 172.21.15.31:0/3238331140 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7eb8039330 msgr2=0x7f7eb803b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:07:25.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:07:25.354+0000 7f7ed8600700 1 --2- 172.21.15.31:0/3238331140 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7eb8039330 0x7f7eb803b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009c60 tx=0x7f7ebc009260 comp rx=0 tx=0).stop 2024-04-25T01:07:25.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:07:25.354+0000 7f7ed8600700 1 -- 172.21.15.31:0/3238331140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ed0076850 msgr2=0x7f7ed019ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:07:25.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:07:25.354+0000 7f7ed8600700 1 --2- 172.21.15.31:0/3238331140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ed0076850 0x7f7ed019ebe0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7ec40098e0 tx=0x7f7ec400bbb0 comp rx=0 tx=0).stop 2024-04-25T01:07:25.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:07:25.355+0000 7f7ed8600700 1 -- 172.21.15.31:0/3238331140 shutdown_connections 2024-04-25T01:07:25.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:07:25.355+0000 7f7ed8600700 1 --2- 172.21.15.31:0/3238331140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ed0076850 0x7f7ed019ebe0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:07:25.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:07:25.355+0000 7f7ed8600700 1 --2- 172.21.15.31:0/3238331140 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7eb8039330 0x7f7eb803b7f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:07:25.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:07:25.355+0000 7f7ed8600700 1 --2- 172.21.15.31:0/3238331140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ed00782c0 0x7f7ed01a61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:07:25.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:07:25.355+0000 7f7ed8600700 1 --2- 172.21.15.31:0/3238331140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ed0077210 0x7f7ed019f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:07:25.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:07:25.355+0000 7f7ed8600700 1 -- 172.21.15.31:0/3238331140 >> 172.21.15.31:0/3238331140 conn(0x7f7ed00ffa10 msgr2=0x7f7ed0102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:07:25.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:07:25.355+0000 7f7ed8600700 1 -- 172.21.15.31:0/3238331140 shutdown_connections 2024-04-25T01:07:25.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:07:25.355+0000 7f7ed8600700 1 -- 172.21.15.31:0/3238331140 wait complete. 2024-04-25T01:07:26.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:25 smithi031 ceph-mon[28053]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:26.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:25 smithi031 ceph-mon[28053]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T01:07:26.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:25 smithi031 ceph-mon[28053]: from='osd.0 [v2:172.21.15.31:6802/788174091,v1:172.21.15.31:6803/788174091]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T01:07:26.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:25 smithi031 ceph-mon[28053]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T01:07:26.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:07:26.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:25 smithi031 ceph-mon[28053]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T01:07:26.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:26.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:26.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:07:26.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:25 smithi169 ceph-mon[32759]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:26.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:25 smithi169 ceph-mon[32759]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T01:07:26.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:25 smithi169 ceph-mon[32759]: from='osd.0 [v2:172.21.15.31:6802/788174091,v1:172.21.15.31:6803/788174091]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T01:07:26.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:25 smithi169 ceph-mon[32759]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T01:07:26.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:07:26.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:25 smithi169 ceph-mon[32759]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T01:07:26.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:26.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:26.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:07:26.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:25 smithi040 ceph-mon[32518]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:26.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:25 smithi040 ceph-mon[32518]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T01:07:26.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:25 smithi040 ceph-mon[32518]: from='osd.0 [v2:172.21.15.31:6802/788174091,v1:172.21.15.31:6803/788174091]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T01:07:26.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:25 smithi040 ceph-mon[32518]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T01:07:26.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:07:26.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:25 smithi040 ceph-mon[32518]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T01:07:26.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:26.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:26.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:07:26.236 DEBUG:teuthology.orchestra.run.smithi031:osd.0> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T01:07:26.241 INFO:tasks.cephadm:Deploying osd.1 on smithi040 with /dev/vg_nvme/lv_4... 2024-04-25T01:07:26.241 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T01:07:26.508 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T01:07:26.906 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:26 smithi040 ceph-mon[32518]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T01:07:26.907 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:26 smithi040 ceph-mon[32518]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T01:07:26.907 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:26 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:07:26.907 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:26 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:07:27.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:26 smithi169 ceph-mon[32759]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T01:07:27.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:26 smithi169 ceph-mon[32759]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T01:07:27.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:26 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:07:27.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:26 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:07:27.207 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:26 smithi031 ceph-mon[28053]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T01:07:27.207 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:26 smithi031 ceph-mon[28053]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T01:07:27.207 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:26 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:07:27.207 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:26 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:07:27.929 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:27 smithi040 ceph-mon[32518]: purged_snaps scrub starts 2024-04-25T01:07:27.929 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:27 smithi040 ceph-mon[32518]: purged_snaps scrub ok 2024-04-25T01:07:27.929 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:27 smithi040 ceph-mon[32518]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:27.929 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:27 smithi040 ceph-mon[32518]: osd.0 [v2:172.21.15.31:6802/788174091,v1:172.21.15.31:6803/788174091] boot 2024-04-25T01:07:27.929 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:27 smithi040 ceph-mon[32518]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T01:07:27.929 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:27 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:07:28.146 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:27 smithi031 ceph-mon[28053]: purged_snaps scrub starts 2024-04-25T01:07:28.147 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:27 smithi031 ceph-mon[28053]: purged_snaps scrub ok 2024-04-25T01:07:28.147 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:27 smithi031 ceph-mon[28053]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:28.147 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:27 smithi031 ceph-mon[28053]: osd.0 [v2:172.21.15.31:6802/788174091,v1:172.21.15.31:6803/788174091] boot 2024-04-25T01:07:28.147 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:27 smithi031 ceph-mon[28053]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T01:07:28.147 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:27 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:07:28.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:27 smithi169 ceph-mon[32759]: purged_snaps scrub starts 2024-04-25T01:07:28.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:27 smithi169 ceph-mon[32759]: purged_snaps scrub ok 2024-04-25T01:07:28.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:27 smithi169 ceph-mon[32759]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:07:28.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:27 smithi169 ceph-mon[32759]: osd.0 [v2:172.21.15.31:6802/788174091,v1:172.21.15.31:6803/788174091] boot 2024-04-25T01:07:28.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:27 smithi169 ceph-mon[32759]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T01:07:28.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:07:28.322 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T01:07:28.322 INFO:teuthology.orchestra.run.smithi040.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T01:07:28.322 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10+0 records in 2024-04-25T01:07:28.323 INFO:teuthology.orchestra.run.smithi040.stderr:10+0 records out 2024-04-25T01:07:28.323 INFO:teuthology.orchestra.run.smithi040.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110115 s, 952 MB/s 2024-04-25T01:07:28.323 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping successful for: 2024-04-25T01:07:28.907 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:28 smithi040 ceph-mon[32518]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T01:07:28.907 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:28 smithi040 ceph-mon[32518]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:28.935 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi040:vg_nvme/lv_4 2024-04-25T01:07:29.180 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T01:07:29.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:28 smithi169 ceph-mon[32759]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T01:07:29.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:28 smithi169 ceph-mon[32759]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:29.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:28 smithi031 ceph-mon[28053]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T01:07:29.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:28 smithi031 ceph-mon[28053]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:30.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.697+0000 7f9162d1f700 1 -- 172.21.15.40:0/2641545837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f915c1034b0 msgr2=0x7f915c103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:07:30.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.697+0000 7f9162d1f700 1 --2- 172.21.15.40:0/2641545837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f915c1034b0 0x7f915c103910 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f91500099c0 tx=0x7f9150009cd0 comp rx=0 tx=0).stop 2024-04-25T01:07:30.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.697+0000 7f9162d1f700 1 -- 172.21.15.40:0/2641545837 shutdown_connections 2024-04-25T01:07:30.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.697+0000 7f9162d1f700 1 --2- 172.21.15.40:0/2641545837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f915c1022b0 0x7f915c1026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:07:30.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.697+0000 7f9162d1f700 1 --2- 172.21.15.40:0/2641545837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f915c103e50 0x7f915c10a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:07:30.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.697+0000 7f9162d1f700 1 --2- 172.21.15.40:0/2641545837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f915c1034b0 0x7f915c103910 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:07:30.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.697+0000 7f9162d1f700 1 -- 172.21.15.40:0/2641545837 >> 172.21.15.40:0/2641545837 conn(0x7f915c0fd850 msgr2=0x7f915c0ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:07:30.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.697+0000 7f9162d1f700 1 -- 172.21.15.40:0/2641545837 shutdown_connections 2024-04-25T01:07:30.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.697+0000 7f9162d1f700 1 -- 172.21.15.40:0/2641545837 wait complete. 2024-04-25T01:07:30.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.698+0000 7f9162d1f700 1 Processor -- start 2024-04-25T01:07:30.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.698+0000 7f9162d1f700 1 -- start start 2024-04-25T01:07:30.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.698+0000 7f9162d1f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f915c1022b0 0x7f915c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:07:30.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.698+0000 7f9162d1f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f915c1034b0 0x7f915c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:07:30.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.698+0000 7f9162d1f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f915c103e50 0x7f915c19fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:07:30.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.698+0000 7f9162d1f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f915c10df30 con 0x7f915c1034b0 2024-04-25T01:07:30.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.698+0000 7f9162d1f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f915c10ddb0 con 0x7f915c103e50 2024-04-25T01:07:30.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.698+0000 7f9162d1f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f915c10e0b0 con 0x7f915c1022b0 2024-04-25T01:07:30.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.698+0000 7f91612bc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f915c103e50 0x7f915c19fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:07:30.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.698+0000 7f91612bc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f915c103e50 0x7f915c19fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57890/0 (socket says 172.21.15.40:57890) 2024-04-25T01:07:30.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.698+0000 7f91612bc700 1 -- 172.21.15.40:0/1510283391 learned_addr learned my addr 172.21.15.40:0/1510283391 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T01:07:30.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.698+0000 7f9160abb700 1 --2- 172.21.15.40:0/1510283391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f915c1022b0 0x7f915c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:07:30.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.698+0000 7f91612bc700 1 -- 172.21.15.40:0/1510283391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f915c1022b0 msgr2=0x7f915c198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:07:30.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.698+0000 7f91612bc700 1 --2- 172.21.15.40:0/1510283391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f915c1022b0 0x7f915c198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:07:30.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.698+0000 7f91612bc700 1 -- 172.21.15.40:0/1510283391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f915c1034b0 msgr2=0x7f915c198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:07:30.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.698+0000 7f91612bc700 1 --2- 172.21.15.40:0/1510283391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f915c1034b0 0x7f915c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:07:30.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.698+0000 7f91612bc700 1 -- 172.21.15.40:0/1510283391 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f915c1a0180 con 0x7f915c103e50 2024-04-25T01:07:30.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.699+0000 7f91612bc700 1 --2- 172.21.15.40:0/1510283391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f915c103e50 0x7f915c19fae0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9154007ee0 tx=0x7f915400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:07:30.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.699+0000 7f9159ffb700 1 -- 172.21.15.40:0/1510283391 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915400dbd0 con 0x7f915c103e50 2024-04-25T01:07:30.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.699+0000 7f9159ffb700 1 -- 172.21.15.40:0/1510283391 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f915400dd70 con 0x7f915c103e50 2024-04-25T01:07:30.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.699+0000 7f9162d1f700 1 -- 172.21.15.40:0/1510283391 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f915c1a0470 con 0x7f915c103e50 2024-04-25T01:07:30.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.699+0000 7f9159ffb700 1 -- 172.21.15.40:0/1510283391 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91540133f0 con 0x7f915c103e50 2024-04-25T01:07:30.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.699+0000 7f9162d1f700 1 -- 172.21.15.40:0/1510283391 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f915c1a0a80 con 0x7f915c103e50 2024-04-25T01:07:30.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.700+0000 7f9162d1f700 1 -- 172.21.15.40:0/1510283391 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f915c068f20 con 0x7f915c103e50 2024-04-25T01:07:30.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.701+0000 7f9159ffb700 1 -- 172.21.15.40:0/1510283391 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f915401a020 con 0x7f915c103e50 2024-04-25T01:07:30.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.702+0000 7f9159ffb700 1 --2- 172.21.15.40:0/1510283391 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f91440a2c90 0x7f91440a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:07:30.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.702+0000 7f9159ffb700 1 -- 172.21.15.40:0/1510283391 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f91540cd3a0 con 0x7f915c103e50 2024-04-25T01:07:30.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.702+0000 7f9160abb700 1 --2- 172.21.15.40:0/1510283391 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f91440a2c90 0x7f91440a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:07:30.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.703+0000 7f9160abb700 1 --2- 172.21.15.40:0/1510283391 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f91440a2c90 0x7f91440a5150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f914c006fd0 tx=0x7f914c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:07:30.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.706+0000 7f9159ffb700 1 -- 172.21.15.40:0/1510283391 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9154010070 con 0x7f915c103e50 2024-04-25T01:07:30.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:30.920+0000 7f9162d1f700 1 -- 172.21.15.40:0/1510283391 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f915c062670 con 0x7f91440a2c90 2024-04-25T01:07:32.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:31 smithi169 ceph-mon[32759]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:32.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:31 smithi169 ceph-mon[32759]: Detected new or changed devices on smithi031 2024-04-25T01:07:32.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:31 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:32.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:31 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:32.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:31 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:07:32.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:31 smithi169 ceph-mon[32759]: Adjusting osd_memory_target on smithi031 to 17094M 2024-04-25T01:07:32.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:31 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:32.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:31 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:32.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:31 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:07:32.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:31 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:32.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:31 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:07:32.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:31 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:07:32.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:31 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:32.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:31 smithi040 ceph-mon[32518]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:32.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:31 smithi040 ceph-mon[32518]: Detected new or changed devices on smithi031 2024-04-25T01:07:32.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:31 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:32.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:31 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:32.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:31 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:07:32.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:31 smithi040 ceph-mon[32518]: Adjusting osd_memory_target on smithi031 to 17094M 2024-04-25T01:07:32.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:31 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:32.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:31 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:32.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:31 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:07:32.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:31 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:32.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:31 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:07:32.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:31 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:07:32.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:31 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:32.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:31 smithi031 ceph-mon[28053]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:32.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:31 smithi031 ceph-mon[28053]: Detected new or changed devices on smithi031 2024-04-25T01:07:32.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:31 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:32.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:31 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:32.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:31 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:07:32.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:31 smithi031 ceph-mon[28053]: Adjusting osd_memory_target on smithi031 to 17094M 2024-04-25T01:07:32.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:31 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:32.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:31 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:32.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:31 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:07:32.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:31 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:32.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:31 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:07:32.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:31 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:07:32.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:31 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:33.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:32 smithi169 ceph-mon[32759]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:07:33.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:32 smithi040 ceph-mon[32518]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:07:33.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:32 smithi031 ceph-mon[28053]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:07:34.166 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:33 smithi031 ceph-mon[28053]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:34.166 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:33 smithi031 ceph-mon[28053]: from='client.? 172.21.15.40:0/3044784031' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85339fe7-0dbc-4485-9334-2b039993c1f1"}]: dispatch 2024-04-25T01:07:34.166 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:33 smithi031 ceph-mon[28053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85339fe7-0dbc-4485-9334-2b039993c1f1"}]: dispatch 2024-04-25T01:07:34.166 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:33 smithi031 ceph-mon[28053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85339fe7-0dbc-4485-9334-2b039993c1f1"}]': finished 2024-04-25T01:07:34.166 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:33 smithi031 ceph-mon[28053]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T01:07:34.167 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:33 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:07:34.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:33 smithi169 ceph-mon[32759]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:34.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:33 smithi169 ceph-mon[32759]: from='client.? 172.21.15.40:0/3044784031' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85339fe7-0dbc-4485-9334-2b039993c1f1"}]: dispatch 2024-04-25T01:07:34.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:33 smithi169 ceph-mon[32759]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85339fe7-0dbc-4485-9334-2b039993c1f1"}]: dispatch 2024-04-25T01:07:34.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:33 smithi169 ceph-mon[32759]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85339fe7-0dbc-4485-9334-2b039993c1f1"}]': finished 2024-04-25T01:07:34.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:33 smithi169 ceph-mon[32759]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T01:07:34.202 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:33 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:07:34.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:33 smithi040 ceph-mon[32518]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:34.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:33 smithi040 ceph-mon[32518]: from='client.? 172.21.15.40:0/3044784031' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85339fe7-0dbc-4485-9334-2b039993c1f1"}]: dispatch 2024-04-25T01:07:34.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:33 smithi040 ceph-mon[32518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85339fe7-0dbc-4485-9334-2b039993c1f1"}]: dispatch 2024-04-25T01:07:34.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:33 smithi040 ceph-mon[32518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85339fe7-0dbc-4485-9334-2b039993c1f1"}]': finished 2024-04-25T01:07:34.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:33 smithi040 ceph-mon[32518]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T01:07:34.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:33 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:07:35.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:34 smithi169 ceph-mon[32759]: from='client.? 172.21.15.40:0/3438867742' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:07:35.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:34 smithi040 ceph-mon[32518]: from='client.? 172.21.15.40:0/3438867742' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:07:35.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:34 smithi031 ceph-mon[28053]: from='client.? 172.21.15.40:0/3438867742' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:07:36.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:35 smithi169 ceph-mon[32759]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:36.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:35 smithi040 ceph-mon[32518]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:36.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:35 smithi031 ceph-mon[28053]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:37.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:36 smithi169 ceph-mon[32759]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:37.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:36 smithi040 ceph-mon[32518]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:37.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:36 smithi031 ceph-mon[28053]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:39.133 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:39 smithi040 ceph-mon[32518]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:39.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:39 smithi169 ceph-mon[32759]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:39.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:39 smithi031 ceph-mon[28053]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:41.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:41 smithi040 ceph-mon[32518]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:41.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T01:07:41.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:41.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:41 smithi169 ceph-mon[32759]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:41.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T01:07:41.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:41.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:41 smithi031 ceph-mon[28053]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:41.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T01:07:41.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:42.180 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:42 smithi040 ceph-mon[32518]: Deploying daemon osd.1 on smithi040 2024-04-25T01:07:42.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:42 smithi169 ceph-mon[32759]: Deploying daemon osd.1 on smithi040 2024-04-25T01:07:42.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:42 smithi031 ceph-mon[28053]: Deploying daemon osd.1 on smithi040 2024-04-25T01:07:43.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:43 smithi040 ceph-mon[32518]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:43.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:43 smithi169 ceph-mon[32759]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:43.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:43 smithi031 ceph-mon[28053]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:45.320 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:45 smithi040 ceph-mon[32518]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:45.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:45 smithi169 ceph-mon[32759]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:45.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:45 smithi031 ceph-mon[28053]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:47.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:47 smithi169 ceph-mon[32759]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:47.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:47 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:47.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:47 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:47.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:47 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:07:47.454 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:47 smithi040 ceph-mon[32518]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:47.454 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:47 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:47.454 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:47 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:47.454 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:47 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:07:47.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:47 smithi031 ceph-mon[28053]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:47.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:47 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:47.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:47 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:47.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:47 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:07:49.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:49 smithi169 ceph-mon[32759]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:49.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:49 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:49.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:49 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:49.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:49 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:49.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:49 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:07:49.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:49 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:49.454 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:49 smithi040 ceph-mon[32518]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:49.454 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:49 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:49.454 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:49 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:49.454 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:49 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:49.454 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:49 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:07:49.454 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:49 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:49.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:49 smithi031 ceph-mon[28053]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:49.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:49 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:49.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:49 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:49.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:49 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:49.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:49 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:07:49.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:49 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:49.898 INFO:teuthology.orchestra.run.smithi040.stdout:Created osd(s) 1 on host 'smithi040' 2024-04-25T01:07:49.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:49.905+0000 7f9159ffb700 1 -- 172.21.15.40:0/1510283391 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f915c062670 con 0x7f91440a2c90 2024-04-25T01:07:49.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:49.908+0000 7f9162d1f700 1 -- 172.21.15.40:0/1510283391 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f91440a2c90 msgr2=0x7f91440a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:07:49.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:49.908+0000 7f9162d1f700 1 --2- 172.21.15.40:0/1510283391 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f91440a2c90 0x7f91440a5150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f914c006fd0 tx=0x7f914c008040 comp rx=0 tx=0).stop 2024-04-25T01:07:49.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:49.908+0000 7f9162d1f700 1 -- 172.21.15.40:0/1510283391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f915c103e50 msgr2=0x7f915c19fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:07:49.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:49.908+0000 7f9162d1f700 1 --2- 172.21.15.40:0/1510283391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f915c103e50 0x7f915c19fae0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9154007ee0 tx=0x7f915400d3b0 comp rx=0 tx=0).stop 2024-04-25T01:07:49.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:49.908+0000 7f9162d1f700 1 -- 172.21.15.40:0/1510283391 shutdown_connections 2024-04-25T01:07:49.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:49.908+0000 7f9162d1f700 1 --2- 172.21.15.40:0/1510283391 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f91440a2c90 0x7f91440a5150 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:07:49.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:49.908+0000 7f9162d1f700 1 --2- 172.21.15.40:0/1510283391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f915c103e50 0x7f915c19fae0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:07:49.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:49.908+0000 7f9162d1f700 1 --2- 172.21.15.40:0/1510283391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f915c1022b0 0x7f915c198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:07:49.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:49.908+0000 7f9162d1f700 1 --2- 172.21.15.40:0/1510283391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f915c1034b0 0x7f915c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:07:49.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:49.908+0000 7f9162d1f700 1 -- 172.21.15.40:0/1510283391 >> 172.21.15.40:0/1510283391 conn(0x7f915c0fd850 msgr2=0x7f915c106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:07:49.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:49.908+0000 7f9162d1f700 1 -- 172.21.15.40:0/1510283391 shutdown_connections 2024-04-25T01:07:49.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:07:49.908+0000 7f9162d1f700 1 -- 172.21.15.40:0/1510283391 wait complete. 2024-04-25T01:07:50.617 DEBUG:teuthology.orchestra.run.smithi040:osd.1> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T01:07:50.619 INFO:tasks.cephadm:Deploying osd.2 on smithi169 with /dev/vg_nvme/lv_4... 2024-04-25T01:07:50.619 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T01:07:50.864 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:07:50.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:50 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:50.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:50 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:50.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:50 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:07:50.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:50 smithi169 ceph-mon[32759]: from='osd.1 [v2:172.21.15.40:6800/3196394509,v1:172.21.15.40:6801/3196394509]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T01:07:50.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:50 smithi169 ceph-mon[32759]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T01:07:51.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:50 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:51.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:50 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:51.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:50 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:07:51.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:50 smithi040 ceph-mon[32518]: from='osd.1 [v2:172.21.15.40:6800/3196394509,v1:172.21.15.40:6801/3196394509]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T01:07:51.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:50 smithi040 ceph-mon[32518]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T01:07:51.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:50 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:51.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:50 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:51.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:50 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:07:51.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:50 smithi031 ceph-mon[28053]: from='osd.1 [v2:172.21.15.40:6800/3196394509,v1:172.21.15.40:6801/3196394509]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T01:07:51.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:50 smithi031 ceph-mon[28053]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T01:07:52.059 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:51 smithi169 ceph-mon[32759]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:52.059 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:51 smithi169 ceph-mon[32759]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T01:07:52.059 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:51 smithi169 ceph-mon[32759]: from='osd.1 [v2:172.21.15.40:6800/3196394509,v1:172.21.15.40:6801/3196394509]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T01:07:52.060 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:51 smithi169 ceph-mon[32759]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T01:07:52.060 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:51 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:07:52.060 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:51 smithi169 ceph-mon[32759]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T01:07:52.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:51 smithi040 ceph-mon[32518]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:52.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:51 smithi040 ceph-mon[32518]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T01:07:52.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:51 smithi040 ceph-mon[32518]: from='osd.1 [v2:172.21.15.40:6800/3196394509,v1:172.21.15.40:6801/3196394509]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T01:07:52.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:51 smithi040 ceph-mon[32518]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T01:07:52.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:51 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:07:52.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:51 smithi040 ceph-mon[32518]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T01:07:52.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:51 smithi031 ceph-mon[28053]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:52.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:51 smithi031 ceph-mon[28053]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T01:07:52.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:51 smithi031 ceph-mon[28053]: from='osd.1 [v2:172.21.15.40:6800/3196394509,v1:172.21.15.40:6801/3196394509]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T01:07:52.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:51 smithi031 ceph-mon[28053]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T01:07:52.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:51 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:07:52.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:51 smithi031 ceph-mon[28053]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T01:07:52.574 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T01:07:52.574 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T01:07:52.575 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-04-25T01:07:52.575 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-04-25T01:07:52.575 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114494 s, 916 MB/s 2024-04-25T01:07:52.575 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-04-25T01:07:53.106 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:52 smithi040 ceph-mon[32518]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-25T01:07:53.106 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:52 smithi040 ceph-mon[32518]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T01:07:53.106 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:52 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:07:53.106 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:52 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:07:53.106 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:52 smithi040 ceph-mon[32518]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:53.107 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:52 smithi040 ceph-mon[32518]: from='osd.1 ' entity='osd.1' 2024-04-25T01:07:53.107 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:07:52 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-1[37137]: 2024-04-25T01:07:52.823+0000 7f1a52efe700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T01:07:53.107 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:07:52 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-1[37137]: 2024-04-25T01:07:52.830+0000 7f1a4b4ea700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:07:53.125 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_4 2024-04-25T01:07:53.136 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:52 smithi169 ceph-mon[32759]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-25T01:07:53.136 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:52 smithi169 ceph-mon[32759]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T01:07:53.136 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:52 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:07:53.136 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:52 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:07:53.136 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:52 smithi169 ceph-mon[32759]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:53.136 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:52 smithi169 ceph-mon[32759]: from='osd.1 ' entity='osd.1' 2024-04-25T01:07:53.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:52 smithi031 ceph-mon[28053]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-25T01:07:53.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:52 smithi031 ceph-mon[28053]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T01:07:53.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:52 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:07:53.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:52 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:07:53.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:52 smithi031 ceph-mon[28053]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:07:53.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:52 smithi031 ceph-mon[28053]: from='osd.1 ' entity='osd.1' 2024-04-25T01:07:53.366 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:07:54.164 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:53 smithi031 ceph-mon[28053]: purged_snaps scrub starts 2024-04-25T01:07:54.164 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:53 smithi031 ceph-mon[28053]: purged_snaps scrub ok 2024-04-25T01:07:54.165 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:53 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:07:54.165 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:53 smithi031 ceph-mon[28053]: Detected new or changed devices on smithi040 2024-04-25T01:07:54.165 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:53 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:54.165 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:53 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:54.165 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:53 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:07:54.165 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:53 smithi031 ceph-mon[28053]: Adjusting osd_memory_target on smithi040 to 17093M 2024-04-25T01:07:54.165 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:53 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:54.165 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:53 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:54.165 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:53 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:07:54.165 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:53 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:54.165 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:53 smithi031 ceph-mon[28053]: osd.1 [v2:172.21.15.40:6800/3196394509,v1:172.21.15.40:6801/3196394509] boot 2024-04-25T01:07:54.165 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:53 smithi031 ceph-mon[28053]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T01:07:54.165 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:53 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:07:54.188 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:53 smithi169 ceph-mon[32759]: purged_snaps scrub starts 2024-04-25T01:07:54.188 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:53 smithi169 ceph-mon[32759]: purged_snaps scrub ok 2024-04-25T01:07:54.188 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:53 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:07:54.188 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:53 smithi169 ceph-mon[32759]: Detected new or changed devices on smithi040 2024-04-25T01:07:54.188 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:53 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:54.188 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:53 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:54.188 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:53 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:07:54.188 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:53 smithi169 ceph-mon[32759]: Adjusting osd_memory_target on smithi040 to 17093M 2024-04-25T01:07:54.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:53 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:54.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:53 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:54.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:53 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:07:54.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:53 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:54.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:53 smithi169 ceph-mon[32759]: osd.1 [v2:172.21.15.40:6800/3196394509,v1:172.21.15.40:6801/3196394509] boot 2024-04-25T01:07:54.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:53 smithi169 ceph-mon[32759]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T01:07:54.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:53 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:07:54.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:53 smithi040 ceph-mon[32518]: purged_snaps scrub starts 2024-04-25T01:07:54.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:53 smithi040 ceph-mon[32518]: purged_snaps scrub ok 2024-04-25T01:07:54.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:53 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:07:54.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:53 smithi040 ceph-mon[32518]: Detected new or changed devices on smithi040 2024-04-25T01:07:54.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:53 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:54.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:53 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:54.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:53 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:07:54.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:53 smithi040 ceph-mon[32518]: Adjusting osd_memory_target on smithi040 to 17093M 2024-04-25T01:07:54.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:53 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:54.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:53 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:54.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:53 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:07:54.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:53 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:07:54.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:53 smithi040 ceph-mon[32518]: osd.1 [v2:172.21.15.40:6800/3196394509,v1:172.21.15.40:6801/3196394509] boot 2024-04-25T01:07:54.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:53 smithi040 ceph-mon[32518]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T01:07:54.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:53 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:07:54.896 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.903+0000 7f4c71ee0700 1 -- 172.21.15.169:0/2230118639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c6c102390 msgr2=0x7f4c6c1027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:07:54.897 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.903+0000 7f4c71ee0700 1 --2- 172.21.15.169:0/2230118639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c6c102390 0x7f4c6c1027f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c009990 tx=0x7f4c5c009ca0 comp rx=0 tx=0).stop 2024-04-25T01:07:54.897 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.903+0000 7f4c71ee0700 1 -- 172.21.15.169:0/2230118639 shutdown_connections 2024-04-25T01:07:54.897 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.903+0000 7f4c71ee0700 1 --2- 172.21.15.169:0/2230118639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c6c101190 0x7f4c6c1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:07:54.897 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.903+0000 7f4c71ee0700 1 --2- 172.21.15.169:0/2230118639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c6c102d30 0x7f4c6c1095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:07:54.897 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.903+0000 7f4c71ee0700 1 --2- 172.21.15.169:0/2230118639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c6c102390 0x7f4c6c1027f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:07:54.897 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.903+0000 7f4c71ee0700 1 -- 172.21.15.169:0/2230118639 >> 172.21.15.169:0/2230118639 conn(0x7f4c6c0fc730 msgr2=0x7f4c6c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:07:54.897 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.903+0000 7f4c71ee0700 1 -- 172.21.15.169:0/2230118639 shutdown_connections 2024-04-25T01:07:54.897 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.903+0000 7f4c71ee0700 1 -- 172.21.15.169:0/2230118639 wait complete. 2024-04-25T01:07:54.897 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.904+0000 7f4c71ee0700 1 Processor -- start 2024-04-25T01:07:54.897 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.904+0000 7f4c71ee0700 1 -- start start 2024-04-25T01:07:54.897 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.904+0000 7f4c71ee0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c6c101190 0x7f4c6c06aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:07:54.898 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.904+0000 7f4c71ee0700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c6c102390 0x7f4c6c19d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:07:54.898 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.904+0000 7f4c71ee0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c6c102d30 0x7f4c6c19f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:07:54.898 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.904+0000 7f4c71ee0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c10ce70 con 0x7f4c6c101190 2024-04-25T01:07:54.898 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.904+0000 7f4c71ee0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c10ccf0 con 0x7f4c6c102d30 2024-04-25T01:07:54.898 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.904+0000 7f4c71ee0700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c10cff0 con 0x7f4c6c102390 2024-04-25T01:07:54.898 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.904+0000 7f4c6affd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c6c102390 0x7f4c6c19d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:07:54.898 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.904+0000 7f4c6affd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c6c102390 0x7f4c6c19d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47494/0 (socket says 172.21.15.169:47494) 2024-04-25T01:07:54.898 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.904+0000 7f4c6affd700 1 -- 172.21.15.169:0/3147151762 learned_addr learned my addr 172.21.15.169:0/3147151762 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T01:07:54.898 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.904+0000 7f4c6bfff700 1 --2- 172.21.15.169:0/3147151762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c6c102d30 0x7f4c6c19f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:07:54.898 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.904+0000 7f4c6affd700 1 -- 172.21.15.169:0/3147151762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c6c102d30 msgr2=0x7f4c6c19f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:07:54.898 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.904+0000 7f4c6affd700 1 --2- 172.21.15.169:0/3147151762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c6c102d30 0x7f4c6c19f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:07:54.898 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.904+0000 7f4c6affd700 1 -- 172.21.15.169:0/3147151762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c6c101190 msgr2=0x7f4c6c06aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:07:54.898 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.904+0000 7f4c6affd700 1 --2- 172.21.15.169:0/3147151762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c6c101190 0x7f4c6c06aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:07:54.899 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.904+0000 7f4c6affd700 1 -- 172.21.15.169:0/3147151762 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6c1a0220 con 0x7f4c6c102390 2024-04-25T01:07:54.899 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.905+0000 7f4c6affd700 1 --2- 172.21.15.169:0/3147151762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c6c102390 0x7f4c6c19d300 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c000c00 tx=0x7f4c5c00bfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:07:54.900 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.908+0000 7f4c68ff9700 1 -- 172.21.15.169:0/3147151762 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c026070 con 0x7f4c6c102390 2024-04-25T01:07:54.900 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.908+0000 7f4c68ff9700 1 -- 172.21.15.169:0/3147151762 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c5c021410 con 0x7f4c6c102390 2024-04-25T01:07:54.900 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.908+0000 7f4c71ee0700 1 -- 172.21.15.169:0/3147151762 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c6c1a04b0 con 0x7f4c6c102390 2024-04-25T01:07:54.900 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.908+0000 7f4c68ff9700 1 -- 172.21.15.169:0/3147151762 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c017650 con 0x7f4c6c102390 2024-04-25T01:07:54.900 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.908+0000 7f4c71ee0700 1 -- 172.21.15.169:0/3147151762 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c6c1a09e0 con 0x7f4c6c102390 2024-04-25T01:07:54.901 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.909+0000 7f4c71ee0700 1 -- 172.21.15.169:0/3147151762 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c6c068f20 con 0x7f4c6c102390 2024-04-25T01:07:54.901 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.910+0000 7f4c68ff9700 1 -- 172.21.15.169:0/3147151762 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f4c5c029070 con 0x7f4c6c102390 2024-04-25T01:07:54.902 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.910+0000 7f4c68ff9700 1 --2- 172.21.15.169:0/3147151762 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f4c580a2c10 0x7f4c580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:07:54.902 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.910+0000 7f4c68ff9700 1 -- 172.21.15.169:0/3147151762 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f4c5c0ce4e0 con 0x7f4c6c102390 2024-04-25T01:07:54.903 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.911+0000 7f4c6b7fe700 1 --2- 172.21.15.169:0/3147151762 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f4c580a2c10 0x7f4c580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:07:54.904 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.912+0000 7f4c6b7fe700 1 --2- 172.21.15.169:0/3147151762 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f4c580a2c10 0x7f4c580a50d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c1021f0 tx=0x7f4c54006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:07:54.907 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:54.915+0000 7f4c68ff9700 1 -- 172.21.15.169:0/3147151762 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4c5c013070 con 0x7f4c6c102390 2024-04-25T01:07:55.014 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:54 smithi040 ceph-mon[32518]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:07:55.111 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:07:55.118+0000 7f4c71ee0700 1 -- 172.21.15.169:0/3147151762 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4c6c062670 con 0x7f4c580a2c10 2024-04-25T01:07:55.111 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:54 smithi169 ceph-mon[32759]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:07:55.189 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:54 smithi031 ceph-mon[28053]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:07:56.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:55 smithi169 ceph-mon[32759]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T01:07:56.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:55 smithi169 ceph-mon[32759]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:07:56.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:55 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:07:56.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:55 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:07:56.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:55 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:56.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:55 smithi040 ceph-mon[32518]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T01:07:56.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:55 smithi040 ceph-mon[32518]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:07:56.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:55 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:07:56.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:55 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:07:56.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:55 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:56.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:55 smithi031 ceph-mon[28053]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T01:07:56.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:55 smithi031 ceph-mon[28053]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:07:56.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:55 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:07:56.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:55 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:07:56.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:55 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:07:57.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:56 smithi169 ceph-mon[32759]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:07:57.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:56 smithi040 ceph-mon[32518]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:07:57.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:56 smithi031 ceph-mon[28053]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:07:58.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:57 smithi169 ceph-mon[32759]: from='client.? 172.21.15.169:0/2230146352' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab62ecb2-8aae-4987-9e2e-589e0f2fee5c"}]: dispatch 2024-04-25T01:07:58.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:57 smithi169 ceph-mon[32759]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab62ecb2-8aae-4987-9e2e-589e0f2fee5c"}]: dispatch 2024-04-25T01:07:58.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:57 smithi169 ceph-mon[32759]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab62ecb2-8aae-4987-9e2e-589e0f2fee5c"}]': finished 2024-04-25T01:07:58.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:57 smithi169 ceph-mon[32759]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T01:07:58.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:57 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:07:58.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:57 smithi040 ceph-mon[32518]: from='client.? 172.21.15.169:0/2230146352' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab62ecb2-8aae-4987-9e2e-589e0f2fee5c"}]: dispatch 2024-04-25T01:07:58.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:57 smithi040 ceph-mon[32518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab62ecb2-8aae-4987-9e2e-589e0f2fee5c"}]: dispatch 2024-04-25T01:07:58.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:57 smithi040 ceph-mon[32518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab62ecb2-8aae-4987-9e2e-589e0f2fee5c"}]': finished 2024-04-25T01:07:58.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:57 smithi040 ceph-mon[32518]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T01:07:58.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:57 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:07:58.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:57 smithi031 ceph-mon[28053]: from='client.? 172.21.15.169:0/2230146352' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab62ecb2-8aae-4987-9e2e-589e0f2fee5c"}]: dispatch 2024-04-25T01:07:58.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:57 smithi031 ceph-mon[28053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab62ecb2-8aae-4987-9e2e-589e0f2fee5c"}]: dispatch 2024-04-25T01:07:58.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:57 smithi031 ceph-mon[28053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab62ecb2-8aae-4987-9e2e-589e0f2fee5c"}]': finished 2024-04-25T01:07:58.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:57 smithi031 ceph-mon[28053]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T01:07:58.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:57 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:07:59.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:58 smithi169 ceph-mon[32759]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:07:59.201 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:07:58 smithi169 ceph-mon[32759]: from='client.? 172.21.15.169:0/288177673' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:07:59.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:58 smithi040 ceph-mon[32518]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:07:59.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:07:58 smithi040 ceph-mon[32518]: from='client.? 172.21.15.169:0/288177673' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:07:59.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:58 smithi031 ceph-mon[28053]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:07:59.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:07:58 smithi031 ceph-mon[28053]: from='client.? 172.21.15.169:0/288177673' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:08:01.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:01 smithi169 ceph-mon[32759]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:01.453 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:01 smithi040 ceph-mon[32518]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:01.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:01 smithi031 ceph-mon[28053]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:03.450 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:03 smithi169 ceph-mon[32759]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:03.453 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:03 smithi040 ceph-mon[32518]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:03.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:03 smithi031 ceph-mon[28053]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:05.360 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:05 smithi169 ceph-mon[32759]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:05.360 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:05 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T01:08:05.360 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:05 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:08:05.361 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:05 smithi169 ceph-mon[32759]: Deploying daemon osd.2 on smithi169 2024-04-25T01:08:05.453 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:05 smithi040 ceph-mon[32518]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:05.454 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:05 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T01:08:05.454 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:05 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:08:05.454 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:05 smithi040 ceph-mon[32518]: Deploying daemon osd.2 on smithi169 2024-04-25T01:08:05.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:05 smithi031 ceph-mon[28053]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:05.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:05 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T01:08:05.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:05 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:08:05.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:05 smithi031 ceph-mon[28053]: Deploying daemon osd.2 on smithi169 2024-04-25T01:08:07.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:07 smithi169 ceph-mon[32759]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:07.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:07 smithi031 ceph-mon[28053]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:07.453 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:07 smithi040 ceph-mon[32518]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:09.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:09 smithi031 ceph-mon[28053]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:09.442 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:09 smithi169 ceph-mon[32759]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:09.453 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:09 smithi040 ceph-mon[32518]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:10.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:10 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:10.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:10 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:10.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:10 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:08:11.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:10 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:11.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:10 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:11.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:10 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:08:11.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:10 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:11.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:10 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:11.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:10 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:08:11.869 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:11 smithi169 ceph-mon[32759]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:12.205 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:11 smithi040 ceph-mon[32518]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:12.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:11 smithi031 ceph-mon[28053]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:13.246 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 2 on host 'smithi169' 2024-04-25T01:08:13.246 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:08:13.254+0000 7f4c68ff9700 1 -- 172.21.15.169:0/3147151762 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4c6c062670 con 0x7f4c580a2c10 2024-04-25T01:08:13.248 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:08:13.256+0000 7f4c71ee0700 1 -- 172.21.15.169:0/3147151762 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f4c580a2c10 msgr2=0x7f4c580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:13.248 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:08:13.256+0000 7f4c71ee0700 1 --2- 172.21.15.169:0/3147151762 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f4c580a2c10 0x7f4c580a50d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c1021f0 tx=0x7f4c54006d20 comp rx=0 tx=0).stop 2024-04-25T01:08:13.249 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:08:13.257+0000 7f4c71ee0700 1 -- 172.21.15.169:0/3147151762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c6c102390 msgr2=0x7f4c6c19d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:13.249 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:08:13.257+0000 7f4c71ee0700 1 --2- 172.21.15.169:0/3147151762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c6c102390 0x7f4c6c19d300 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c000c00 tx=0x7f4c5c00bfc0 comp rx=0 tx=0).stop 2024-04-25T01:08:13.249 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:08:13.257+0000 7f4c71ee0700 1 -- 172.21.15.169:0/3147151762 shutdown_connections 2024-04-25T01:08:13.249 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:08:13.257+0000 7f4c71ee0700 1 --2- 172.21.15.169:0/3147151762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c6c101190 0x7f4c6c06aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:13.249 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:08:13.257+0000 7f4c71ee0700 1 --2- 172.21.15.169:0/3147151762 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f4c580a2c10 0x7f4c580a50d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:13.249 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:08:13.257+0000 7f4c71ee0700 1 --2- 172.21.15.169:0/3147151762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c6c102d30 0x7f4c6c19f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:13.249 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:08:13.257+0000 7f4c71ee0700 1 --2- 172.21.15.169:0/3147151762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c6c102390 0x7f4c6c19d300 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:13.249 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:08:13.257+0000 7f4c71ee0700 1 -- 172.21.15.169:0/3147151762 >> 172.21.15.169:0/3147151762 conn(0x7f4c6c0fc730 msgr2=0x7f4c6c105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:13.249 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:08:13.257+0000 7f4c71ee0700 1 -- 172.21.15.169:0/3147151762 shutdown_connections 2024-04-25T01:08:13.249 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:08:13.257+0000 7f4c71ee0700 1 -- 172.21.15.169:0/3147151762 wait complete. 2024-04-25T01:08:13.450 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:13 smithi169 ceph-mon[32759]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:13.450 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:13.450 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:13.450 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:08:13.450 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:08:13.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:13.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:13.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:13.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:08:13.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:13 smithi040 ceph-mon[32518]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:13.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:13.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:13.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:08:13.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:08:13.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:13.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:13.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:13.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:08:13.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:13 smithi031 ceph-mon[28053]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:13.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:13.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:13.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:08:13.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:08:13.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:13.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:13.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:13.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:08:13.821 DEBUG:teuthology.orchestra.run.smithi169:osd.2> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T01:08:13.864 INFO:tasks.cephadm:Deploying osd.3 on smithi031 with /dev/vg_nvme/lv_3... 2024-04-25T01:08:13.864 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T01:08:14.339 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:08:14.679 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:14 smithi031 ceph-mon[28053]: from='osd.2 [v2:172.21.15.169:6800/1955211197,v1:172.21.15.169:6801/1955211197]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T01:08:14.679 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:14 smithi031 ceph-mon[28053]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T01:08:14.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:14 smithi169 ceph-mon[32759]: from='osd.2 [v2:172.21.15.169:6800/1955211197,v1:172.21.15.169:6801/1955211197]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T01:08:14.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:14 smithi169 ceph-mon[32759]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T01:08:14.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:14 smithi040 ceph-mon[32518]: from='osd.2 [v2:172.21.15.169:6800/1955211197,v1:172.21.15.169:6801/1955211197]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T01:08:14.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:14 smithi040 ceph-mon[32518]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T01:08:15.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:15 smithi031 ceph-mon[28053]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:15.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:15 smithi031 ceph-mon[28053]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T01:08:15.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:15 smithi031 ceph-mon[28053]: from='osd.2 [v2:172.21.15.169:6800/1955211197,v1:172.21.15.169:6801/1955211197]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T01:08:15.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:15 smithi031 ceph-mon[28053]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T01:08:15.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:15 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:08:15.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:15 smithi031 ceph-mon[28053]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T01:08:15.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:15 smithi169 ceph-mon[32759]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:15.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:15 smithi169 ceph-mon[32759]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T01:08:15.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:15 smithi169 ceph-mon[32759]: from='osd.2 [v2:172.21.15.169:6800/1955211197,v1:172.21.15.169:6801/1955211197]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T01:08:15.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:15 smithi169 ceph-mon[32759]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T01:08:15.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:15 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:08:15.697 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:15 smithi169 ceph-mon[32759]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T01:08:15.697 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:08:15 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-2[37904]: 2024-04-25T01:08:15.438+0000 7f6fbb3c2700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T01:08:15.697 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:08:15 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-2[37904]: 2024-04-25T01:08:15.443+0000 7f6fb79b6700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:08:15.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:15 smithi040 ceph-mon[32518]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:15.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:15 smithi040 ceph-mon[32518]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T01:08:15.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:15 smithi040 ceph-mon[32518]: from='osd.2 [v2:172.21.15.169:6800/1955211197,v1:172.21.15.169:6801/1955211197]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T01:08:15.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:15 smithi040 ceph-mon[32518]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T01:08:15.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:15 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:08:15.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:15 smithi040 ceph-mon[32518]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T01:08:16.504 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:16 smithi169 ceph-mon[32759]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T01:08:16.504 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:16 smithi169 ceph-mon[32759]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T01:08:16.504 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:16 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:08:16.504 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:16 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:08:16.504 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:16 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:08:16.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:16 smithi040 ceph-mon[32518]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T01:08:16.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:16 smithi040 ceph-mon[32518]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T01:08:16.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:16 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:08:16.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:16 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:08:16.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:16 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:08:16.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:16 smithi031 ceph-mon[28053]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T01:08:16.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:16 smithi031 ceph-mon[28053]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T01:08:16.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:16 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:08:16.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:16 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:08:16.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:16 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:08:17.102 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T01:08:17.102 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T01:08:17.103 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2024-04-25T01:08:17.103 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2024-04-25T01:08:17.103 INFO:teuthology.orchestra.run.smithi031.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0383378 s, 274 MB/s 2024-04-25T01:08:17.103 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2024-04-25T01:08:17.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:17 smithi169 ceph-mon[32759]: purged_snaps scrub starts 2024-04-25T01:08:17.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:17 smithi169 ceph-mon[32759]: purged_snaps scrub ok 2024-04-25T01:08:17.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:17 smithi169 ceph-mon[32759]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:17.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:17 smithi169 ceph-mon[32759]: osd.2 [v2:172.21.15.169:6800/1955211197,v1:172.21.15.169:6801/1955211197] boot 2024-04-25T01:08:17.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:17 smithi169 ceph-mon[32759]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T01:08:17.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:08:17.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:17 smithi169 ceph-mon[32759]: Detected new or changed devices on smithi169 2024-04-25T01:08:17.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:17.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:17.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:08:17.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:17 smithi169 ceph-mon[32759]: Adjusting osd_memory_target on smithi169 to 17088M 2024-04-25T01:08:17.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:17.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:08:17.702 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:08:17.702 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:17.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:17 smithi040 ceph-mon[32518]: purged_snaps scrub starts 2024-04-25T01:08:17.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:17 smithi040 ceph-mon[32518]: purged_snaps scrub ok 2024-04-25T01:08:17.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:17 smithi040 ceph-mon[32518]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:17.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:17 smithi040 ceph-mon[32518]: osd.2 [v2:172.21.15.169:6800/1955211197,v1:172.21.15.169:6801/1955211197] boot 2024-04-25T01:08:17.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:17 smithi040 ceph-mon[32518]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T01:08:17.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:08:17.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:17 smithi040 ceph-mon[32518]: Detected new or changed devices on smithi169 2024-04-25T01:08:17.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:17.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:17.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:08:17.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:17 smithi040 ceph-mon[32518]: Adjusting osd_memory_target on smithi169 to 17088M 2024-04-25T01:08:17.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:17.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:08:17.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:08:17.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:17.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:17 smithi031 ceph-mon[28053]: purged_snaps scrub starts 2024-04-25T01:08:17.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:17 smithi031 ceph-mon[28053]: purged_snaps scrub ok 2024-04-25T01:08:17.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:17 smithi031 ceph-mon[28053]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:08:17.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:17 smithi031 ceph-mon[28053]: osd.2 [v2:172.21.15.169:6800/1955211197,v1:172.21.15.169:6801/1955211197] boot 2024-04-25T01:08:17.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:17 smithi031 ceph-mon[28053]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T01:08:17.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:08:17.745 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:17 smithi031 ceph-mon[28053]: Detected new or changed devices on smithi169 2024-04-25T01:08:17.745 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:17.745 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:17.745 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:08:17.745 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:17 smithi031 ceph-mon[28053]: Adjusting osd_memory_target on smithi169 to 17088M 2024-04-25T01:08:17.745 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:17.745 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:08:17.745 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:08:17.745 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:17.815 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_3 2024-04-25T01:08:18.270 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:08:18.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:18 smithi169 ceph-mon[32759]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T01:08:18.953 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:18 smithi040 ceph-mon[32518]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T01:08:19.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:18 smithi031 ceph-mon[28053]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T01:08:19.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:19 smithi031 ceph-mon[28053]: pgmap v92: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:19.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:19 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T01:08:19.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:19 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T01:08:19.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:19 smithi031 ceph-mon[28053]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T01:08:19.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:19 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T01:08:19.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:19 smithi169 ceph-mon[32759]: pgmap v92: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:19.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:19 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T01:08:19.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:19 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T01:08:19.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:19 smithi169 ceph-mon[32759]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T01:08:19.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:19 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T01:08:19.953 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:19 smithi040 ceph-mon[32518]: pgmap v92: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:19.953 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:19 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T01:08:19.953 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:19 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T01:08:19.953 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:19 smithi040 ceph-mon[32518]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T01:08:19.953 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:19 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T01:08:20.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.292+0000 7f4423a00700 1 -- 172.21.15.31:0/2797449675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f441c1022a0 msgr2=0x7f441c1026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:20.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.292+0000 7f4423a00700 1 --2- 172.21.15.31:0/2797449675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f441c1022a0 0x7f441c1026c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f44100098f0 tx=0x7f4410009c00 comp rx=0 tx=0).stop 2024-04-25T01:08:20.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.292+0000 7f4423a00700 1 -- 172.21.15.31:0/2797449675 shutdown_connections 2024-04-25T01:08:20.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.292+0000 7f4423a00700 1 --2- 172.21.15.31:0/2797449675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f441c103e40 0x7f441c10a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:20.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.292+0000 7f4423a00700 1 --2- 172.21.15.31:0/2797449675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f441c1022a0 0x7f441c1026c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:20.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.292+0000 7f4423a00700 1 --2- 172.21.15.31:0/2797449675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f441c1034a0 0x7f441c103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:20.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.292+0000 7f4423a00700 1 -- 172.21.15.31:0/2797449675 >> 172.21.15.31:0/2797449675 conn(0x7f441c0fd800 msgr2=0x7f441c0ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:20.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.292+0000 7f4423a00700 1 -- 172.21.15.31:0/2797449675 shutdown_connections 2024-04-25T01:08:20.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.292+0000 7f4423a00700 1 -- 172.21.15.31:0/2797449675 wait complete. 2024-04-25T01:08:20.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.293+0000 7f4423a00700 1 Processor -- start 2024-04-25T01:08:20.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.294+0000 7f4423a00700 1 -- start start 2024-04-25T01:08:20.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.294+0000 7f4423a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f441c1022a0 0x7f441c075fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:20.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.294+0000 7f4423a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f441c1034a0 0x7f441c074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:20.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.294+0000 7f4423a00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f441c103e40 0x7f441c074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:20.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.294+0000 7f4423a00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f441c10dee0 con 0x7f441c1022a0 2024-04-25T01:08:20.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.294+0000 7f4423a00700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f441c10dd60 con 0x7f441c103e40 2024-04-25T01:08:20.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.294+0000 7f4423a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f441c10e060 con 0x7f441c1034a0 2024-04-25T01:08:20.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.294+0000 7f4421000700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f441c1022a0 0x7f441c075fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:20.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.294+0000 7f441be00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f441c1034a0 0x7f441c074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:20.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.294+0000 7f441be00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f441c1034a0 0x7f441c074620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:46380/0 (socket says 172.21.15.31:46380) 2024-04-25T01:08:20.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.294+0000 7f4421000700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f441c1022a0 0x7f441c075fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58086/0 (socket says 172.21.15.31:58086) 2024-04-25T01:08:20.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.294+0000 7f441be00700 1 -- 172.21.15.31:0/911159022 learned_addr learned my addr 172.21.15.31:0/911159022 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:08:20.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.294+0000 7f4421a00700 1 --2- 172.21.15.31:0/911159022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f441c103e40 0x7f441c074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:20.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.295+0000 7f441be00700 1 -- 172.21.15.31:0/911159022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f441c103e40 msgr2=0x7f441c074b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:20.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.295+0000 7f441be00700 1 --2- 172.21.15.31:0/911159022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f441c103e40 0x7f441c074b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:20.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.295+0000 7f441be00700 1 -- 172.21.15.31:0/911159022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f441c1022a0 msgr2=0x7f441c075fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:20.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.295+0000 7f441be00700 1 --2- 172.21.15.31:0/911159022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f441c1022a0 0x7f441c075fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:20.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.295+0000 7f441be00700 1 -- 172.21.15.31:0/911159022 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f441c075410 con 0x7f441c1034a0 2024-04-25T01:08:20.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.295+0000 7f441be00700 1 --2- 172.21.15.31:0/911159022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f441c1034a0 0x7f441c074620 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f440800e9f0 tx=0x7f440800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:20.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.295+0000 7f4419600700 1 -- 172.21.15.31:0/911159022 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4408004a30 con 0x7f441c1034a0 2024-04-25T01:08:20.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.296+0000 7f4423a00700 1 -- 172.21.15.31:0/911159022 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f441c1a2740 con 0x7f441c1034a0 2024-04-25T01:08:20.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.296+0000 7f4419600700 1 -- 172.21.15.31:0/911159022 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f440800cca0 con 0x7f441c1034a0 2024-04-25T01:08:20.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.296+0000 7f4419600700 1 -- 172.21.15.31:0/911159022 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44080056c0 con 0x7f441c1034a0 2024-04-25T01:08:20.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.296+0000 7f4423a00700 1 -- 172.21.15.31:0/911159022 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f441c1a2c00 con 0x7f441c1034a0 2024-04-25T01:08:20.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.297+0000 7f4423a00700 1 -- 172.21.15.31:0/911159022 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f441c068f20 con 0x7f441c1034a0 2024-04-25T01:08:20.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.299+0000 7f4419600700 1 -- 172.21.15.31:0/911159022 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f440800c7c0 con 0x7f441c1034a0 2024-04-25T01:08:20.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.300+0000 7f4419600700 1 --2- 172.21.15.31:0/911159022 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f44040a2c10 0x7f44040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:20.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.301+0000 7f4419600700 1 -- 172.21.15.31:0/911159022 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f44080cdd70 con 0x7f441c1034a0 2024-04-25T01:08:20.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.301+0000 7f4421000700 1 --2- 172.21.15.31:0/911159022 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f44040a2c10 0x7f44040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:20.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.301+0000 7f4421000700 1 --2- 172.21.15.31:0/911159022 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f44040a2c10 0x7f44040a50d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f44100098c0 tx=0x7f4410019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:20.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.303+0000 7f4419600700 1 -- 172.21.15.31:0/911159022 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f440809c0e0 con 0x7f441c1034a0 2024-04-25T01:08:20.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.539+0000 7f4423a00700 1 -- 172.21.15.31:0/911159022 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f441c062670 con 0x7f44040a2c10 2024-04-25T01:08:20.713 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 01:08:20 smithi031 sudo[46552]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T01:08:20.713 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 01:08:20 smithi031 sudo[46552]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:08:20.713 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 01:08:20 smithi031 sudo[46552]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:08:20.713 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 01:08:20 smithi031 sudo[46552]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:08:20.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:20.734+0000 7f4419600700 1 -- 172.21.15.31:0/911159022 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4408098050 con 0x7f441c1034a0 2024-04-25T01:08:21.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:20 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T01:08:21.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:20 smithi031 ceph-mon[28053]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T01:08:21.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:20 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:08:21.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:20 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:08:21.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:20 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:08:21.200 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:20 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T01:08:21.200 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:20 smithi169 ceph-mon[32759]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T01:08:21.200 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:20 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:08:21.200 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:20 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:08:21.200 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:20 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:08:21.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:20 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T01:08:21.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:20 smithi040 ceph-mon[32518]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T01:08:21.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:20 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:08:21.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:20 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:08:21.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:20 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:08:21.470 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 01:08:21 smithi031 sudo[46622]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T01:08:21.470 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 01:08:21 smithi031 sudo[46622]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:08:21.470 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 01:08:21 smithi031 sudo[46622]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:08:21.470 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 01:08:21 smithi031 sudo[46622]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:08:21.953 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:08:21 smithi040 sudo[40852]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T01:08:21.953 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:08:21 smithi040 sudo[40852]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:08:21.953 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:08:21 smithi040 sudo[40852]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:08:21.953 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:08:21 smithi040 sudo[40852]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:08:21.954 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:21 smithi040 ceph-mon[32518]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:21.954 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:21 smithi040 ceph-mon[32518]: from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:08:21.954 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:21 smithi040 ceph-mon[32518]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T01:08:21.954 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:21 smithi040 ceph-mon[32518]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T01:08:21.954 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:21 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:22.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:21 smithi031 ceph-mon[28053]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:22.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:21 smithi031 ceph-mon[28053]: from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:08:22.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:21 smithi031 ceph-mon[28053]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T01:08:22.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:21 smithi031 ceph-mon[28053]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T01:08:22.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:21 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:22.200 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:21 smithi169 ceph-mon[32759]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:22.200 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:21 smithi169 ceph-mon[32759]: from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:08:22.200 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:21 smithi169 ceph-mon[32759]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T01:08:22.200 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:21 smithi169 ceph-mon[32759]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T01:08:22.200 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:21 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:22.453 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:08:22 smithi040 sudo[40855]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T01:08:22.453 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:08:22 smithi040 sudo[40855]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:08:22.453 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:08:22 smithi040 sudo[40855]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:08:22.454 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:08:22 smithi040 sudo[40855]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:08:22.569 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:08:22 smithi169 sudo[41538]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T01:08:22.569 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:08:22 smithi169 sudo[41538]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:08:22.569 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:08:22 smithi169 sudo[41538]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:08:22.569 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:08:22 smithi169 sudo[41538]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:08:22.950 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:08:22 smithi169 sudo[41541]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T01:08:22.950 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:08:22 smithi169 sudo[41541]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:08:22.950 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:08:22 smithi169 sudo[41541]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:08:22.951 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:08:22 smithi169 sudo[41541]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:08:23.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:23 smithi031 sudo[46725]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T01:08:23.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:23 smithi031 sudo[46725]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:08:23.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:23 smithi031 sudo[46725]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:08:23.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:23 smithi031 sudo[46725]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:08:24.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:23 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:24.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:23 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:24.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:23 smithi169 ceph-mon[32759]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T01:08:24.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:24 smithi169 sudo[41544]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T01:08:24.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:24 smithi169 sudo[41544]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:08:24.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:24 smithi169 sudo[41544]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:08:24.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:24 smithi169 sudo[41544]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:08:24.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:23 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:24.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:23 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:24.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:23 smithi040 ceph-mon[32518]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T01:08:24.262 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:23 smithi031 sudo[46746]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T01:08:24.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:23 smithi031 sudo[46746]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:08:24.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:23 smithi031 sudo[46746]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:08:24.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:23 smithi031 sudo[46746]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:08:24.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:23 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:24.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:23 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:24.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:23 smithi031 ceph-mon[28053]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T01:08:24.450 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:24 smithi169 sudo[41547]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T01:08:24.450 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:24 smithi169 sudo[41547]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:08:24.450 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:24 smithi169 sudo[41547]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:08:24.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:24 smithi169 sudo[41547]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:08:25.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:24 smithi040 sudo[40858]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T01:08:25.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:24 smithi040 sudo[40858]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:08:25.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:24 smithi040 sudo[40858]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:08:25.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:24 smithi040 sudo[40858]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:08:25.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:25 smithi031 ceph-mon[28053]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:25.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:25 smithi031 ceph-mon[28053]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T01:08:25.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:08:25.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:08:25.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:08:25.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:25 smithi031 ceph-mon[28053]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T01:08:25.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:08:25.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:08:25.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:08:25.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:25 smithi031 ceph-mon[28053]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T01:08:25.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:08:25.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:08:25.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:08:25.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:25 smithi169 ceph-mon[32759]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:25.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:25 smithi169 ceph-mon[32759]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T01:08:25.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:08:25.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:08:25.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:08:25.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:25 smithi169 ceph-mon[32759]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T01:08:25.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:08:25.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:08:25.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:08:25.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:25 smithi169 ceph-mon[32759]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T01:08:25.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:08:25.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:08:25.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:08:25.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:25 smithi040 sudo[40861]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T01:08:25.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:25 smithi040 sudo[40861]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:08:25.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:25 smithi040 sudo[40861]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:08:25.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:25 smithi040 sudo[40861]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:08:25.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:25 smithi040 ceph-mon[32518]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:25.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:25 smithi040 ceph-mon[32518]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T01:08:25.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:08:25.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:08:25.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:08:25.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:25 smithi040 ceph-mon[32518]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T01:08:25.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:08:25.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:08:25.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:08:25.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:25 smithi040 ceph-mon[32518]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T01:08:25.705 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:08:25.705 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:08:25.705 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:08:26.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:26 smithi031 ceph-mon[28053]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:26.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:26 smithi031 ceph-mon[28053]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T01:08:26.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:26 smithi031 ceph-mon[28053]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T01:08:26.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:26 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2145017431' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b5a7881-3401-495c-8ca4-c90685919d47"}]: dispatch 2024-04-25T01:08:26.511 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:26 smithi031 ceph-mon[28053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b5a7881-3401-495c-8ca4-c90685919d47"}]: dispatch 2024-04-25T01:08:26.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:26 smithi031 ceph-mon[28053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8b5a7881-3401-495c-8ca4-c90685919d47"}]': finished 2024-04-25T01:08:26.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:26 smithi031 ceph-mon[28053]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T01:08:26.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:26 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:08:26.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:26 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2069957278' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:08:26.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:26 smithi169 ceph-mon[32759]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:26.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:26 smithi169 ceph-mon[32759]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T01:08:26.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:26 smithi169 ceph-mon[32759]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T01:08:26.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:26 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2145017431' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b5a7881-3401-495c-8ca4-c90685919d47"}]: dispatch 2024-04-25T01:08:26.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:26 smithi169 ceph-mon[32759]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b5a7881-3401-495c-8ca4-c90685919d47"}]: dispatch 2024-04-25T01:08:26.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:26 smithi169 ceph-mon[32759]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8b5a7881-3401-495c-8ca4-c90685919d47"}]': finished 2024-04-25T01:08:26.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:26 smithi169 ceph-mon[32759]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T01:08:26.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:26 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:08:26.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:26 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2069957278' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:08:26.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:26 smithi040 ceph-mon[32518]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:26.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:26 smithi040 ceph-mon[32518]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T01:08:26.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:26 smithi040 ceph-mon[32518]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T01:08:26.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:26 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2145017431' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b5a7881-3401-495c-8ca4-c90685919d47"}]: dispatch 2024-04-25T01:08:26.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:26 smithi040 ceph-mon[32518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b5a7881-3401-495c-8ca4-c90685919d47"}]: dispatch 2024-04-25T01:08:26.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:26 smithi040 ceph-mon[32518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8b5a7881-3401-495c-8ca4-c90685919d47"}]': finished 2024-04-25T01:08:26.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:26 smithi040 ceph-mon[32518]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T01:08:26.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:26 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:08:26.704 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:26 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2069957278' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:08:27.450 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:27 smithi169 ceph-mon[32759]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:27.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:27 smithi040 ceph-mon[32518]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:27.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:27 smithi031 ceph-mon[28053]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:29.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:29 smithi169 ceph-mon[32759]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:29.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:29 smithi040 ceph-mon[32518]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:29.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:29 smithi031 ceph-mon[28053]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:31.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:31 smithi031 ceph-mon[28053]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-04-25T01:08:31.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:31 smithi169 ceph-mon[32759]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-04-25T01:08:31.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:31 smithi040 ceph-mon[32518]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-04-25T01:08:33.439 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:33 smithi031 ceph-mon[28053]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-04-25T01:08:33.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:33 smithi169 ceph-mon[32759]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-04-25T01:08:33.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:33 smithi040 ceph-mon[32518]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-04-25T01:08:35.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:35 smithi169 ceph-mon[32759]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-04-25T01:08:35.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:35 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T01:08:35.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:35 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:08:35.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:35 smithi040 ceph-mon[32518]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-04-25T01:08:35.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:35 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T01:08:35.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:35 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:08:35.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:35 smithi031 ceph-mon[28053]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-04-25T01:08:35.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:35 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T01:08:35.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:35 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:08:36.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:36 smithi031 ceph-mon[28053]: Deploying daemon osd.3 on smithi031 2024-04-25T01:08:36.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:36 smithi169 ceph-mon[32759]: Deploying daemon osd.3 on smithi031 2024-04-25T01:08:36.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:36 smithi040 ceph-mon[32518]: Deploying daemon osd.3 on smithi031 2024-04-25T01:08:37.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:37 smithi031 ceph-mon[28053]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 861 B/s rd, 24 KiB/s wr, 2 op/s 2024-04-25T01:08:37.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:37 smithi169 ceph-mon[32759]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 861 B/s rd, 24 KiB/s wr, 2 op/s 2024-04-25T01:08:37.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:37 smithi040 ceph-mon[32518]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 861 B/s rd, 24 KiB/s wr, 2 op/s 2024-04-25T01:08:39.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:39 smithi031 ceph-mon[28053]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-04-25T01:08:39.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:39 smithi169 ceph-mon[32759]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-04-25T01:08:39.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:39 smithi040 ceph-mon[32518]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-04-25T01:08:41.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:41 smithi169 ceph-mon[32759]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-04-25T01:08:41.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:41 smithi040 ceph-mon[32518]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-04-25T01:08:41.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:41 smithi031 ceph-mon[28053]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-04-25T01:08:43.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:43 smithi031 ceph-mon[28053]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:43.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:43 smithi169 ceph-mon[32759]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:43.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:43 smithi040 ceph-mon[32518]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:44.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:44 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:44.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:44 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:08:44.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:44 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:44.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:44 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:44.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:44 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:08:44.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:44 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:44.953 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:44 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:44.953 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:44 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:08:44.953 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:44 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:45.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:45 smithi031 ceph-mon[28053]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:45.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:45 smithi169 ceph-mon[32759]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:45.952 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:45 smithi040 ceph-mon[32518]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:47.530 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:47 smithi031 ceph-mon[28053]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:47.531 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:47 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:47.531 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:47 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:47.531 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:47 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:08:47.531 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:47 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:08:47.531 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:47 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:47.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:47 smithi169 ceph-mon[32759]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:47.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:47 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:47.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:47 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:47.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:47 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:08:47.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:47 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:08:47.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:47 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:47.952 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:47 smithi040 ceph-mon[32518]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:47.952 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:47 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:47.953 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:47 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:47.953 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:47 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:08:47.953 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:47 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:08:47.953 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:47 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:48.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:48.396+0000 7f4419600700 1 -- 172.21.15.31:0/911159022 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f441c062670 con 0x7f44040a2c10 2024-04-25T01:08:48.387 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 3 on host 'smithi031' 2024-04-25T01:08:48.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:48.400+0000 7f4423a00700 1 -- 172.21.15.31:0/911159022 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f44040a2c10 msgr2=0x7f44040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:48.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:48.400+0000 7f4423a00700 1 --2- 172.21.15.31:0/911159022 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f44040a2c10 0x7f44040a50d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f44100098c0 tx=0x7f4410019040 comp rx=0 tx=0).stop 2024-04-25T01:08:48.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:48.400+0000 7f4423a00700 1 -- 172.21.15.31:0/911159022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f441c1034a0 msgr2=0x7f441c074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:48.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:48.400+0000 7f4423a00700 1 --2- 172.21.15.31:0/911159022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f441c1034a0 0x7f441c074620 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f440800e9f0 tx=0x7f440800ed00 comp rx=0 tx=0).stop 2024-04-25T01:08:48.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:48.400+0000 7f4423a00700 1 -- 172.21.15.31:0/911159022 shutdown_connections 2024-04-25T01:08:48.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:48.400+0000 7f4423a00700 1 --2- 172.21.15.31:0/911159022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f441c1022a0 0x7f441c075fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:48.400+0000 7f4423a00700 1 --2- 172.21.15.31:0/911159022 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f44040a2c10 0x7f44040a50d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:48.400+0000 7f4423a00700 1 --2- 172.21.15.31:0/911159022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f441c103e40 0x7f441c074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:48.400+0000 7f4423a00700 1 --2- 172.21.15.31:0/911159022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f441c1034a0 0x7f441c074620 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:48.400+0000 7f4423a00700 1 -- 172.21.15.31:0/911159022 >> 172.21.15.31:0/911159022 conn(0x7f441c0fd800 msgr2=0x7f441c106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:48.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:48.400+0000 7f4423a00700 1 -- 172.21.15.31:0/911159022 shutdown_connections 2024-04-25T01:08:48.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:08:48.400+0000 7f4423a00700 1 -- 172.21.15.31:0/911159022 wait complete. 2024-04-25T01:08:48.741 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:48 smithi031 ceph-mon[28053]: from='osd.3 [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T01:08:48.741 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:48 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:48.741 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:48 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:48.741 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:48 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:08:48.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:48 smithi169 ceph-mon[32759]: from='osd.3 [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T01:08:48.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:48 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:48.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:48 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:48.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:48 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:08:48.952 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:48 smithi040 ceph-mon[32518]: from='osd.3 [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T01:08:48.953 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:48 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:48.953 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:48 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:48.953 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:48 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:08:49.118 DEBUG:teuthology.orchestra.run.smithi031:osd.3> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T01:08:49.126 INFO:tasks.cephadm:Deploying osd.4 on smithi040 with /dev/vg_nvme/lv_3... 2024-04-25T01:08:49.126 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T01:08:49.380 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T01:08:49.752 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:49 smithi040 ceph-mon[32518]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:49.753 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:49 smithi040 ceph-mon[32518]: from='osd.3 [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T01:08:49.753 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:49 smithi040 ceph-mon[32518]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T01:08:49.753 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:49 smithi040 ceph-mon[32518]: from='osd.3 [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T01:08:49.753 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:49 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:08:49.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:49 smithi031 ceph-mon[28053]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:49.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:49 smithi031 ceph-mon[28053]: from='osd.3 [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T01:08:49.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:49 smithi031 ceph-mon[28053]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T01:08:49.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:49 smithi031 ceph-mon[28053]: from='osd.3 [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T01:08:49.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:49 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:08:49.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:49 smithi169 ceph-mon[32759]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:49.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:49 smithi169 ceph-mon[32759]: from='osd.3 [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T01:08:49.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:49 smithi169 ceph-mon[32759]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T01:08:49.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:49 smithi169 ceph-mon[32759]: from='osd.3 [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T01:08:49.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:49 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:08:50.618 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:50 smithi040 ceph-mon[32518]: purged_snaps scrub starts 2024-04-25T01:08:50.619 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:50 smithi040 ceph-mon[32518]: purged_snaps scrub ok 2024-04-25T01:08:50.619 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:50 smithi040 ceph-mon[32518]: from='osd.3 [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T01:08:50.619 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:50 smithi040 ceph-mon[32518]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T01:08:50.619 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:50 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:08:50.619 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:50 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:08:50.619 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:50 smithi040 ceph-mon[32518]: osdmap e26: 4 total, 3 up, 4 in 2024-04-25T01:08:50.619 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:50 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:08:50.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:50 smithi031 ceph-mon[28053]: purged_snaps scrub starts 2024-04-25T01:08:50.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:50 smithi031 ceph-mon[28053]: purged_snaps scrub ok 2024-04-25T01:08:50.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:50 smithi031 ceph-mon[28053]: from='osd.3 [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T01:08:50.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:50 smithi031 ceph-mon[28053]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T01:08:50.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:50 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:08:50.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:50 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:08:50.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:50 smithi031 ceph-mon[28053]: osdmap e26: 4 total, 3 up, 4 in 2024-04-25T01:08:50.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:50 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:08:50.758 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:08:50 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-3[48821]: 2024-04-25T01:08:50.651+0000 7f7e20a00700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T01:08:50.758 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:08:50 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-3[48821]: 2024-04-25T01:08:50.675+0000 7f7e17200700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:08:50.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:50 smithi169 ceph-mon[32759]: purged_snaps scrub starts 2024-04-25T01:08:50.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:50 smithi169 ceph-mon[32759]: purged_snaps scrub ok 2024-04-25T01:08:50.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:50 smithi169 ceph-mon[32759]: from='osd.3 [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T01:08:50.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:50 smithi169 ceph-mon[32759]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T01:08:50.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:50 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:08:50.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:50 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:08:50.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:50 smithi169 ceph-mon[32759]: osdmap e26: 4 total, 3 up, 4 in 2024-04-25T01:08:50.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:50 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:08:51.211 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T01:08:51.211 INFO:teuthology.orchestra.run.smithi040.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T01:08:51.211 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10+0 records in 2024-04-25T01:08:51.212 INFO:teuthology.orchestra.run.smithi040.stderr:10+0 records out 2024-04-25T01:08:51.212 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111121 s, 944 MB/s 2024-04-25T01:08:51.212 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping successful for: 2024-04-25T01:08:51.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:51 smithi031 ceph-mon[28053]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:51.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:51 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:08:51.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:51 smithi031 ceph-mon[28053]: from='osd.3 [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643]' entity='osd.3' 2024-04-25T01:08:51.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:51 smithi031 ceph-mon[28053]: osd.3 [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643] boot 2024-04-25T01:08:51.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:51 smithi031 ceph-mon[28053]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T01:08:51.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:51 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:08:51.929 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:51 smithi040 ceph-mon[32518]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:51.929 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:51 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:08:51.929 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:51 smithi040 ceph-mon[32518]: from='osd.3 [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643]' entity='osd.3' 2024-04-25T01:08:51.929 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:51 smithi040 ceph-mon[32518]: osd.3 [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643] boot 2024-04-25T01:08:51.929 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:51 smithi040 ceph-mon[32518]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T01:08:51.929 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:51 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:08:51.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:51 smithi169 ceph-mon[32759]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:08:51.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:51 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:08:51.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:51 smithi169 ceph-mon[32759]: from='osd.3 [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643]' entity='osd.3' 2024-04-25T01:08:51.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:51 smithi169 ceph-mon[32759]: osd.3 [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643] boot 2024-04-25T01:08:51.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:51 smithi169 ceph-mon[32759]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T01:08:51.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:51 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:08:51.957 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi040:vg_nvme/lv_3 2024-04-25T01:08:52.212 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T01:08:53.693 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:53 smithi040 ceph-mon[32518]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:08:53.694 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:53 smithi040 ceph-mon[32518]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T01:08:53.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:53 smithi169 ceph-mon[32759]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:08:53.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:53 smithi169 ceph-mon[32759]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T01:08:53.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:53 smithi031 ceph-mon[28053]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:08:53.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:53 smithi031 ceph-mon[28053]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T01:08:53.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.702+0000 7fc0fa692700 1 -- 172.21.15.40:0/297720739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f41056e0 msgr2=0x7fc0f4107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:53.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.702+0000 7fc0fa692700 1 --2- 172.21.15.40:0/297720739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f41056e0 0x7fc0f4107ad0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009990 tx=0x7fc0e8009ca0 comp rx=0 tx=0).stop 2024-04-25T01:08:53.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.702+0000 7fc0fa692700 1 -- 172.21.15.40:0/297720739 shutdown_connections 2024-04-25T01:08:53.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.702+0000 7fc0fa692700 1 --2- 172.21.15.40:0/297720739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0f4108010 0x7fc0f410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:53.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.702+0000 7fc0fa692700 1 --2- 172.21.15.40:0/297720739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0f4102db0 0x7fc0f41051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.702+0000 7fc0fa692700 1 --2- 172.21.15.40:0/297720739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f41056e0 0x7fc0f4107ad0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.702+0000 7fc0fa692700 1 -- 172.21.15.40:0/297720739 >> 172.21.15.40:0/297720739 conn(0x7fc0f40fc720 msgr2=0x7fc0f40feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.702+0000 7fc0fa692700 1 -- 172.21.15.40:0/297720739 shutdown_connections 2024-04-25T01:08:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.702+0000 7fc0fa692700 1 -- 172.21.15.40:0/297720739 wait complete. 2024-04-25T01:08:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.703+0000 7fc0fa692700 1 Processor -- start 2024-04-25T01:08:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.703+0000 7fc0fa692700 1 -- start start 2024-04-25T01:08:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.703+0000 7fc0fa692700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0f4102db0 0x7fc0f41985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.703+0000 7fc0fa692700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0f41056e0 0x7fc0f4198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.703+0000 7fc0fa692700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f4108010 0x7fc0f419fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.703+0000 7fc0fa692700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f410df50 con 0x7fc0f41056e0 2024-04-25T01:08:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.703+0000 7fc0fa692700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f410ddd0 con 0x7fc0f4108010 2024-04-25T01:08:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.703+0000 7fc0fa692700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f410e0d0 con 0x7fc0f4102db0 2024-04-25T01:08:53.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.703+0000 7fc0f8c2f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f4108010 0x7fc0f419fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.704+0000 7fc0f8c2f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f4108010 0x7fc0f419fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53998/0 (socket says 172.21.15.40:53998) 2024-04-25T01:08:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.704+0000 7fc0f8c2f700 1 -- 172.21.15.40:0/248658200 learned_addr learned my addr 172.21.15.40:0/248658200 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T01:08:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.704+0000 7fc0f3fff700 1 --2- 172.21.15.40:0/248658200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0f4102db0 0x7fc0f41985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.704+0000 7fc0f8c2f700 1 -- 172.21.15.40:0/248658200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0f4102db0 msgr2=0x7fc0f41985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.704+0000 7fc0f8c2f700 1 --2- 172.21.15.40:0/248658200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0f4102db0 0x7fc0f41985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.704+0000 7fc0f8c2f700 1 -- 172.21.15.40:0/248658200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0f41056e0 msgr2=0x7fc0f4198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.704+0000 7fc0f8c2f700 1 --2- 172.21.15.40:0/248658200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0f41056e0 0x7fc0f4198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.704+0000 7fc0f8c2f700 1 -- 172.21.15.40:0/248658200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f41a0220 con 0x7fc0f4108010 2024-04-25T01:08:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.704+0000 7fc0f8c2f700 1 --2- 172.21.15.40:0/248658200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f4108010 0x7fc0f419fb80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00b9e0 tx=0x7fc0ec00bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.704+0000 7fc0f17fa700 1 -- 172.21.15.40:0/248658200 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec019070 con 0x7fc0f4108010 2024-04-25T01:08:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.704+0000 7fc0f17fa700 1 -- 172.21.15.40:0/248658200 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0ec014440 con 0x7fc0f4108010 2024-04-25T01:08:53.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.704+0000 7fc0fa692700 1 -- 172.21.15.40:0/248658200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f41a0510 con 0x7fc0f4108010 2024-04-25T01:08:53.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.704+0000 7fc0f17fa700 1 -- 172.21.15.40:0/248658200 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec0125a0 con 0x7fc0f4108010 2024-04-25T01:08:53.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.704+0000 7fc0fa692700 1 -- 172.21.15.40:0/248658200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f41a0ad0 con 0x7fc0f4108010 2024-04-25T01:08:53.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.705+0000 7fc0fa692700 1 -- 172.21.15.40:0/248658200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0f4068f20 con 0x7fc0f4108010 2024-04-25T01:08:53.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.706+0000 7fc0f17fa700 1 -- 172.21.15.40:0/248658200 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc0ec01c070 con 0x7fc0f4108010 2024-04-25T01:08:53.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.707+0000 7fc0f17fa700 1 --2- 172.21.15.40:0/248658200 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc0dc0a2c20 0x7fc0dc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:53.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.707+0000 7fc0f17fa700 1 -- 172.21.15.40:0/248658200 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3698+0+0 (secure 0 0 0) 0x7fc0ec0cd220 con 0x7fc0f4108010 2024-04-25T01:08:53.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.708+0000 7fc0f3fff700 1 --2- 172.21.15.40:0/248658200 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc0dc0a2c20 0x7fc0dc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:53.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.709+0000 7fc0f3fff700 1 --2- 172.21.15.40:0/248658200 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc0dc0a2c20 0x7fc0dc0a50e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4006fd0 tx=0x7fc0e4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:53.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.712+0000 7fc0f17fa700 1 -- 172.21.15.40:0/248658200 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc0ec09b380 con 0x7fc0f4108010 2024-04-25T01:08:53.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:08:53.926+0000 7fc0fa692700 1 -- 172.21.15.40:0/248658200 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc0f4062670 con 0x7fc0dc0a2c20 2024-04-25T01:08:54.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:54 smithi169 ceph-mon[32759]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T01:08:54.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:54 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:08:54.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:54 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:08:54.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:54 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:08:54.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:54 smithi040 ceph-mon[32518]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T01:08:54.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:54 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:08:54.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:54 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:08:54.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:54 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:08:54.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:54 smithi031 ceph-mon[28053]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T01:08:54.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:54 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:08:54.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:54 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:08:54.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:54 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:08:55.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:55 smithi169 ceph-mon[32759]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:08:55.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:55 smithi169 ceph-mon[32759]: pgmap v120: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T01:08:55.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:55 smithi169 ceph-mon[32759]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T01:08:55.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:55 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:55.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:55 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:55.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:55 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:08:55.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:55 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:08:55.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:55 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:55.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:55 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:08:55.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:55 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:08:55.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:55 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:55.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:55 smithi040 ceph-mon[32518]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:08:55.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:55 smithi040 ceph-mon[32518]: pgmap v120: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T01:08:55.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:55 smithi040 ceph-mon[32518]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T01:08:55.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:55 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:55.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:55 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:55.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:55 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:08:55.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:55 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:08:55.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:55 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:55.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:55 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:08:55.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:55 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:08:55.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:55 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:55.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:55 smithi031 ceph-mon[28053]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:08:55.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:55 smithi031 ceph-mon[28053]: pgmap v120: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T01:08:55.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:55 smithi031 ceph-mon[28053]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T01:08:55.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:55 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:55.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:55 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:55.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:55 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:08:55.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:55 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:08:55.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:55 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:55.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:55 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:08:55.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:55 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:08:55.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:55 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:08:56.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:56 smithi169 ceph-mon[32759]: Detected new or changed devices on smithi031 2024-04-25T01:08:56.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:56 smithi169 ceph-mon[32759]: Adjusting osd_memory_target on smithi031 to 8547M 2024-04-25T01:08:56.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:56 smithi169 ceph-mon[32759]: osdmap e31: 4 total, 4 up, 4 in 2024-04-25T01:08:56.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:56 smithi040 ceph-mon[32518]: Detected new or changed devices on smithi031 2024-04-25T01:08:56.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:56 smithi040 ceph-mon[32518]: Adjusting osd_memory_target on smithi031 to 8547M 2024-04-25T01:08:56.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:56 smithi040 ceph-mon[32518]: osdmap e31: 4 total, 4 up, 4 in 2024-04-25T01:08:56.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:56 smithi031 ceph-mon[28053]: Detected new or changed devices on smithi031 2024-04-25T01:08:56.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:56 smithi031 ceph-mon[28053]: Adjusting osd_memory_target on smithi031 to 8547M 2024-04-25T01:08:56.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:56 smithi031 ceph-mon[28053]: osdmap e31: 4 total, 4 up, 4 in 2024-04-25T01:08:57.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:57 smithi169 ceph-mon[32759]: pgmap v123: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T01:08:57.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:57 smithi169 ceph-mon[32759]: from='client.? 172.21.15.40:0/3454864960' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56deb61d-a334-4cf2-8ac7-fbb6e6947092"}]: dispatch 2024-04-25T01:08:57.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:57 smithi169 ceph-mon[32759]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56deb61d-a334-4cf2-8ac7-fbb6e6947092"}]: dispatch 2024-04-25T01:08:57.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:57 smithi169 ceph-mon[32759]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56deb61d-a334-4cf2-8ac7-fbb6e6947092"}]': finished 2024-04-25T01:08:57.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:57 smithi169 ceph-mon[32759]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T01:08:57.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:57 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:08:57.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:57 smithi169 ceph-mon[32759]: from='client.? 172.21.15.40:0/3681778880' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:08:57.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:57 smithi040 ceph-mon[32518]: pgmap v123: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T01:08:57.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:57 smithi040 ceph-mon[32518]: from='client.? 172.21.15.40:0/3454864960' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56deb61d-a334-4cf2-8ac7-fbb6e6947092"}]: dispatch 2024-04-25T01:08:57.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:57 smithi040 ceph-mon[32518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56deb61d-a334-4cf2-8ac7-fbb6e6947092"}]: dispatch 2024-04-25T01:08:57.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:57 smithi040 ceph-mon[32518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56deb61d-a334-4cf2-8ac7-fbb6e6947092"}]': finished 2024-04-25T01:08:57.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:57 smithi040 ceph-mon[32518]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T01:08:57.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:57 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:08:57.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:57 smithi040 ceph-mon[32518]: from='client.? 172.21.15.40:0/3681778880' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:08:57.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:57 smithi031 ceph-mon[28053]: pgmap v123: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T01:08:57.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:57 smithi031 ceph-mon[28053]: from='client.? 172.21.15.40:0/3454864960' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56deb61d-a334-4cf2-8ac7-fbb6e6947092"}]: dispatch 2024-04-25T01:08:57.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:57 smithi031 ceph-mon[28053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56deb61d-a334-4cf2-8ac7-fbb6e6947092"}]: dispatch 2024-04-25T01:08:57.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:57 smithi031 ceph-mon[28053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56deb61d-a334-4cf2-8ac7-fbb6e6947092"}]': finished 2024-04-25T01:08:57.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:57 smithi031 ceph-mon[28053]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T01:08:57.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:57 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:08:57.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:57 smithi031 ceph-mon[28053]: from='client.? 172.21.15.40:0/3681778880' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:08:59.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:08:59 smithi031 ceph-mon[28053]: pgmap v125: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-04-25T01:08:59.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:08:59 smithi169 ceph-mon[32759]: pgmap v125: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-04-25T01:08:59.952 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:08:59 smithi040 ceph-mon[32518]: pgmap v125: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-04-25T01:09:01.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:01 smithi031 ceph-mon[28053]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:01.916 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:01 smithi040 ceph-mon[32518]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:01.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:01 smithi169 ceph-mon[32759]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:03.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:03 smithi031 ceph-mon[28053]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:03.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:03 smithi169 ceph-mon[32759]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:03.952 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:03 smithi040 ceph-mon[32518]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:04.756 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:04 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T01:09:04.756 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:04 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:04.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:04 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T01:09:04.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:04 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:04.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:04 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T01:09:04.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:04 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:05.653 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:05 smithi040 ceph-mon[32518]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:05.654 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:05 smithi040 ceph-mon[32518]: Deploying daemon osd.4 on smithi040 2024-04-25T01:09:05.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:05 smithi031 ceph-mon[28053]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:05.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:05 smithi031 ceph-mon[28053]: Deploying daemon osd.4 on smithi040 2024-04-25T01:09:05.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:05 smithi169 ceph-mon[32759]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:05.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:05 smithi169 ceph-mon[32759]: Deploying daemon osd.4 on smithi040 2024-04-25T01:09:07.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:07 smithi031 ceph-mon[28053]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:07.798 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:07 smithi040 ceph-mon[32518]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:07.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:07 smithi169 ceph-mon[32759]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:09.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:09 smithi040 ceph-mon[32518]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:09.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:09 smithi031 ceph-mon[28053]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:09.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:09 smithi169 ceph-mon[32759]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:10.869 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:10 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:10.869 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:10 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:10.869 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:10 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:09:10.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:10 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:10.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:10 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:10.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:10 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:09:11.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:10 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:11.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:10 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:11.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:10 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:09:11.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:11 smithi040 ceph-mon[32518]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:11.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:11 smithi169 ceph-mon[32759]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:12.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:11 smithi031 ceph-mon[28053]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:12.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:09:12.806+0000 7fc0f17fa700 1 -- 172.21.15.40:0/248658200 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc0f4062670 con 0x7fc0dc0a2c20 2024-04-25T01:09:12.798 INFO:teuthology.orchestra.run.smithi040.stdout:Created osd(s) 4 on host 'smithi040' 2024-04-25T01:09:12.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:09:12.809+0000 7fc0fa692700 1 -- 172.21.15.40:0/248658200 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc0dc0a2c20 msgr2=0x7fc0dc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:09:12.809+0000 7fc0fa692700 1 --2- 172.21.15.40:0/248658200 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc0dc0a2c20 0x7fc0dc0a50e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4006fd0 tx=0x7fc0e4008040 comp rx=0 tx=0).stop 2024-04-25T01:09:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:09:12.809+0000 7fc0fa692700 1 -- 172.21.15.40:0/248658200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f4108010 msgr2=0x7fc0f419fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:09:12.809+0000 7fc0fa692700 1 --2- 172.21.15.40:0/248658200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f4108010 0x7fc0f419fb80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00b9e0 tx=0x7fc0ec00bda0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:09:12.810+0000 7fc0fa692700 1 -- 172.21.15.40:0/248658200 shutdown_connections 2024-04-25T01:09:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:09:12.810+0000 7fc0fa692700 1 --2- 172.21.15.40:0/248658200 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc0dc0a2c20 0x7fc0dc0a50e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:09:12.810+0000 7fc0fa692700 1 --2- 172.21.15.40:0/248658200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0f4108010 0x7fc0f419fb80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:09:12.810+0000 7fc0fa692700 1 --2- 172.21.15.40:0/248658200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0f4102db0 0x7fc0f41985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:09:12.810+0000 7fc0fa692700 1 --2- 172.21.15.40:0/248658200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0f41056e0 0x7fc0f4198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:09:12.810+0000 7fc0fa692700 1 -- 172.21.15.40:0/248658200 >> 172.21.15.40:0/248658200 conn(0x7fc0f40fc720 msgr2=0x7fc0f4103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:09:12.810+0000 7fc0fa692700 1 -- 172.21.15.40:0/248658200 shutdown_connections 2024-04-25T01:09:12.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:09:12.810+0000 7fc0fa692700 1 -- 172.21.15.40:0/248658200 wait complete. 2024-04-25T01:09:13.395 DEBUG:teuthology.orchestra.run.smithi040:osd.4> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T01:09:13.397 INFO:tasks.cephadm:Deploying osd.5 on smithi169 with /dev/vg_nvme/lv_3... 2024-04-25T01:09:13.397 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T01:09:13.644 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:09:13.702 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:09:13 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4[43687]: 2024-04-25T01:09:13.447+0000 7f8c09d577c0 -1 osd.4 0 log_to_monitors true 2024-04-25T01:09:14.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:13 smithi031 ceph-mon[28053]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:14.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:14.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:14.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:13 smithi031 ceph-mon[28053]: from='osd.4 [v2:172.21.15.40:6808/1913779091,v1:172.21.15.40:6809/1913779091]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T01:09:14.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:13 smithi031 ceph-mon[28053]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T01:09:14.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:14.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:14.019 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:13 smithi169 ceph-mon[32759]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:14.019 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:14.019 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:14.019 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:13 smithi169 ceph-mon[32759]: from='osd.4 [v2:172.21.15.40:6808/1913779091,v1:172.21.15.40:6809/1913779091]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T01:09:14.019 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:13 smithi169 ceph-mon[32759]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T01:09:14.019 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:14.019 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:14.166 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:13 smithi040 ceph-mon[32518]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:14.167 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:14.167 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:14.167 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:13 smithi040 ceph-mon[32518]: from='osd.4 [v2:172.21.15.40:6808/1913779091,v1:172.21.15.40:6809/1913779091]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T01:09:14.167 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:13 smithi040 ceph-mon[32518]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T01:09:14.167 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:14.167 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:14.849 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:14 smithi169 ceph-mon[32759]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T01:09:14.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:14 smithi169 ceph-mon[32759]: from='osd.4 [v2:172.21.15.40:6808/1913779091,v1:172.21.15.40:6809/1913779091]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T01:09:14.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:14 smithi169 ceph-mon[32759]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T01:09:14.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:14 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:09:14.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:14 smithi169 ceph-mon[32759]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T01:09:15.015 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:09:14 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4[43687]: 2024-04-25T01:09:14.773+0000 7f8bfebbb700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T01:09:15.015 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:09:14 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4[43687]: 2024-04-25T01:09:14.780+0000 7f8bfb1af700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:09:15.015 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:14 smithi040 ceph-mon[32518]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T01:09:15.015 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:14 smithi040 ceph-mon[32518]: from='osd.4 [v2:172.21.15.40:6808/1913779091,v1:172.21.15.40:6809/1913779091]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T01:09:15.015 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:14 smithi040 ceph-mon[32518]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T01:09:15.015 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:14 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:09:15.015 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:14 smithi040 ceph-mon[32518]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T01:09:15.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:14 smithi031 ceph-mon[28053]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T01:09:15.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:14 smithi031 ceph-mon[28053]: from='osd.4 [v2:172.21.15.40:6808/1913779091,v1:172.21.15.40:6809/1913779091]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T01:09:15.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:14 smithi031 ceph-mon[28053]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T01:09:15.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:14 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:09:15.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:14 smithi031 ceph-mon[28053]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T01:09:15.416 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T01:09:15.416 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T01:09:15.416 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-04-25T01:09:15.417 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-04-25T01:09:15.417 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115426 s, 908 MB/s 2024-04-25T01:09:15.417 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-04-25T01:09:15.990 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_3 2024-04-25T01:09:16.100 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:15 smithi040 ceph-mon[32518]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:16.100 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:15 smithi040 ceph-mon[32518]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-25T01:09:16.100 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:15 smithi040 ceph-mon[32518]: osdmap e34: 5 total, 4 up, 5 in 2024-04-25T01:09:16.100 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:15 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:09:16.100 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:15 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:09:16.199 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:15 smithi169 ceph-mon[32759]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:16.199 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:15 smithi169 ceph-mon[32759]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-25T01:09:16.199 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:15 smithi169 ceph-mon[32759]: osdmap e34: 5 total, 4 up, 5 in 2024-04-25T01:09:16.199 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:15 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:09:16.199 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:15 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:09:16.231 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:09:16.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:15 smithi031 ceph-mon[28053]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:16.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:15 smithi031 ceph-mon[28053]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-25T01:09:16.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:15 smithi031 ceph-mon[28053]: osdmap e34: 5 total, 4 up, 5 in 2024-04-25T01:09:16.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:15 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:09:16.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:15 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:09:16.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:16 smithi169 ceph-mon[32759]: purged_snaps scrub starts 2024-04-25T01:09:16.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:16 smithi169 ceph-mon[32759]: purged_snaps scrub ok 2024-04-25T01:09:16.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:16 smithi169 ceph-mon[32759]: osd.4 [v2:172.21.15.40:6808/1913779091,v1:172.21.15.40:6809/1913779091] boot 2024-04-25T01:09:16.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:16 smithi169 ceph-mon[32759]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T01:09:16.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:16 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:09:16.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:16 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:16.952 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:16 smithi040 ceph-mon[32518]: purged_snaps scrub starts 2024-04-25T01:09:16.952 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:16 smithi040 ceph-mon[32518]: purged_snaps scrub ok 2024-04-25T01:09:16.952 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:16 smithi040 ceph-mon[32518]: osd.4 [v2:172.21.15.40:6808/1913779091,v1:172.21.15.40:6809/1913779091] boot 2024-04-25T01:09:16.952 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:16 smithi040 ceph-mon[32518]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T01:09:16.952 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:16 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:09:16.952 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:16 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:17.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:16 smithi031 ceph-mon[28053]: purged_snaps scrub starts 2024-04-25T01:09:17.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:16 smithi031 ceph-mon[28053]: purged_snaps scrub ok 2024-04-25T01:09:17.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:16 smithi031 ceph-mon[28053]: osd.4 [v2:172.21.15.40:6808/1913779091,v1:172.21.15.40:6809/1913779091] boot 2024-04-25T01:09:17.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:16 smithi031 ceph-mon[28053]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T01:09:17.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:16 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:09:17.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:16 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:17.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.660+0000 7f96a5769700 1 -- 172.21.15.169:0/244026263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96a01022c0 msgr2=0x7f96a01026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:17.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.660+0000 7f96a5769700 1 --2- 172.21.15.169:0/244026263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96a01022c0 0x7f96a01026e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9690009990 tx=0x7f9690009ca0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.660+0000 7f96a5769700 1 -- 172.21.15.169:0/244026263 shutdown_connections 2024-04-25T01:09:17.654 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.660+0000 7f96a5769700 1 --2- 172.21.15.169:0/244026263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96a0103e60 0x7f96a010a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.654 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.660+0000 7f96a5769700 1 --2- 172.21.15.169:0/244026263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96a01022c0 0x7f96a01026e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.654 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.660+0000 7f96a5769700 1 --2- 172.21.15.169:0/244026263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96a01034c0 0x7f96a0103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.654 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.660+0000 7f96a5769700 1 -- 172.21.15.169:0/244026263 >> 172.21.15.169:0/244026263 conn(0x7f96a00fd880 msgr2=0x7f96a00ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:17.654 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.660+0000 7f96a5769700 1 -- 172.21.15.169:0/244026263 shutdown_connections 2024-04-25T01:09:17.654 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.660+0000 7f96a5769700 1 -- 172.21.15.169:0/244026263 wait complete. 2024-04-25T01:09:17.654 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.660+0000 7f96a5769700 1 Processor -- start 2024-04-25T01:09:17.654 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.661+0000 7f96a5769700 1 -- start start 2024-04-25T01:09:17.654 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.661+0000 7f96a5769700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96a01022c0 0x7f96a0198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.654 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.661+0000 7f96a5769700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96a01034c0 0x7f96a01989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.654 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.661+0000 7f96a5769700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96a0103e60 0x7f96a019fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.654 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.661+0000 7f96a5769700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f96a010df00 con 0x7f96a0103e60 2024-04-25T01:09:17.655 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.661+0000 7f96a5769700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f96a010dd80 con 0x7f96a01034c0 2024-04-25T01:09:17.655 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.661+0000 7f96a5769700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f96a010e080 con 0x7f96a01022c0 2024-04-25T01:09:17.655 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.661+0000 7f969effd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96a01022c0 0x7f96a0198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:17.655 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.661+0000 7f969effd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96a01022c0 0x7f96a0198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33642/0 (socket says 172.21.15.169:33642) 2024-04-25T01:09:17.655 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.661+0000 7f969effd700 1 -- 172.21.15.169:0/3298071290 learned_addr learned my addr 172.21.15.169:0/3298071290 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T01:09:17.655 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.661+0000 7f96967fc700 1 --2- 172.21.15.169:0/3298071290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96a01034c0 0x7f96a01989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:17.655 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.661+0000 7f969effd700 1 -- 172.21.15.169:0/3298071290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96a01034c0 msgr2=0x7f96a01989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:17.655 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.661+0000 7f969effd700 1 --2- 172.21.15.169:0/3298071290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96a01034c0 0x7f96a01989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.655 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.661+0000 7f969effd700 1 -- 172.21.15.169:0/3298071290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96a0103e60 msgr2=0x7f96a019fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:17.655 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.661+0000 7f969effd700 1 --2- 172.21.15.169:0/3298071290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96a0103e60 0x7f96a019fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.655 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.661+0000 7f969effd700 1 -- 172.21.15.169:0/3298071290 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a01a00c0 con 0x7f96a01022c0 2024-04-25T01:09:17.655 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.661+0000 7f969effd700 1 --2- 172.21.15.169:0/3298071290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96a01022c0 0x7f96a0198460 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9690000c00 tx=0x7f9690009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:17.655 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.661+0000 7f969cff9700 1 -- 172.21.15.169:0/3298071290 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9690027070 con 0x7f96a01022c0 2024-04-25T01:09:17.655 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.662+0000 7f969cff9700 1 -- 172.21.15.169:0/3298071290 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9690004150 con 0x7f96a01022c0 2024-04-25T01:09:17.656 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.662+0000 7f96a5769700 1 -- 172.21.15.169:0/3298071290 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a01a0350 con 0x7f96a01022c0 2024-04-25T01:09:17.656 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.662+0000 7f969cff9700 1 -- 172.21.15.169:0/3298071290 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969000f6c0 con 0x7f96a01022c0 2024-04-25T01:09:17.656 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.662+0000 7f96a5769700 1 -- 172.21.15.169:0/3298071290 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a01a0820 con 0x7f96a01022c0 2024-04-25T01:09:17.656 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.662+0000 7f96a5769700 1 -- 172.21.15.169:0/3298071290 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96a0068f20 con 0x7f96a01022c0 2024-04-25T01:09:17.657 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.664+0000 7f969cff9700 1 -- 172.21.15.169:0/3298071290 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f969002a070 con 0x7f96a01022c0 2024-04-25T01:09:17.657 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.664+0000 7f969cff9700 1 --2- 172.21.15.169:0/3298071290 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f968c0a2c20 0x7f968c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.657 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.664+0000 7f969cff9700 1 -- 172.21.15.169:0/3298071290 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4101+0+0 (secure 0 0 0) 0x7f96900cf810 con 0x7f96a01022c0 2024-04-25T01:09:17.657 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.665+0000 7f96967fc700 1 --2- 172.21.15.169:0/3298071290 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f968c0a2c20 0x7f968c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:17.657 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.666+0000 7f96967fc700 1 --2- 172.21.15.169:0/3298071290 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f968c0a2c20 0x7f968c0a50e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9688005ea0 tx=0x7f9688005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:17.659 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.668+0000 7f969cff9700 1 -- 172.21.15.169:0/3298071290 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9690031020 con 0x7f96a01022c0 2024-04-25T01:09:17.866 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:17.875+0000 7f96a5769700 1 -- 172.21.15.169:0/3298071290 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f96a0062670 con 0x7f968c0a2c20 2024-04-25T01:09:17.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:17 smithi169 ceph-mon[32759]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:17.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:17 smithi169 ceph-mon[32759]: Detected new or changed devices on smithi040 2024-04-25T01:09:17.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:17.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:09:17.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:17 smithi169 ceph-mon[32759]: osdmap e36: 5 total, 5 up, 5 in 2024-04-25T01:09:17.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:09:17.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:17 smithi169 ceph-mon[32759]: Adjusting osd_memory_target on smithi040 to 8546M 2024-04-25T01:09:17.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:17.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:17.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:09:17.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:17.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:09:17.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:17.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:09:17.951 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:18.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:17 smithi040 ceph-mon[32518]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:18.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:17 smithi040 ceph-mon[32518]: Detected new or changed devices on smithi040 2024-04-25T01:09:18.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:18.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:09:18.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:17 smithi040 ceph-mon[32518]: osdmap e36: 5 total, 5 up, 5 in 2024-04-25T01:09:18.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:09:18.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:17 smithi040 ceph-mon[32518]: Adjusting osd_memory_target on smithi040 to 8546M 2024-04-25T01:09:18.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:18.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:18.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:09:18.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:18.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:09:18.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:18.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:09:18.204 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:18.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:17 smithi031 ceph-mon[28053]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T01:09:18.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:17 smithi031 ceph-mon[28053]: Detected new or changed devices on smithi040 2024-04-25T01:09:18.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:18.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:09:18.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:17 smithi031 ceph-mon[28053]: osdmap e36: 5 total, 5 up, 5 in 2024-04-25T01:09:18.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:09:18.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:17 smithi031 ceph-mon[28053]: Adjusting osd_memory_target on smithi040 to 8546M 2024-04-25T01:09:18.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:18.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:18.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:09:18.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:18.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:09:18.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:18.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:09:18.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:19.199 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:18 smithi169 ceph-mon[32759]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:09:19.199 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:18 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:09:19.199 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:18 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:09:19.199 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:18 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:19.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:18 smithi040 ceph-mon[32518]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:09:19.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:18 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:09:19.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:18 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:09:19.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:18 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:19.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:18 smithi031 ceph-mon[28053]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:09:19.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:09:19.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:09:19.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:19.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:19 smithi169 ceph-mon[32759]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:20.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:19 smithi040 ceph-mon[32518]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:20.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:19 smithi031 ceph-mon[28053]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:21.199 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:20 smithi169 ceph-mon[32759]: from='client.? 172.21.15.169:0/4216810716' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8da55693-f9c7-49a7-99e0-165078f8001b"}]: dispatch 2024-04-25T01:09:21.199 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:20 smithi169 ceph-mon[32759]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8da55693-f9c7-49a7-99e0-165078f8001b"}]: dispatch 2024-04-25T01:09:21.199 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:20 smithi169 ceph-mon[32759]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8da55693-f9c7-49a7-99e0-165078f8001b"}]': finished 2024-04-25T01:09:21.199 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:20 smithi169 ceph-mon[32759]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T01:09:21.199 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:20 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:09:21.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:20 smithi040 ceph-mon[32518]: from='client.? 172.21.15.169:0/4216810716' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8da55693-f9c7-49a7-99e0-165078f8001b"}]: dispatch 2024-04-25T01:09:21.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:20 smithi040 ceph-mon[32518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8da55693-f9c7-49a7-99e0-165078f8001b"}]: dispatch 2024-04-25T01:09:21.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:20 smithi040 ceph-mon[32518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8da55693-f9c7-49a7-99e0-165078f8001b"}]': finished 2024-04-25T01:09:21.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:20 smithi040 ceph-mon[32518]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T01:09:21.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:20 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:09:21.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:20 smithi031 ceph-mon[28053]: from='client.? 172.21.15.169:0/4216810716' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8da55693-f9c7-49a7-99e0-165078f8001b"}]: dispatch 2024-04-25T01:09:21.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:20 smithi031 ceph-mon[28053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8da55693-f9c7-49a7-99e0-165078f8001b"}]: dispatch 2024-04-25T01:09:21.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:20 smithi031 ceph-mon[28053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8da55693-f9c7-49a7-99e0-165078f8001b"}]': finished 2024-04-25T01:09:21.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:20 smithi031 ceph-mon[28053]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T01:09:21.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:20 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:09:22.199 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:21 smithi169 ceph-mon[32759]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:22.199 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:21 smithi169 ceph-mon[32759]: from='client.? 172.21.15.169:0/2937642510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:09:22.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:21 smithi040 ceph-mon[32518]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:22.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:21 smithi040 ceph-mon[32518]: from='client.? 172.21.15.169:0/2937642510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:09:22.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:21 smithi031 ceph-mon[28053]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:22.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:21 smithi031 ceph-mon[28053]: from='client.? 172.21.15.169:0/2937642510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:09:24.199 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:23 smithi169 ceph-mon[32759]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:24.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:23 smithi040 ceph-mon[32518]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:24.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:23 smithi031 ceph-mon[28053]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:25.910 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:25 smithi169 ceph-mon[32759]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:26.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:25 smithi040 ceph-mon[32518]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:26.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:25 smithi031 ceph-mon[28053]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:27.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:27 smithi169 ceph-mon[32759]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:27.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T01:09:27.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:28.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:27 smithi040 ceph-mon[32518]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:28.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:27 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T01:09:28.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:27 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:28.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:27 smithi031 ceph-mon[28053]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:28.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:27 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T01:09:28.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:27 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:28.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:28 smithi169 ceph-mon[32759]: Deploying daemon osd.5 on smithi169 2024-04-25T01:09:29.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:28 smithi040 ceph-mon[32518]: Deploying daemon osd.5 on smithi169 2024-04-25T01:09:29.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:28 smithi031 ceph-mon[28053]: Deploying daemon osd.5 on smithi169 2024-04-25T01:09:29.948 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:29 smithi169 ceph-mon[32759]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:30.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:29 smithi040 ceph-mon[32518]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:30.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:29 smithi031 ceph-mon[28053]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:31.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:31 smithi169 ceph-mon[32759]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:32.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:31 smithi040 ceph-mon[32518]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:32.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:31 smithi031 ceph-mon[28053]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:34.009 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:33 smithi031 ceph-mon[28053]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:34.009 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:33 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:34.009 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:33 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:09:34.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:33 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:34.199 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:33 smithi169 ceph-mon[32759]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:34.199 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:33 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:34.199 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:33 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:09:34.199 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:33 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:34.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:33 smithi040 ceph-mon[32518]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:34.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:33 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:34.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:33 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:09:34.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:33 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:35.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:35 smithi169 ceph-mon[32759]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:35.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:35 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:35.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:35 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:35.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:35 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:35.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:35 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:09:35.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:35 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:36.009 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:35 smithi031 ceph-mon[28053]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:36.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:35 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:36.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:35 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:36.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:35 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:36.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:35 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:09:36.010 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:35 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:36.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:35 smithi040 ceph-mon[32518]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:36.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:35 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:36.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:35 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:36.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:35 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:36.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:35 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:09:36.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:35 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:36.349 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:36.359+0000 7f969cff9700 1 -- 172.21.15.169:0/3298071290 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f96a0062670 con 0x7f968c0a2c20 2024-04-25T01:09:36.350 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 5 on host 'smithi169' 2024-04-25T01:09:36.352 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:36.362+0000 7f96a5769700 1 -- 172.21.15.169:0/3298071290 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f968c0a2c20 msgr2=0x7f968c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:36.352 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:36.362+0000 7f96a5769700 1 --2- 172.21.15.169:0/3298071290 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f968c0a2c20 0x7f968c0a50e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9688005ea0 tx=0x7f9688005e30 comp rx=0 tx=0).stop 2024-04-25T01:09:36.352 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:36.362+0000 7f96a5769700 1 -- 172.21.15.169:0/3298071290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96a01022c0 msgr2=0x7f96a0198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:36.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:36.362+0000 7f96a5769700 1 --2- 172.21.15.169:0/3298071290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96a01022c0 0x7f96a0198460 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9690000c00 tx=0x7f9690009ee0 comp rx=0 tx=0).stop 2024-04-25T01:09:36.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:36.362+0000 7f96a5769700 1 -- 172.21.15.169:0/3298071290 shutdown_connections 2024-04-25T01:09:36.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:36.362+0000 7f96a5769700 1 --2- 172.21.15.169:0/3298071290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96a0103e60 0x7f96a019fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:36.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:36.362+0000 7f96a5769700 1 --2- 172.21.15.169:0/3298071290 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f968c0a2c20 0x7f968c0a50e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:36.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:36.362+0000 7f96a5769700 1 --2- 172.21.15.169:0/3298071290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96a01022c0 0x7f96a0198460 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:36.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:36.362+0000 7f96a5769700 1 --2- 172.21.15.169:0/3298071290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96a01034c0 0x7f96a01989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:36.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:36.362+0000 7f96a5769700 1 -- 172.21.15.169:0/3298071290 >> 172.21.15.169:0/3298071290 conn(0x7f96a00fd880 msgr2=0x7f96a0106760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:36.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:36.362+0000 7f96a5769700 1 -- 172.21.15.169:0/3298071290 shutdown_connections 2024-04-25T01:09:36.353 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:09:36.362+0000 7f96a5769700 1 -- 172.21.15.169:0/3298071290 wait complete. 2024-04-25T01:09:36.975 DEBUG:teuthology.orchestra.run.smithi169:osd.5> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T01:09:36.977 INFO:tasks.cephadm:Deploying osd.6 on smithi031 with /dev/vg_nvme/lv_2... 2024-04-25T01:09:36.977 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T01:09:37.435 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:09:37.476 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:37 smithi169 ceph-mon[32759]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:37.476 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:37 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:37.476 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:37 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:37.476 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:37 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:09:37.476 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:37 smithi169 ceph-mon[32759]: from='osd.5 [v2:172.21.15.169:6808/2549650781,v1:172.21.15.169:6809/2549650781]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T01:09:37.476 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:37 smithi169 ceph-mon[32759]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T01:09:37.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:37 smithi031 ceph-mon[28053]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:37.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:37 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:37.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:37 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:37.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:37 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:09:37.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:37 smithi031 ceph-mon[28053]: from='osd.5 [v2:172.21.15.169:6808/2549650781,v1:172.21.15.169:6809/2549650781]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T01:09:37.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:37 smithi031 ceph-mon[28053]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T01:09:37.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:37 smithi040 ceph-mon[32518]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:37.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:37 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:37.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:37 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:37.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:37 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:09:37.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:37 smithi040 ceph-mon[32518]: from='osd.5 [v2:172.21.15.169:6808/2549650781,v1:172.21.15.169:6809/2549650781]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T01:09:37.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:37 smithi040 ceph-mon[32518]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T01:09:38.499 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:38 smithi169 ceph-mon[32759]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T01:09:38.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:38 smithi169 ceph-mon[32759]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T01:09:38.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:38 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:09:38.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:38 smithi169 ceph-mon[32759]: from='osd.5 [v2:172.21.15.169:6808/2549650781,v1:172.21.15.169:6809/2549650781]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T01:09:38.500 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:38 smithi169 ceph-mon[32759]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T01:09:38.500 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:09:38 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:09:38.323+0000 7fe9d74ab700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T01:09:38.500 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:09:38 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:09:38.332+0000 7fe9d3a9f700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:09:38.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:38 smithi031 ceph-mon[28053]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T01:09:38.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:38 smithi031 ceph-mon[28053]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T01:09:38.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:38 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:09:38.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:38 smithi031 ceph-mon[28053]: from='osd.5 [v2:172.21.15.169:6808/2549650781,v1:172.21.15.169:6809/2549650781]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T01:09:38.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:38 smithi031 ceph-mon[28053]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T01:09:38.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:38 smithi040 ceph-mon[32518]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T01:09:38.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:38 smithi040 ceph-mon[32518]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T01:09:38.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:38 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:09:38.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:38 smithi040 ceph-mon[32518]: from='osd.5 [v2:172.21.15.169:6808/2549650781,v1:172.21.15.169:6809/2549650781]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T01:09:38.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:38 smithi040 ceph-mon[32518]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T01:09:39.449 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:39 smithi169 ceph-mon[32759]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:39.449 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:39 smithi169 ceph-mon[32759]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T01:09:39.449 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:39 smithi169 ceph-mon[32759]: osdmap e39: 6 total, 5 up, 6 in 2024-04-25T01:09:39.449 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:39 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:09:39.696 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:39 smithi031 ceph-mon[28053]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:39.696 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:39 smithi031 ceph-mon[28053]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T01:09:39.696 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:39 smithi031 ceph-mon[28053]: osdmap e39: 6 total, 5 up, 6 in 2024-04-25T01:09:39.696 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:39 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:09:39.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:39 smithi040 ceph-mon[32518]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:09:39.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:39 smithi040 ceph-mon[32518]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T01:09:39.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:39 smithi040 ceph-mon[32518]: osdmap e39: 6 total, 5 up, 6 in 2024-04-25T01:09:39.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:39 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:09:40.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:40 smithi169 ceph-mon[32759]: purged_snaps scrub starts 2024-04-25T01:09:40.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:40 smithi169 ceph-mon[32759]: purged_snaps scrub ok 2024-04-25T01:09:40.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:40 smithi169 ceph-mon[32759]: osd.5 [v2:172.21.15.169:6808/2549650781,v1:172.21.15.169:6809/2549650781] boot 2024-04-25T01:09:40.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:40 smithi169 ceph-mon[32759]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T01:09:40.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:40 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:09:40.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:40 smithi169 ceph-mon[32759]: Detected new or changed devices on smithi169 2024-04-25T01:09:40.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:40 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:40.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:40 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:40.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:40 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:09:40.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:40 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:09:40.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:40 smithi169 ceph-mon[32759]: Adjusting osd_memory_target on smithi169 to 8544M 2024-04-25T01:09:40.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:40 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:40.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:40 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:40.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:40 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:09:40.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:40 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:40.700 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:40 smithi169 ceph-mon[32759]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T01:09:40.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:40 smithi040 ceph-mon[32518]: purged_snaps scrub starts 2024-04-25T01:09:40.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:40 smithi040 ceph-mon[32518]: purged_snaps scrub ok 2024-04-25T01:09:40.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:40 smithi040 ceph-mon[32518]: osd.5 [v2:172.21.15.169:6808/2549650781,v1:172.21.15.169:6809/2549650781] boot 2024-04-25T01:09:40.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:40 smithi040 ceph-mon[32518]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T01:09:40.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:40 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:09:40.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:40 smithi040 ceph-mon[32518]: Detected new or changed devices on smithi169 2024-04-25T01:09:40.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:40 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:40.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:40 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:40.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:40 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:09:40.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:40 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:09:40.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:40 smithi040 ceph-mon[32518]: Adjusting osd_memory_target on smithi169 to 8544M 2024-04-25T01:09:40.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:40 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:40.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:40 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:40.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:40 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:09:40.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:40 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:40.703 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:40 smithi040 ceph-mon[32518]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T01:09:40.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:40 smithi031 ceph-mon[28053]: purged_snaps scrub starts 2024-04-25T01:09:40.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:40 smithi031 ceph-mon[28053]: purged_snaps scrub ok 2024-04-25T01:09:40.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:40 smithi031 ceph-mon[28053]: osd.5 [v2:172.21.15.169:6808/2549650781,v1:172.21.15.169:6809/2549650781] boot 2024-04-25T01:09:40.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:40 smithi031 ceph-mon[28053]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T01:09:40.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:40 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:09:40.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:40 smithi031 ceph-mon[28053]: Detected new or changed devices on smithi169 2024-04-25T01:09:40.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:40 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:40.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:40 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:40.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:40 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:09:40.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:40 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:09:40.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:40 smithi031 ceph-mon[28053]: Adjusting osd_memory_target on smithi169 to 8544M 2024-04-25T01:09:40.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:40 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:40.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:40 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:40.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:40 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:09:40.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:40 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:09:40.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:40 smithi031 ceph-mon[28053]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T01:09:40.909 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T01:09:40.909 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T01:09:40.909 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2024-04-25T01:09:40.909 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2024-04-25T01:09:40.909 INFO:teuthology.orchestra.run.smithi031.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0340979 s, 308 MB/s 2024-04-25T01:09:40.910 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2024-04-25T01:09:41.968 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_2 2024-04-25T01:09:42.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:41 smithi169 ceph-mon[32759]: pgmap v154: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:09:42.199 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:41 smithi169 ceph-mon[32759]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T01:09:42.201 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:41 smithi040 ceph-mon[32518]: pgmap v154: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:09:42.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:41 smithi040 ceph-mon[32518]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T01:09:42.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:41 smithi031 ceph-mon[28053]: pgmap v154: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:09:42.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:41 smithi031 ceph-mon[28053]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T01:09:42.429 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:09:43.640 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:43 smithi031 ceph-mon[28053]: pgmap v157: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:09:43.641 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:43 smithi031 ceph-mon[28053]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T01:09:43.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:43 smithi169 ceph-mon[32759]: pgmap v157: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:09:43.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:43 smithi169 ceph-mon[32759]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T01:09:43.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:43 smithi040 ceph-mon[32518]: pgmap v157: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:09:43.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:43 smithi040 ceph-mon[32518]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T01:09:44.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:44 smithi031 ceph-mon[28053]: osdmap e44: 6 total, 6 up, 6 in 2024-04-25T01:09:44.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.552+0000 7fe9ab800700 1 -- 172.21.15.31:0/1091540450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4101e30 msgr2=0x7fe9a41022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:44.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.552+0000 7fe9ab800700 1 --2- 172.21.15.31:0/1091540450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4101e30 0x7fe9a41022b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00b0a0 tx=0x7fe99c00b3b0 comp rx=0 tx=0).stop 2024-04-25T01:09:44.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.552+0000 7fe9ab800700 1 -- 172.21.15.31:0/1091540450 shutdown_connections 2024-04-25T01:09:44.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.552+0000 7fe9ab800700 1 --2- 172.21.15.31:0/1091540450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4101e30 0x7fe9a41022b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:44.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.552+0000 7fe9ab800700 1 --2- 172.21.15.31:0/1091540450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9a4100b10 0x7fe9a4100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:44.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.552+0000 7fe9ab800700 1 --2- 172.21.15.31:0/1091540450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9a4101470 0x7fe9a41018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:44.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.552+0000 7fe9ab800700 1 -- 172.21.15.31:0/1091540450 >> 172.21.15.31:0/1091540450 conn(0x7fe9a40fc730 msgr2=0x7fe9a40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:44.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.552+0000 7fe9ab800700 1 -- 172.21.15.31:0/1091540450 shutdown_connections 2024-04-25T01:09:44.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.552+0000 7fe9ab800700 1 -- 172.21.15.31:0/1091540450 wait complete. 2024-04-25T01:09:44.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.553+0000 7fe9ab800700 1 Processor -- start 2024-04-25T01:09:44.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.553+0000 7fe9ab800700 1 -- start start 2024-04-25T01:09:44.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.554+0000 7fe9ab800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4100b10 0x7fe9a419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:44.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.554+0000 7fe9ab800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9a4101470 0x7fe9a419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:44.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.554+0000 7fe9ab800700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9a4101e30 0x7fe9a41a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:44.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.554+0000 7fe9ab800700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a41100a0 con 0x7fe9a4100b10 2024-04-25T01:09:44.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.554+0000 7fe9ab800700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a410ff20 con 0x7fe9a4101e30 2024-04-25T01:09:44.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.554+0000 7fe9ab800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a4110220 con 0x7fe9a4101470 2024-04-25T01:09:44.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.554+0000 7fe9a8e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4100b10 0x7fe9a419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:44.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.554+0000 7fe99be00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9a4101470 0x7fe9a419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:44.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.554+0000 7fe9a8e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4100b10 0x7fe9a419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56396/0 (socket says 172.21.15.31:56396) 2024-04-25T01:09:44.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.554+0000 7fe9a8e00700 1 -- 172.21.15.31:0/1932629869 learned_addr learned my addr 172.21.15.31:0/1932629869 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:09:44.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.554+0000 7fe9a9800700 1 --2- 172.21.15.31:0/1932629869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9a4101e30 0x7fe9a41a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:44.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.555+0000 7fe99be00700 1 -- 172.21.15.31:0/1932629869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9a4101e30 msgr2=0x7fe9a41a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:44.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.555+0000 7fe99be00700 1 --2- 172.21.15.31:0/1932629869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9a4101e30 0x7fe9a41a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:44.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.555+0000 7fe99be00700 1 -- 172.21.15.31:0/1932629869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4100b10 msgr2=0x7fe9a419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:44.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.555+0000 7fe99be00700 1 --2- 172.21.15.31:0/1932629869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4100b10 0x7fe9a419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:44.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.555+0000 7fe99be00700 1 -- 172.21.15.31:0/1932629869 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a41a22f0 con 0x7fe9a4101470 2024-04-25T01:09:44.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.555+0000 7fe99be00700 1 --2- 172.21.15.31:0/1932629869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9a4101470 0x7fe9a419abd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00ea20 tx=0x7fe98c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:44.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.555+0000 7fe9a2000700 1 -- 172.21.15.31:0/1932629869 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c00cbd0 con 0x7fe9a4101470 2024-04-25T01:09:44.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.555+0000 7fe9ab800700 1 -- 172.21.15.31:0/1932629869 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a41a25e0 con 0x7fe9a4101470 2024-04-25T01:09:44.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.555+0000 7fe9a2000700 1 -- 172.21.15.31:0/1932629869 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c00ee60 con 0x7fe9a4101470 2024-04-25T01:09:44.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.555+0000 7fe9a2000700 1 -- 172.21.15.31:0/1932629869 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c018680 con 0x7fe9a4101470 2024-04-25T01:09:44.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.556+0000 7fe9ab800700 1 -- 172.21.15.31:0/1932629869 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a41a2b70 con 0x7fe9a4101470 2024-04-25T01:09:44.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.557+0000 7fe9ab800700 1 -- 172.21.15.31:0/1932629869 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9a4068f20 con 0x7fe9a4101470 2024-04-25T01:09:44.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.559+0000 7fe9a2000700 1 -- 172.21.15.31:0/1932629869 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe98c018820 con 0x7fe9a4101470 2024-04-25T01:09:44.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.560+0000 7fe9a2000700 1 --2- 172.21.15.31:0/1932629869 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe9880a2ca0 0x7fe9880a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:44.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.561+0000 7fe9a2000700 1 -- 172.21.15.31:0/1932629869 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4533+0+0 (secure 0 0 0) 0x7fe98c0ce370 con 0x7fe9a4101470 2024-04-25T01:09:44.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.561+0000 7fe9a8e00700 1 --2- 172.21.15.31:0/1932629869 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe9880a2ca0 0x7fe9880a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:44.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.561+0000 7fe9a8e00700 1 --2- 172.21.15.31:0/1932629869 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe9880a2ca0 0x7fe9880a5160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe994009d20 tx=0x7fe994009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:44.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.565+0000 7fe9a2000700 1 -- 172.21.15.31:0/1932629869 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe98c09c0f0 con 0x7fe9a4101470 2024-04-25T01:09:44.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:44 smithi169 ceph-mon[32759]: osdmap e44: 6 total, 6 up, 6 in 2024-04-25T01:09:44.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:44 smithi040 ceph-mon[32518]: osdmap e44: 6 total, 6 up, 6 in 2024-04-25T01:09:44.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:09:44.804+0000 7fe9ab800700 1 -- 172.21.15.31:0/1932629869 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe9a4062670 con 0x7fe9880a2ca0 2024-04-25T01:09:45.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:45 smithi169 ceph-mon[32759]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T01:09:45.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:45 smithi169 ceph-mon[32759]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:09:45.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:45 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:09:45.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:45 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:09:45.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:45 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:45.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:45 smithi040 ceph-mon[32518]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T01:09:45.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:45 smithi040 ceph-mon[32518]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:09:45.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:45 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:09:45.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:45 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:09:45.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:45 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:45.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:45 smithi031 ceph-mon[28053]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T01:09:45.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:45 smithi031 ceph-mon[28053]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:09:45.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:45 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:09:45.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:45 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:09:45.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:45 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:47.593 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:47 smithi031 ceph-mon[28053]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-25T01:09:47.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:47 smithi169 ceph-mon[32759]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-25T01:09:47.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:47 smithi040 ceph-mon[32518]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-25T01:09:49.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:49 smithi169 ceph-mon[32759]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T01:09:49.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:49 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/852588556' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc8350d2-1ad6-43cb-89f4-d336afc99ee3"}]: dispatch 2024-04-25T01:09:49.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:49 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/852588556' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc8350d2-1ad6-43cb-89f4-d336afc99ee3"}]': finished 2024-04-25T01:09:49.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:49 smithi169 ceph-mon[32759]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T01:09:49.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:49 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:09:49.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:49 smithi040 ceph-mon[32518]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T01:09:49.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:49 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/852588556' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc8350d2-1ad6-43cb-89f4-d336afc99ee3"}]: dispatch 2024-04-25T01:09:49.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:49 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/852588556' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc8350d2-1ad6-43cb-89f4-d336afc99ee3"}]': finished 2024-04-25T01:09:49.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:49 smithi040 ceph-mon[32518]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T01:09:49.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:49 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:09:49.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:49 smithi031 ceph-mon[28053]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T01:09:49.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:49 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/852588556' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc8350d2-1ad6-43cb-89f4-d336afc99ee3"}]: dispatch 2024-04-25T01:09:49.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:49 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/852588556' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc8350d2-1ad6-43cb-89f4-d336afc99ee3"}]': finished 2024-04-25T01:09:49.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:49 smithi031 ceph-mon[28053]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T01:09:49.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:49 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:09:50.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:50 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3324415562' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:09:50.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:50 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3324415562' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:09:50.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:50 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3324415562' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:09:51.701 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:51 smithi169 ceph-mon[32759]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-25T01:09:51.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:51 smithi040 ceph-mon[32518]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-25T01:09:51.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:51 smithi031 ceph-mon[28053]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-25T01:09:53.630 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:53 smithi031 ceph-mon[28053]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-25T01:09:53.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:53 smithi169 ceph-mon[32759]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-25T01:09:53.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:53 smithi040 ceph-mon[32518]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-25T01:09:55.644 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:55 smithi031 ceph-mon[28053]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:09:55.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:55 smithi169 ceph-mon[32759]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:09:55.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:55 smithi040 ceph-mon[32518]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:09:57.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:57 smithi169 ceph-mon[32759]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:09:57.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:57 smithi040 ceph-mon[32518]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:09:57.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:57 smithi031 ceph-mon[28053]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:09:59.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:59 smithi169 ceph-mon[32759]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:09:59.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:59 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T01:09:59.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:59 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:59.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:09:59 smithi169 ceph-mon[32759]: Deploying daemon osd.6 on smithi031 2024-04-25T01:09:59.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:59 smithi040 ceph-mon[32518]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:09:59.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:59 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T01:09:59.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:59 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:59.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:09:59 smithi040 ceph-mon[32518]: Deploying daemon osd.6 on smithi031 2024-04-25T01:09:59.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:59 smithi031 ceph-mon[28053]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:09:59.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:59 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T01:09:59.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:59 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:09:59.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:09:59 smithi031 ceph-mon[28053]: Deploying daemon osd.6 on smithi031 2024-04-25T01:10:00.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:00 smithi169 ceph-mon[32759]: overall HEALTH_OK 2024-04-25T01:10:00.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:00 smithi040 ceph-mon[32518]: overall HEALTH_OK 2024-04-25T01:10:00.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:00 smithi031 ceph-mon[28053]: overall HEALTH_OK 2024-04-25T01:10:01.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:01 smithi169 ceph-mon[32759]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:01.700 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:01 smithi031 ceph-mon[28053]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:01.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:01 smithi040 ceph-mon[32518]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:03.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:03 smithi169 ceph-mon[32759]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:03.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:03 smithi040 ceph-mon[32518]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:03.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:03 smithi031 ceph-mon[28053]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:05.664 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:05 smithi031 ceph-mon[28053]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:05.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:05 smithi169 ceph-mon[32759]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:05.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:05 smithi040 ceph-mon[32518]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:07.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:07 smithi169 ceph-mon[32759]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:07.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:07 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:07.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:07 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:07.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:07 smithi040 ceph-mon[32518]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:07.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:07 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:07.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:07 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:07.743 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:07 smithi031 ceph-mon[28053]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:07.743 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:07 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:07.743 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:07 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:08.685 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:08 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:10:08.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:08 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:10:08.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:08 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:10:09.451 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:09 smithi031 ceph-mon[28053]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:09.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:09 smithi169 ceph-mon[32759]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:09.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:09 smithi040 ceph-mon[32518]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:11.498 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:11 smithi031 ceph-mon[28053]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:11.499 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:11 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:11.499 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:11 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:11.499 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:11 smithi031 ceph-mon[28053]: from='osd.6 [v2:172.21.15.31:6818/611507624,v1:172.21.15.31:6819/611507624]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T01:10:11.499 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:11 smithi031 ceph-mon[28053]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T01:10:11.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:11 smithi169 ceph-mon[32759]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:11.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:11 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:11.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:11 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:11.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:11 smithi169 ceph-mon[32759]: from='osd.6 [v2:172.21.15.31:6818/611507624,v1:172.21.15.31:6819/611507624]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T01:10:11.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:11 smithi169 ceph-mon[32759]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T01:10:11.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:11 smithi040 ceph-mon[32518]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:11.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:11 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:11.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:11 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:11.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:11 smithi040 ceph-mon[32518]: from='osd.6 [v2:172.21.15.31:6818/611507624,v1:172.21.15.31:6819/611507624]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T01:10:11.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:11 smithi040 ceph-mon[32518]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T01:10:12.438 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 6 on host 'smithi031' 2024-04-25T01:10:12.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:10:12.446+0000 7fe9a2000700 1 -- 172.21.15.31:0/1932629869 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe9a4062670 con 0x7fe9880a2ca0 2024-04-25T01:10:12.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:10:12.450+0000 7fe9ab800700 1 -- 172.21.15.31:0/1932629869 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe9880a2ca0 msgr2=0x7fe9880a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:10:12.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:10:12.450+0000 7fe9ab800700 1 --2- 172.21.15.31:0/1932629869 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe9880a2ca0 0x7fe9880a5160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe994009d20 tx=0x7fe994009260 comp rx=0 tx=0).stop 2024-04-25T01:10:12.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:10:12.450+0000 7fe9ab800700 1 -- 172.21.15.31:0/1932629869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9a4101470 msgr2=0x7fe9a419abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:10:12.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:10:12.450+0000 7fe9ab800700 1 --2- 172.21.15.31:0/1932629869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9a4101470 0x7fe9a419abd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00ea20 tx=0x7fe98c00ed30 comp rx=0 tx=0).stop 2024-04-25T01:10:12.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:10:12.450+0000 7fe9ab800700 1 -- 172.21.15.31:0/1932629869 shutdown_connections 2024-04-25T01:10:12.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:10:12.450+0000 7fe9ab800700 1 --2- 172.21.15.31:0/1932629869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4100b10 0x7fe9a419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:12.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:10:12.450+0000 7fe9ab800700 1 --2- 172.21.15.31:0/1932629869 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe9880a2ca0 0x7fe9880a5160 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:12.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:10:12.450+0000 7fe9ab800700 1 --2- 172.21.15.31:0/1932629869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9a4101e30 0x7fe9a41a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:12.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:10:12.450+0000 7fe9ab800700 1 --2- 172.21.15.31:0/1932629869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9a4101470 0x7fe9a419abd0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:12.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:10:12.450+0000 7fe9ab800700 1 -- 172.21.15.31:0/1932629869 >> 172.21.15.31:0/1932629869 conn(0x7fe9a40fc730 msgr2=0x7fe9a40fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:10:12.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:10:12.450+0000 7fe9ab800700 1 -- 172.21.15.31:0/1932629869 shutdown_connections 2024-04-25T01:10:12.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:10:12.451+0000 7fe9ab800700 1 -- 172.21.15.31:0/1932629869 wait complete. 2024-04-25T01:10:13.176 DEBUG:teuthology.orchestra.run.smithi031:osd.6> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T01:10:13.182 INFO:tasks.cephadm:Deploying osd.7 on smithi040 with /dev/vg_nvme/lv_2... 2024-04-25T01:10:13.182 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T01:10:13.434 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:13 smithi031 ceph-mon[28053]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:13.434 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:13 smithi031 ceph-mon[28053]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T01:10:13.434 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:13 smithi031 ceph-mon[28053]: from='osd.6 [v2:172.21.15.31:6818/611507624,v1:172.21.15.31:6819/611507624]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T01:10:13.434 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:13 smithi031 ceph-mon[28053]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T01:10:13.434 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:10:13.434 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:13 smithi031 ceph-mon[28053]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T01:10:13.434 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:13.435 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:13.437 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T01:10:13.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:13 smithi169 ceph-mon[32759]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:13.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:13 smithi169 ceph-mon[32759]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T01:10:13.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:13 smithi169 ceph-mon[32759]: from='osd.6 [v2:172.21.15.31:6818/611507624,v1:172.21.15.31:6819/611507624]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T01:10:13.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:13 smithi169 ceph-mon[32759]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T01:10:13.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:10:13.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:13 smithi169 ceph-mon[32759]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T01:10:13.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:13.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:13.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:13 smithi040 ceph-mon[32518]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:13.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:13 smithi040 ceph-mon[32518]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T01:10:13.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:13 smithi040 ceph-mon[32518]: from='osd.6 [v2:172.21.15.31:6818/611507624,v1:172.21.15.31:6819/611507624]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T01:10:13.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:13 smithi040 ceph-mon[32518]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T01:10:13.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:10:13.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:13 smithi040 ceph-mon[32518]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T01:10:13.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:13.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:14.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:14 smithi040 ceph-mon[32518]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T01:10:14.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:14 smithi040 ceph-mon[32518]: osdmap e47: 7 total, 6 up, 7 in 2024-04-25T01:10:14.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:14 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:10:14.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:14 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:10:14.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:14 smithi031 ceph-mon[28053]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T01:10:14.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:14 smithi031 ceph-mon[28053]: osdmap e47: 7 total, 6 up, 7 in 2024-04-25T01:10:14.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:14 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:10:14.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:14 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:10:14.510 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:10:14 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:10:14.318+0000 7f4992600700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T01:10:14.510 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:10:14 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:10:14.344+0000 7f4988e00700 -1 osd.6 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:10:14.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:14 smithi169 ceph-mon[32759]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T01:10:14.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:14 smithi169 ceph-mon[32759]: osdmap e47: 7 total, 6 up, 7 in 2024-04-25T01:10:14.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:14 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:10:14.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:14 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:10:15.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:15 smithi031 ceph-mon[28053]: purged_snaps scrub starts 2024-04-25T01:10:15.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:15 smithi031 ceph-mon[28053]: purged_snaps scrub ok 2024-04-25T01:10:15.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:15 smithi031 ceph-mon[28053]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:15.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:15 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:10:15.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:15 smithi031 ceph-mon[28053]: from='osd.6 ' entity='osd.6' 2024-04-25T01:10:15.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:15 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:10:15.431 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T01:10:15.432 INFO:teuthology.orchestra.run.smithi040.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T01:10:15.432 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10+0 records in 2024-04-25T01:10:15.432 INFO:teuthology.orchestra.run.smithi040.stderr:10+0 records out 2024-04-25T01:10:15.432 INFO:teuthology.orchestra.run.smithi040.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011302 s, 928 MB/s 2024-04-25T01:10:15.432 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping successful for: 2024-04-25T01:10:15.458 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:15 smithi040 ceph-mon[32518]: purged_snaps scrub starts 2024-04-25T01:10:15.458 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:15 smithi040 ceph-mon[32518]: purged_snaps scrub ok 2024-04-25T01:10:15.458 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:15 smithi040 ceph-mon[32518]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:15.458 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:15 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:10:15.458 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:15 smithi040 ceph-mon[32518]: from='osd.6 ' entity='osd.6' 2024-04-25T01:10:15.459 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:15 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:10:15.599 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:15 smithi169 ceph-mon[32759]: purged_snaps scrub starts 2024-04-25T01:10:15.599 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:15 smithi169 ceph-mon[32759]: purged_snaps scrub ok 2024-04-25T01:10:15.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:15 smithi169 ceph-mon[32759]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:15.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:15 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:10:15.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:15 smithi169 ceph-mon[32759]: from='osd.6 ' entity='osd.6' 2024-04-25T01:10:15.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:15 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:10:16.018 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi040:vg_nvme/lv_2 2024-04-25T01:10:16.264 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T01:10:16.608 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:16 smithi040 ceph-mon[32518]: osd.6 [v2:172.21.15.31:6818/611507624,v1:172.21.15.31:6819/611507624] boot 2024-04-25T01:10:16.609 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:16 smithi040 ceph-mon[32518]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T01:10:16.609 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:16 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:10:16.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:16 smithi169 ceph-mon[32759]: osd.6 [v2:172.21.15.31:6818/611507624,v1:172.21.15.31:6819/611507624] boot 2024-04-25T01:10:16.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:16 smithi169 ceph-mon[32759]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T01:10:16.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:16 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:10:16.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:16 smithi031 ceph-mon[28053]: osd.6 [v2:172.21.15.31:6818/611507624,v1:172.21.15.31:6819/611507624] boot 2024-04-25T01:10:16.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:16 smithi031 ceph-mon[28053]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T01:10:16.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:16 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:10:17.501 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:17 smithi040 ceph-mon[32518]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:17.501 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:17 smithi040 ceph-mon[32518]: osdmap e49: 7 total, 7 up, 7 in 2024-04-25T01:10:17.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:17 smithi031 ceph-mon[28053]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:17.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:17 smithi031 ceph-mon[28053]: osdmap e49: 7 total, 7 up, 7 in 2024-04-25T01:10:17.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:17 smithi169 ceph-mon[32759]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:10:17.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:17 smithi169 ceph-mon[32759]: osdmap e49: 7 total, 7 up, 7 in 2024-04-25T01:10:17.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.750+0000 7fae0fac7700 1 -- 172.21.15.40:0/3853777211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae08105660 msgr2=0x7fae08107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:10:17.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.750+0000 7fae0fac7700 1 --2- 172.21.15.40:0/3853777211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae08105660 0x7fae08107a50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fae04009990 tx=0x7fae04009ca0 comp rx=0 tx=0).stop 2024-04-25T01:10:17.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.750+0000 7fae0fac7700 1 -- 172.21.15.40:0/3853777211 shutdown_connections 2024-04-25T01:10:17.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.750+0000 7fae0fac7700 1 --2- 172.21.15.40:0/3853777211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae08107f90 0x7fae0810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:17.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.750+0000 7fae0fac7700 1 --2- 172.21.15.40:0/3853777211 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae08102d30 0x7fae08105120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:17.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.750+0000 7fae0fac7700 1 --2- 172.21.15.40:0/3853777211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae08105660 0x7fae08107a50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:17.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.750+0000 7fae0fac7700 1 -- 172.21.15.40:0/3853777211 >> 172.21.15.40:0/3853777211 conn(0x7fae080fc760 msgr2=0x7fae080feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:10:17.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.751+0000 7fae0fac7700 1 -- 172.21.15.40:0/3853777211 shutdown_connections 2024-04-25T01:10:17.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.751+0000 7fae0fac7700 1 -- 172.21.15.40:0/3853777211 wait complete. 2024-04-25T01:10:17.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.751+0000 7fae0fac7700 1 Processor -- start 2024-04-25T01:10:17.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.751+0000 7fae0fac7700 1 -- start start 2024-04-25T01:10:17.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.751+0000 7fae0fac7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae08102d30 0x7fae08102690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:10:17.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.751+0000 7fae0fac7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae08105660 0x7fae08100ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:10:17.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.751+0000 7fae0fac7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae08107f90 0x7fae08101220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:10:17.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.751+0000 7fae0fac7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fae0810dfc0 con 0x7fae08105660 2024-04-25T01:10:17.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.751+0000 7fae0fac7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fae0810de40 con 0x7fae08107f90 2024-04-25T01:10:17.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.751+0000 7fae0fac7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fae0810e140 con 0x7fae08102d30 2024-04-25T01:10:17.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.752+0000 7fae0e064700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae08107f90 0x7fae08101220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:10:17.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.752+0000 7fae0e064700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae08107f90 0x7fae08101220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57244/0 (socket says 172.21.15.40:57244) 2024-04-25T01:10:17.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.752+0000 7fae0e064700 1 -- 172.21.15.40:0/1972926690 learned_addr learned my addr 172.21.15.40:0/1972926690 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T01:10:17.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.752+0000 7fae0d863700 1 --2- 172.21.15.40:0/1972926690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae08102d30 0x7fae08102690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:10:17.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.752+0000 7fae0e064700 1 -- 172.21.15.40:0/1972926690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae08102d30 msgr2=0x7fae08102690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:10:17.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.752+0000 7fae0e064700 1 --2- 172.21.15.40:0/1972926690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae08102d30 0x7fae08102690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.752+0000 7fae0e064700 1 -- 172.21.15.40:0/1972926690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae08105660 msgr2=0x7fae08100ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:10:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.752+0000 7fae0e064700 1 --2- 172.21.15.40:0/1972926690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae08105660 0x7fae08100ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.752+0000 7fae0e064700 1 -- 172.21.15.40:0/1972926690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae08101ad0 con 0x7fae08107f90 2024-04-25T01:10:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.752+0000 7fae0e064700 1 --2- 172.21.15.40:0/1972926690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae08107f90 0x7fae08101220 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fae0000b9e0 tx=0x7fae0000bcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:10:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.752+0000 7fadfeffd700 1 -- 172.21.15.40:0/1972926690 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae00019070 con 0x7fae08107f90 2024-04-25T01:10:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.752+0000 7fadfeffd700 1 -- 172.21.15.40:0/1972926690 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae0000ce10 con 0x7fae08107f90 2024-04-25T01:10:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.752+0000 7fae0fac7700 1 -- 172.21.15.40:0/1972926690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae081a2790 con 0x7fae08107f90 2024-04-25T01:10:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.752+0000 7fadfeffd700 1 -- 172.21.15.40:0/1972926690 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae00012360 con 0x7fae08107f90 2024-04-25T01:10:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.752+0000 7fae0fac7700 1 -- 172.21.15.40:0/1972926690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae081a2cd0 con 0x7fae08107f90 2024-04-25T01:10:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.753+0000 7fae0fac7700 1 -- 172.21.15.40:0/1972926690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae08068f20 con 0x7fae08107f90 2024-04-25T01:10:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.754+0000 7fadfeffd700 1 -- 172.21.15.40:0/1972926690 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fae0001c070 con 0x7fae08107f90 2024-04-25T01:10:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.755+0000 7fadfeffd700 1 --2- 172.21.15.40:0/1972926690 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fadf40a2c20 0x7fadf40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:10:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.755+0000 7fadfeffd700 1 -- 172.21.15.40:0/1972926690 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4965+0+0 (secure 0 0 0) 0x7fae000cdbe0 con 0x7fae08107f90 2024-04-25T01:10:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.756+0000 7fae0d863700 1 --2- 172.21.15.40:0/1972926690 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fadf40a2c20 0x7fadf40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:10:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.756+0000 7fae0d863700 1 --2- 172.21.15.40:0/1972926690 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fadf40a2c20 0x7fadf40a50e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009a10 tx=0x7fadf8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:10:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.759+0000 7fadfeffd700 1 -- 172.21.15.40:0/1972926690 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fae0009b7b0 con 0x7fae08107f90 2024-04-25T01:10:17.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:17.973+0000 7fae0fac7700 1 -- 172.21.15.40:0/1972926690 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fae08062670 con 0x7fadf40a2c20 2024-04-25T01:10:18.624 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:18 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:10:18.624 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:18 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:10:18.624 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:18 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:10:18.624 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:18 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:18.625 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:18 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:18.625 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:18 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:10:18.625 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:18 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:10:18.625 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:18 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:10:18.625 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:18 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:18.625 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:18 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:10:18.625 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:18 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:10:18.625 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:18 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:18.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:18 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:10:18.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:18 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:10:18.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:18 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:10:18.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:18 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:18.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:18 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:18.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:18 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:10:18.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:18 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:10:18.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:18 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:10:18.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:18 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:18.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:18 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:10:18.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:18 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:10:18.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:18 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:18.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:10:18.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:10:18.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:10:18.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:18.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:18.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:10:18.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:10:18.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:10:18.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:18.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:10:18.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:10:18.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:19.638 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:19 smithi040 ceph-mon[32518]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:10:19.638 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:19 smithi040 ceph-mon[32518]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:19.638 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:19 smithi040 ceph-mon[32518]: Detected new or changed devices on smithi031 2024-04-25T01:10:19.639 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:19 smithi040 ceph-mon[32518]: Adjusting osd_memory_target on smithi031 to 5698M 2024-04-25T01:10:19.639 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:19 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:10:19.639 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:19 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:10:19.639 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:19 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:10:19.639 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:19 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:19.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:19 smithi169 ceph-mon[32759]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:10:19.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:19 smithi169 ceph-mon[32759]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:19.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:19 smithi169 ceph-mon[32759]: Detected new or changed devices on smithi031 2024-04-25T01:10:19.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:19 smithi169 ceph-mon[32759]: Adjusting osd_memory_target on smithi031 to 5698M 2024-04-25T01:10:19.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:19 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:10:19.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:19 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:10:19.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:19 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:10:19.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:19 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:19.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:19 smithi031 ceph-mon[28053]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:10:19.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:19 smithi031 ceph-mon[28053]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:19.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:19 smithi031 ceph-mon[28053]: Detected new or changed devices on smithi031 2024-04-25T01:10:19.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:19 smithi031 ceph-mon[28053]: Adjusting osd_memory_target on smithi031 to 5698M 2024-04-25T01:10:19.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:19 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:10:19.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:19 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:10:19.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:19 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:10:19.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:19 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:21.632 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:21 smithi031 ceph-mon[28053]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:21.632 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:21 smithi031 ceph-mon[28053]: from='client.? 172.21.15.40:0/2995110932' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3121b58b-8c94-40b5-9d48-f56552d7c39b"}]: dispatch 2024-04-25T01:10:21.632 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:21 smithi031 ceph-mon[28053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3121b58b-8c94-40b5-9d48-f56552d7c39b"}]: dispatch 2024-04-25T01:10:21.632 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:21 smithi031 ceph-mon[28053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3121b58b-8c94-40b5-9d48-f56552d7c39b"}]': finished 2024-04-25T01:10:21.633 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:21 smithi031 ceph-mon[28053]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T01:10:21.633 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:21 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:10:21.633 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:21 smithi031 ceph-mon[28053]: from='client.? 172.21.15.40:0/1168383885' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:10:21.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:21 smithi169 ceph-mon[32759]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:21.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:21 smithi169 ceph-mon[32759]: from='client.? 172.21.15.40:0/2995110932' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3121b58b-8c94-40b5-9d48-f56552d7c39b"}]: dispatch 2024-04-25T01:10:21.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:21 smithi169 ceph-mon[32759]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3121b58b-8c94-40b5-9d48-f56552d7c39b"}]: dispatch 2024-04-25T01:10:21.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:21 smithi169 ceph-mon[32759]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3121b58b-8c94-40b5-9d48-f56552d7c39b"}]': finished 2024-04-25T01:10:21.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:21 smithi169 ceph-mon[32759]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T01:10:21.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:21 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:10:21.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:21 smithi169 ceph-mon[32759]: from='client.? 172.21.15.40:0/1168383885' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:10:21.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:21 smithi040 ceph-mon[32518]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:21.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:21 smithi040 ceph-mon[32518]: from='client.? 172.21.15.40:0/2995110932' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3121b58b-8c94-40b5-9d48-f56552d7c39b"}]: dispatch 2024-04-25T01:10:21.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:21 smithi040 ceph-mon[32518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3121b58b-8c94-40b5-9d48-f56552d7c39b"}]: dispatch 2024-04-25T01:10:21.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:21 smithi040 ceph-mon[32518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3121b58b-8c94-40b5-9d48-f56552d7c39b"}]': finished 2024-04-25T01:10:21.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:21 smithi040 ceph-mon[32518]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T01:10:21.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:21 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:10:21.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:21 smithi040 ceph-mon[32518]: from='client.? 172.21.15.40:0/1168383885' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:10:23.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:23 smithi169 ceph-mon[32759]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:23.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:23 smithi040 ceph-mon[32518]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:23.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:23 smithi031 ceph-mon[28053]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:25.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:25 smithi169 ceph-mon[32759]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:25.700 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:25 smithi040 ceph-mon[32518]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:25.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:25 smithi031 ceph-mon[28053]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:27.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:27 smithi031 ceph-mon[28053]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:27.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:27 smithi169 ceph-mon[32759]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:27.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:27 smithi040 ceph-mon[32518]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:28.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:28 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T01:10:28.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:28 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:10:28.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:28 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T01:10:28.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:28 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:10:28.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:28 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T01:10:28.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:28 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:10:29.534 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:29 smithi040 ceph-mon[32518]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:29.534 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:29 smithi040 ceph-mon[32518]: Deploying daemon osd.7 on smithi040 2024-04-25T01:10:29.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:29 smithi169 ceph-mon[32759]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:29.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:29 smithi169 ceph-mon[32759]: Deploying daemon osd.7 on smithi040 2024-04-25T01:10:29.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:29 smithi031 ceph-mon[28053]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:29.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:29 smithi031 ceph-mon[28053]: Deploying daemon osd.7 on smithi040 2024-04-25T01:10:31.683 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:31 smithi040 ceph-mon[32518]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:31.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:31 smithi169 ceph-mon[32759]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:31.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:31 smithi031 ceph-mon[28053]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:33.675 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:33 smithi040 ceph-mon[32518]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:33.675 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:33 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:33.675 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:33 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:33.675 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:33 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:10:33.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:33 smithi169 ceph-mon[32759]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:33.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:33 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:33.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:33 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:33.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:33 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:10:33.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:33 smithi031 ceph-mon[28053]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:33.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:33 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:33.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:33 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:33.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:33 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:10:35.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:35 smithi040 ceph-mon[32518]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:35.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:35 smithi169 ceph-mon[32759]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:35.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:35 smithi031 ceph-mon[28053]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:37.149 INFO:teuthology.orchestra.run.smithi040.stdout:Created osd(s) 7 on host 'smithi040' 2024-04-25T01:10:37.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:37.159+0000 7fadfeffd700 1 -- 172.21.15.40:0/1972926690 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fae08062670 con 0x7fadf40a2c20 2024-04-25T01:10:37.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:37.161+0000 7fae0fac7700 1 -- 172.21.15.40:0/1972926690 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fadf40a2c20 msgr2=0x7fadf40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:10:37.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:37.161+0000 7fae0fac7700 1 --2- 172.21.15.40:0/1972926690 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fadf40a2c20 0x7fadf40a50e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009a10 tx=0x7fadf8009190 comp rx=0 tx=0).stop 2024-04-25T01:10:37.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:37.161+0000 7fae0fac7700 1 -- 172.21.15.40:0/1972926690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae08107f90 msgr2=0x7fae08101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:10:37.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:37.161+0000 7fae0fac7700 1 --2- 172.21.15.40:0/1972926690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae08107f90 0x7fae08101220 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fae0000b9e0 tx=0x7fae0000bcf0 comp rx=0 tx=0).stop 2024-04-25T01:10:37.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:37.161+0000 7fae0fac7700 1 -- 172.21.15.40:0/1972926690 shutdown_connections 2024-04-25T01:10:37.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:37.161+0000 7fae0fac7700 1 --2- 172.21.15.40:0/1972926690 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fadf40a2c20 0x7fadf40a50e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:37.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:37.161+0000 7fae0fac7700 1 --2- 172.21.15.40:0/1972926690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae08107f90 0x7fae08101220 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:37.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:37.161+0000 7fae0fac7700 1 --2- 172.21.15.40:0/1972926690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae08102d30 0x7fae08102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:37.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:37.161+0000 7fae0fac7700 1 --2- 172.21.15.40:0/1972926690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae08105660 0x7fae08100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:37.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:37.161+0000 7fae0fac7700 1 -- 172.21.15.40:0/1972926690 >> 172.21.15.40:0/1972926690 conn(0x7fae080fc760 msgr2=0x7fae08103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:10:37.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:37.161+0000 7fae0fac7700 1 -- 172.21.15.40:0/1972926690 shutdown_connections 2024-04-25T01:10:37.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:10:37.161+0000 7fae0fac7700 1 -- 172.21.15.40:0/1972926690 wait complete. 2024-04-25T01:10:37.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:37 smithi040 ceph-mon[32518]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:37.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:37 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:37.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:37 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:37.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:37 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:10:37.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:37 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:10:37.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:37 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:37.452 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:37 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:37.452 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:37 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:37.452 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:37 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:10:37.452 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:37 smithi040 ceph-mon[32518]: from='osd.7 [v2:172.21.15.40:6816/3776085849,v1:172.21.15.40:6817/3776085849]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T01:10:37.452 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:37 smithi040 ceph-mon[32518]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T01:10:37.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:37 smithi031 ceph-mon[28053]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:37.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:37 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:37.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:37 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:37.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:37 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:10:37.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:37 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:10:37.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:37 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:37.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:37 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:37.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:37 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:37.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:37 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:10:37.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:37 smithi031 ceph-mon[28053]: from='osd.7 [v2:172.21.15.40:6816/3776085849,v1:172.21.15.40:6817/3776085849]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T01:10:37.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:37 smithi031 ceph-mon[28053]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T01:10:37.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:37 smithi169 ceph-mon[32759]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:37.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:37 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:37.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:37 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:37.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:37 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:10:37.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:37 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:10:37.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:37 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:37.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:37 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:37.698 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:37 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:37.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:37 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:10:37.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:37 smithi169 ceph-mon[32759]: from='osd.7 [v2:172.21.15.40:6816/3776085849,v1:172.21.15.40:6817/3776085849]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T01:10:37.699 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:37 smithi169 ceph-mon[32759]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T01:10:37.731 DEBUG:teuthology.orchestra.run.smithi040:osd.7> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T01:10:37.733 INFO:tasks.cephadm:Deploying osd.8 on smithi169 with /dev/vg_nvme/lv_2... 2024-04-25T01:10:37.733 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T01:10:37.983 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:10:39.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:39 smithi169 ceph-mon[32759]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:39.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:39 smithi169 ceph-mon[32759]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T01:10:39.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:39 smithi169 ceph-mon[32759]: from='osd.7 [v2:172.21.15.40:6816/3776085849,v1:172.21.15.40:6817/3776085849]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T01:10:39.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:39 smithi169 ceph-mon[32759]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T01:10:39.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:39 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:10:39.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:39 smithi169 ceph-mon[32759]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T01:10:39.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:39 smithi040 ceph-mon[32518]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:39.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:39 smithi040 ceph-mon[32518]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T01:10:39.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:39 smithi040 ceph-mon[32518]: from='osd.7 [v2:172.21.15.40:6816/3776085849,v1:172.21.15.40:6817/3776085849]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T01:10:39.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:39 smithi040 ceph-mon[32518]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T01:10:39.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:39 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:10:39.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:39 smithi040 ceph-mon[32518]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T01:10:39.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:39 smithi031 ceph-mon[28053]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:39.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:39 smithi031 ceph-mon[28053]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T01:10:39.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:39 smithi031 ceph-mon[28053]: from='osd.7 [v2:172.21.15.40:6816/3776085849,v1:172.21.15.40:6817/3776085849]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T01:10:39.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:39 smithi031 ceph-mon[28053]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T01:10:39.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:39 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:10:39.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:39 smithi031 ceph-mon[28053]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T01:10:39.691 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T01:10:39.691 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T01:10:39.691 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-04-25T01:10:39.691 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-04-25T01:10:39.692 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115671 s, 907 MB/s 2024-04-25T01:10:39.692 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-04-25T01:10:40.270 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:40 smithi169 ceph-mon[32759]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-25T01:10:40.270 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:40 smithi169 ceph-mon[32759]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T01:10:40.270 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:40 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:10:40.270 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:40 smithi169 ceph-mon[32759]: from='osd.7 ' entity='osd.7' 2024-04-25T01:10:40.270 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:40 smithi169 ceph-mon[32759]: osd.7 [v2:172.21.15.40:6816/3776085849,v1:172.21.15.40:6817/3776085849] boot 2024-04-25T01:10:40.270 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:40 smithi169 ceph-mon[32759]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T01:10:40.270 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:40 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:10:40.296 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_2 2024-04-25T01:10:40.442 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:40 smithi040 ceph-mon[32518]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-25T01:10:40.443 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:40 smithi040 ceph-mon[32518]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T01:10:40.443 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:40 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:10:40.443 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:40 smithi040 ceph-mon[32518]: from='osd.7 ' entity='osd.7' 2024-04-25T01:10:40.443 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:40 smithi040 ceph-mon[32518]: osd.7 [v2:172.21.15.40:6816/3776085849,v1:172.21.15.40:6817/3776085849] boot 2024-04-25T01:10:40.443 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:40 smithi040 ceph-mon[32518]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T01:10:40.443 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:40 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:10:40.443 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:10:40 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:10:40.004+0000 7fa7aec79700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T01:10:40.443 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:10:40 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:10:40.017+0000 7fa7a7265700 -1 osd.7 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:10:40.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:40 smithi031 ceph-mon[28053]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-25T01:10:40.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:40 smithi031 ceph-mon[28053]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T01:10:40.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:40 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:10:40.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:40 smithi031 ceph-mon[28053]: from='osd.7 ' entity='osd.7' 2024-04-25T01:10:40.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:40 smithi031 ceph-mon[28053]: osd.7 [v2:172.21.15.40:6816/3776085849,v1:172.21.15.40:6817/3776085849] boot 2024-04-25T01:10:40.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:40 smithi031 ceph-mon[28053]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T01:10:40.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:40 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:10:40.543 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:10:41.447 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:41 smithi169 ceph-mon[32759]: purged_snaps scrub starts 2024-04-25T01:10:41.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:41 smithi169 ceph-mon[32759]: purged_snaps scrub ok 2024-04-25T01:10:41.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:41 smithi169 ceph-mon[32759]: pgmap v196: 1 pgs: 1 unknown; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:41.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:41 smithi169 ceph-mon[32759]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T01:10:41.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:41 smithi169 ceph-mon[32759]: Detected new or changed devices on smithi040 2024-04-25T01:10:41.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:41.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:41.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:10:41.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:10:41.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:10:41.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:41 smithi169 ceph-mon[32759]: Adjusting osd_memory_target on smithi040 to 5697M 2024-04-25T01:10:41.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:41.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:10:41.448 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:10:41.449 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:41.449 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:41 smithi169 ceph-mon[32759]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-25T01:10:41.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:41 smithi040 ceph-mon[32518]: purged_snaps scrub starts 2024-04-25T01:10:41.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:41 smithi040 ceph-mon[32518]: purged_snaps scrub ok 2024-04-25T01:10:41.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:41 smithi040 ceph-mon[32518]: pgmap v196: 1 pgs: 1 unknown; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:41.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:41 smithi040 ceph-mon[32518]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T01:10:41.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:41 smithi040 ceph-mon[32518]: Detected new or changed devices on smithi040 2024-04-25T01:10:41.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:41.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:41.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:10:41.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:10:41.451 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:10:41.452 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:41 smithi040 ceph-mon[32518]: Adjusting osd_memory_target on smithi040 to 5697M 2024-04-25T01:10:41.452 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:41.452 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:10:41.452 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:10:41.452 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:41.452 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:41 smithi040 ceph-mon[32518]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-25T01:10:41.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:41 smithi031 ceph-mon[28053]: purged_snaps scrub starts 2024-04-25T01:10:41.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:41 smithi031 ceph-mon[28053]: purged_snaps scrub ok 2024-04-25T01:10:41.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:41 smithi031 ceph-mon[28053]: pgmap v196: 1 pgs: 1 unknown; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:10:41.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:41 smithi031 ceph-mon[28053]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T01:10:41.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:41 smithi031 ceph-mon[28053]: Detected new or changed devices on smithi040 2024-04-25T01:10:41.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:41.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:41.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:10:41.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:10:41.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:10:41.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:41 smithi031 ceph-mon[28053]: Adjusting osd_memory_target on smithi040 to 5697M 2024-04-25T01:10:41.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:41.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:10:41.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:10:41.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:41.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:41 smithi031 ceph-mon[28053]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-25T01:10:41.971 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.980+0000 7f0f1ff74700 1 -- 172.21.15.169:0/1813657813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f181005a0 msgr2=0x7f0f181009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:10:41.971 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.980+0000 7f0f1ff74700 1 --2- 172.21.15.169:0/1813657813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f181005a0 0x7f0f181009c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0f14009990 tx=0x7f0f14009ca0 comp rx=0 tx=0).stop 2024-04-25T01:10:41.971 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.980+0000 7f0f1ff74700 1 -- 172.21.15.169:0/1813657813 shutdown_connections 2024-04-25T01:10:41.971 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.980+0000 7f0f1ff74700 1 --2- 172.21.15.169:0/1813657813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f18101cd0 0x7f0f18106570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:41.971 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.980+0000 7f0f1ff74700 1 --2- 172.21.15.169:0/1813657813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f181005a0 0x7f0f181009c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:41.971 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.980+0000 7f0f1ff74700 1 --2- 172.21.15.169:0/1813657813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f18101310 0x7f0f18101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:41.972 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.980+0000 7f0f1ff74700 1 -- 172.21.15.169:0/1813657813 >> 172.21.15.169:0/1813657813 conn(0x7f0f180fbf00 msgr2=0x7f0f180fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:10:41.972 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.980+0000 7f0f1ff74700 1 -- 172.21.15.169:0/1813657813 shutdown_connections 2024-04-25T01:10:41.972 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.980+0000 7f0f1ff74700 1 -- 172.21.15.169:0/1813657813 wait complete. 2024-04-25T01:10:41.972 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.980+0000 7f0f1ff74700 1 Processor -- start 2024-04-25T01:10:41.972 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.980+0000 7f0f1ff74700 1 -- start start 2024-04-25T01:10:41.972 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.981+0000 7f0f1ff74700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f181005a0 0x7f0f1819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:10:41.972 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.981+0000 7f0f1ff74700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f18101310 0x7f0f1819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:10:41.972 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.981+0000 7f0f1ff74700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f18101cd0 0x7f0f181a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:10:41.972 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.981+0000 7f0f1ff74700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0f18114540 con 0x7f0f18101310 2024-04-25T01:10:41.972 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.981+0000 7f0f1ff74700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0f181143c0 con 0x7f0f18101cd0 2024-04-25T01:10:41.972 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.981+0000 7f0f1ff74700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0f181146c0 con 0x7f0f181005a0 2024-04-25T01:10:41.972 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.981+0000 7f0f1dd10700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f181005a0 0x7f0f1819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:10:41.972 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.981+0000 7f0f1dd10700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f181005a0 0x7f0f1819eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:41256/0 (socket says 172.21.15.169:41256) 2024-04-25T01:10:41.973 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.981+0000 7f0f1dd10700 1 -- 172.21.15.169:0/1357625594 learned_addr learned my addr 172.21.15.169:0/1357625594 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T01:10:41.973 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.981+0000 7f0f1e511700 1 --2- 172.21.15.169:0/1357625594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f18101cd0 0x7f0f181a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:10:41.973 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.981+0000 7f0f1dd10700 1 -- 172.21.15.169:0/1357625594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f18101cd0 msgr2=0x7f0f181a6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:10:41.973 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.981+0000 7f0f1dd10700 1 --2- 172.21.15.169:0/1357625594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f18101cd0 0x7f0f181a6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:41.973 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.981+0000 7f0f1dd10700 1 -- 172.21.15.169:0/1357625594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f18101310 msgr2=0x7f0f1819eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:10:41.973 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.981+0000 7f0f1dd10700 1 --2- 172.21.15.169:0/1357625594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f18101310 0x7f0f1819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:41.973 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.981+0000 7f0f1dd10700 1 -- 172.21.15.169:0/1357625594 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f181a6710 con 0x7f0f181005a0 2024-04-25T01:10:41.973 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.981+0000 7f0f1e511700 1 --2- 172.21.15.169:0/1357625594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f18101cd0 0x7f0f181a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:10:41.973 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.981+0000 7f0f1dd10700 1 --2- 172.21.15.169:0/1357625594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f181005a0 0x7f0f1819eab0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0f14004790 tx=0x7f0f14009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:10:41.973 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.981+0000 7f0f0effd700 1 -- 172.21.15.169:0/1357625594 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f14027070 con 0x7f0f181005a0 2024-04-25T01:10:41.973 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.981+0000 7f0f0effd700 1 -- 172.21.15.169:0/1357625594 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f14004210 con 0x7f0f181005a0 2024-04-25T01:10:41.973 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.981+0000 7f0f1ff74700 1 -- 172.21.15.169:0/1357625594 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f181a69a0 con 0x7f0f181005a0 2024-04-25T01:10:41.973 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.981+0000 7f0f0effd700 1 -- 172.21.15.169:0/1357625594 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1400f460 con 0x7f0f181005a0 2024-04-25T01:10:41.973 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.981+0000 7f0f1ff74700 1 -- 172.21.15.169:0/1357625594 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f181a6ed0 con 0x7f0f181005a0 2024-04-25T01:10:41.974 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.982+0000 7f0f1ff74700 1 -- 172.21.15.169:0/1357625594 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f18068f20 con 0x7f0f181005a0 2024-04-25T01:10:41.974 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.983+0000 7f0f0effd700 1 -- 172.21.15.169:0/1357625594 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0f1402a070 con 0x7f0f181005a0 2024-04-25T01:10:41.974 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.984+0000 7f0f0effd700 1 --2- 172.21.15.169:0/1357625594 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0f040a2ca0 0x7f0f040a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:10:41.975 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.984+0000 7f0f0effd700 1 -- 172.21.15.169:0/1357625594 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5426+0+0 (secure 0 0 0) 0x7f0f140cf710 con 0x7f0f181005a0 2024-04-25T01:10:41.975 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.985+0000 7f0f1d50f700 1 --2- 172.21.15.169:0/1357625594 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0f040a2ca0 0x7f0f040a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:10:41.975 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.986+0000 7f0f1d50f700 1 --2- 172.21.15.169:0/1357625594 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0f040a2ca0 0x7f0f040a5160 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0f08005950 tx=0x7f0f080058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:10:41.976 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:41.988+0000 7f0f0effd700 1 -- 172.21.15.169:0/1357625594 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0f14013070 con 0x7f0f181005a0 2024-04-25T01:10:42.179 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:10:42.190+0000 7f0f1ff74700 1 -- 172.21.15.169:0/1357625594 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0f18062670 con 0x7f0f040a2ca0 2024-04-25T01:10:42.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:42 smithi031 ceph-mon[28053]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T01:10:42.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:42 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:10:42.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:42 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:10:42.760 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:42 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:10:42.820 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:42 smithi169 ceph-mon[32759]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T01:10:42.820 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:42 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:10:42.820 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:42 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:10:42.820 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:42 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:10:42.951 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:42 smithi040 ceph-mon[32518]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T01:10:42.951 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:42 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:10:42.951 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:42 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:10:42.951 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:42 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:10:43.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:43 smithi031 ceph-mon[28053]: pgmap v200: 1 pgs: 1 unknown; 577 KiB data, 392 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:10:43.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:43 smithi031 ceph-mon[28053]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:10:43.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:43 smithi031 ceph-mon[28053]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T01:10:43.822 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:43 smithi169 ceph-mon[32759]: pgmap v200: 1 pgs: 1 unknown; 577 KiB data, 392 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:10:43.822 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:43 smithi169 ceph-mon[32759]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:10:43.822 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:43 smithi169 ceph-mon[32759]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T01:10:43.951 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:43 smithi040 ceph-mon[32518]: pgmap v200: 1 pgs: 1 unknown; 577 KiB data, 392 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:10:43.951 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:43 smithi040 ceph-mon[32518]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:10:43.951 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:43 smithi040 ceph-mon[32518]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T01:10:44.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:44 smithi031 ceph-mon[28053]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T01:10:44.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:44 smithi169 ceph-mon[32759]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T01:10:44.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:44 smithi040 ceph-mon[32518]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T01:10:45.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:45 smithi031 ceph-mon[28053]: pgmap v203: 1 pgs: 1 unknown; 577 KiB data, 393 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:10:45.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:45 smithi031 ceph-mon[28053]: from='client.? 172.21.15.169:0/2820849713' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f60407df-91f5-477a-9702-ddaef24faf82"}]: dispatch 2024-04-25T01:10:45.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:45 smithi031 ceph-mon[28053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f60407df-91f5-477a-9702-ddaef24faf82"}]: dispatch 2024-04-25T01:10:45.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:45 smithi031 ceph-mon[28053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f60407df-91f5-477a-9702-ddaef24faf82"}]': finished 2024-04-25T01:10:45.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:45 smithi031 ceph-mon[28053]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T01:10:45.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:45 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:10:45.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:45 smithi031 ceph-mon[28053]: from='client.? 172.21.15.169:0/964830929' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:10:45.948 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:45 smithi169 ceph-mon[32759]: pgmap v203: 1 pgs: 1 unknown; 577 KiB data, 393 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:10:45.948 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:45 smithi169 ceph-mon[32759]: from='client.? 172.21.15.169:0/2820849713' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f60407df-91f5-477a-9702-ddaef24faf82"}]: dispatch 2024-04-25T01:10:45.948 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:45 smithi169 ceph-mon[32759]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f60407df-91f5-477a-9702-ddaef24faf82"}]: dispatch 2024-04-25T01:10:45.948 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:45 smithi169 ceph-mon[32759]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f60407df-91f5-477a-9702-ddaef24faf82"}]': finished 2024-04-25T01:10:45.948 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:45 smithi169 ceph-mon[32759]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T01:10:45.948 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:45 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:10:45.948 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:45 smithi169 ceph-mon[32759]: from='client.? 172.21.15.169:0/964830929' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:10:45.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:45 smithi040 ceph-mon[32518]: pgmap v203: 1 pgs: 1 unknown; 577 KiB data, 393 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:10:45.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:45 smithi040 ceph-mon[32518]: from='client.? 172.21.15.169:0/2820849713' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f60407df-91f5-477a-9702-ddaef24faf82"}]: dispatch 2024-04-25T01:10:45.951 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:45 smithi040 ceph-mon[32518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f60407df-91f5-477a-9702-ddaef24faf82"}]: dispatch 2024-04-25T01:10:45.951 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:45 smithi040 ceph-mon[32518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f60407df-91f5-477a-9702-ddaef24faf82"}]': finished 2024-04-25T01:10:45.951 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:45 smithi040 ceph-mon[32518]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T01:10:45.951 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:45 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:10:45.951 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:45 smithi040 ceph-mon[32518]: from='client.? 172.21.15.169:0/964830929' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:10:46.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:46 smithi169 ceph-mon[32759]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-25T01:10:46.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:46 smithi169 ceph-mon[32759]: Cluster is now healthy 2024-04-25T01:10:46.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:46 smithi040 ceph-mon[32518]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-25T01:10:46.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:46 smithi040 ceph-mon[32518]: Cluster is now healthy 2024-04-25T01:10:47.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:46 smithi031 ceph-mon[28053]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-25T01:10:47.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:46 smithi031 ceph-mon[28053]: Cluster is now healthy 2024-04-25T01:10:47.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:47 smithi169 ceph-mon[32759]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:10:47.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:47 smithi040 ceph-mon[32518]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:10:48.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:47 smithi031 ceph-mon[28053]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:10:49.922 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:49 smithi169 ceph-mon[32759]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:10:49.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:49 smithi040 ceph-mon[32518]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:10:50.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:49 smithi031 ceph-mon[28053]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:10:51.926 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:51 smithi169 ceph-mon[32759]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:10:51.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:51 smithi040 ceph-mon[32518]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:10:52.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:51 smithi031 ceph-mon[28053]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:10:52.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:52 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T01:10:52.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:52 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:10:52.951 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:52 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T01:10:52.952 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:52 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:10:53.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:52 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T01:10:53.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:52 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:10:53.880 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:53 smithi169 ceph-mon[32759]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:10:53.881 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:53 smithi169 ceph-mon[32759]: Deploying daemon osd.8 on smithi169 2024-04-25T01:10:53.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:53 smithi040 ceph-mon[32518]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:10:53.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:53 smithi040 ceph-mon[32518]: Deploying daemon osd.8 on smithi169 2024-04-25T01:10:54.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:53 smithi031 ceph-mon[28053]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:10:54.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:53 smithi031 ceph-mon[28053]: Deploying daemon osd.8 on smithi169 2024-04-25T01:10:55.820 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:55 smithi169 ceph-mon[32759]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:10:55.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:55 smithi040 ceph-mon[32518]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:10:56.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:55 smithi031 ceph-mon[28053]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:10:57.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:57 smithi169 ceph-mon[32759]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:10:57.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:57 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:57.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:57 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:10:57.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:57 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:57.951 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:57 smithi040 ceph-mon[32518]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:10:57.951 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:57 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:57.951 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:57 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:10:57.951 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:57 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:58.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:57 smithi031 ceph-mon[28053]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:10:58.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:57 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:58.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:57 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:10:58.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:57 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:10:59.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:10:59 smithi169 ceph-mon[32759]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:10:59.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:10:59 smithi040 ceph-mon[32518]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:11:00.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:10:59 smithi031 ceph-mon[28053]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:11:00.762 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:11:00.773+0000 7f0f0effd700 1 -- 172.21.15.169:0/1357625594 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0f18062670 con 0x7f0f040a2ca0 2024-04-25T01:11:00.762 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 8 on host 'smithi169' 2024-04-25T01:11:00.764 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:11:00.775+0000 7f0f1ff74700 1 -- 172.21.15.169:0/1357625594 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0f040a2ca0 msgr2=0x7f0f040a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:00.764 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:11:00.775+0000 7f0f1ff74700 1 --2- 172.21.15.169:0/1357625594 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0f040a2ca0 0x7f0f040a5160 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0f08005950 tx=0x7f0f080058e0 comp rx=0 tx=0).stop 2024-04-25T01:11:00.764 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:11:00.775+0000 7f0f1ff74700 1 -- 172.21.15.169:0/1357625594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f181005a0 msgr2=0x7f0f1819eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:00.764 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:11:00.775+0000 7f0f1ff74700 1 --2- 172.21.15.169:0/1357625594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f181005a0 0x7f0f1819eab0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0f14004790 tx=0x7f0f14009580 comp rx=0 tx=0).stop 2024-04-25T01:11:00.764 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:11:00.775+0000 7f0f1ff74700 1 -- 172.21.15.169:0/1357625594 shutdown_connections 2024-04-25T01:11:00.764 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:11:00.775+0000 7f0f1ff74700 1 --2- 172.21.15.169:0/1357625594 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0f040a2ca0 0x7f0f040a5160 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:00.764 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:11:00.775+0000 7f0f1ff74700 1 --2- 172.21.15.169:0/1357625594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f18101cd0 0x7f0f181a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:00.764 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:11:00.775+0000 7f0f1ff74700 1 --2- 172.21.15.169:0/1357625594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f181005a0 0x7f0f1819eab0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:00.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:11:00.775+0000 7f0f1ff74700 1 --2- 172.21.15.169:0/1357625594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f18101310 0x7f0f1819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:00.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:11:00.775+0000 7f0f1ff74700 1 -- 172.21.15.169:0/1357625594 >> 172.21.15.169:0/1357625594 conn(0x7f0f180fbf00 msgr2=0x7f0f18102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:00.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:11:00.775+0000 7f0f1ff74700 1 -- 172.21.15.169:0/1357625594 shutdown_connections 2024-04-25T01:11:00.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:11:00.775+0000 7f0f1ff74700 1 -- 172.21.15.169:0/1357625594 wait complete. 2024-04-25T01:11:01.371 DEBUG:teuthology.orchestra.run.smithi169:osd.8> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T01:11:01.373 INFO:tasks.cephadm:Deploying osd.9 on smithi031 with /dev/vg_nvme/lv_1... 2024-04-25T01:11:01.373 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T01:11:01.838 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:11:01.992 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:01 smithi031 ceph-mon[28053]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:11:01.992 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:01 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:01.992 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:01 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:01.992 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:01 smithi031 ceph-mon[28053]: from='osd.8 [v2:172.21.15.169:6816/2736546122,v1:172.21.15.169:6817/2736546122]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T01:11:01.992 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:01 smithi031 ceph-mon[28053]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T01:11:01.992 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:01 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:01.992 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:01 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:02.030 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:01 smithi169 ceph-mon[32759]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:11:02.031 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:01 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:02.031 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:01 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:02.031 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:01 smithi169 ceph-mon[32759]: from='osd.8 [v2:172.21.15.169:6816/2736546122,v1:172.21.15.169:6817/2736546122]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T01:11:02.031 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:01 smithi169 ceph-mon[32759]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T01:11:02.031 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:01 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:02.031 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:01 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:02.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:01 smithi040 ceph-mon[32518]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:11:02.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:01 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:02.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:01 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:02.201 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:01 smithi040 ceph-mon[32518]: from='osd.8 [v2:172.21.15.169:6816/2736546122,v1:172.21.15.169:6817/2736546122]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T01:11:02.201 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:01 smithi040 ceph-mon[32518]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T01:11:02.201 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:01 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:02.201 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:01 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:03.000 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:02 smithi169 ceph-mon[32759]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T01:11:03.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:02 smithi169 ceph-mon[32759]: from='osd.8 [v2:172.21.15.169:6816/2736546122,v1:172.21.15.169:6817/2736546122]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T01:11:03.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:02 smithi169 ceph-mon[32759]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T01:11:03.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:02 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:11:03.001 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:02 smithi169 ceph-mon[32759]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T01:11:03.001 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:11:02 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:11:02.760+0000 7fae79a2d700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T01:11:03.002 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:11:02 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:11:02.788+0000 7fae76021700 -1 osd.8 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:11:03.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:02 smithi031 ceph-mon[28053]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T01:11:03.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:02 smithi031 ceph-mon[28053]: from='osd.8 [v2:172.21.15.169:6816/2736546122,v1:172.21.15.169:6817/2736546122]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T01:11:03.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:02 smithi031 ceph-mon[28053]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T01:11:03.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:02 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:11:03.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:02 smithi031 ceph-mon[28053]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T01:11:03.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:02 smithi040 ceph-mon[32518]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T01:11:03.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:02 smithi040 ceph-mon[32518]: from='osd.8 [v2:172.21.15.169:6816/2736546122,v1:172.21.15.169:6817/2736546122]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T01:11:03.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:02 smithi040 ceph-mon[32518]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T01:11:03.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:02 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:11:03.201 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:02 smithi040 ceph-mon[32518]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T01:11:03.794 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:03 smithi031 ceph-mon[28053]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:11:03.794 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:03 smithi031 ceph-mon[28053]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T01:11:03.794 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:03 smithi031 ceph-mon[28053]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T01:11:03.794 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:03 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:11:03.795 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:03 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:11:04.058 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:03 smithi169 ceph-mon[32759]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:11:04.058 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:03 smithi169 ceph-mon[32759]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T01:11:04.058 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:03 smithi169 ceph-mon[32759]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T01:11:04.059 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:03 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:11:04.059 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:03 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:11:04.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:03 smithi040 ceph-mon[32518]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:11:04.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:03 smithi040 ceph-mon[32518]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T01:11:04.201 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:03 smithi040 ceph-mon[32518]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T01:11:04.201 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:03 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:11:04.201 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:03 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:11:04.838 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T01:11:04.838 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T01:11:04.838 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2024-04-25T01:11:04.838 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2024-04-25T01:11:04.839 INFO:teuthology.orchestra.run.smithi031.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.038496 s, 272 MB/s 2024-04-25T01:11:04.839 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 2024-04-25T01:11:04.839 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2024-04-25T01:11:05.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:04 smithi169 ceph-mon[32759]: purged_snaps scrub starts 2024-04-25T01:11:05.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:04 smithi169 ceph-mon[32759]: purged_snaps scrub ok 2024-04-25T01:11:05.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:04 smithi169 ceph-mon[32759]: osd.8 [v2:172.21.15.169:6816/2736546122,v1:172.21.15.169:6817/2736546122] boot 2024-04-25T01:11:05.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:04 smithi169 ceph-mon[32759]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T01:11:05.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:04 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:11:05.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:04 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:05.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:04 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:05.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:04 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:11:05.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:04 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:11:05.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:04 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:11:05.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:04 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:05.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:04 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:11:05.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:04 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:11:05.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:04 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:05.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:04 smithi169 ceph-mon[32759]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T01:11:05.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:04 smithi040 ceph-mon[32518]: purged_snaps scrub starts 2024-04-25T01:11:05.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:04 smithi040 ceph-mon[32518]: purged_snaps scrub ok 2024-04-25T01:11:05.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:04 smithi040 ceph-mon[32518]: osd.8 [v2:172.21.15.169:6816/2736546122,v1:172.21.15.169:6817/2736546122] boot 2024-04-25T01:11:05.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:04 smithi040 ceph-mon[32518]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T01:11:05.201 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:04 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:11:05.201 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:04 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:05.201 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:04 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:05.201 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:04 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:11:05.201 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:04 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:11:05.201 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:04 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:11:05.201 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:04 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:05.201 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:04 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:11:05.201 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:04 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:11:05.201 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:04 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:05.201 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:04 smithi040 ceph-mon[32518]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T01:11:05.258 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:04 smithi031 ceph-mon[28053]: purged_snaps scrub starts 2024-04-25T01:11:05.258 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:04 smithi031 ceph-mon[28053]: purged_snaps scrub ok 2024-04-25T01:11:05.258 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:04 smithi031 ceph-mon[28053]: osd.8 [v2:172.21.15.169:6816/2736546122,v1:172.21.15.169:6817/2736546122] boot 2024-04-25T01:11:05.258 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:04 smithi031 ceph-mon[28053]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T01:11:05.258 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:04 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:11:05.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:04 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:05.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:04 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:05.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:04 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:11:05.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:04 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:11:05.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:04 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:11:05.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:04 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:05.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:04 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:11:05.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:04 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:11:05.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:04 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:05.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:04 smithi031 ceph-mon[28053]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T01:11:05.778 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_1 2024-04-25T01:11:06.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:05 smithi031 ceph-mon[28053]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:11:06.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:05 smithi031 ceph-mon[28053]: Detected new or changed devices on smithi169 2024-04-25T01:11:06.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:05 smithi031 ceph-mon[28053]: Adjusting osd_memory_target on smithi169 to 5696M 2024-04-25T01:11:06.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:05 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:11:06.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:05 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:11:06.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:05 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:11:06.009 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:05 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:06.009 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:05 smithi031 ceph-mon[28053]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T01:11:06.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:05 smithi169 ceph-mon[32759]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:11:06.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:05 smithi169 ceph-mon[32759]: Detected new or changed devices on smithi169 2024-04-25T01:11:06.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:05 smithi169 ceph-mon[32759]: Adjusting osd_memory_target on smithi169 to 5696M 2024-04-25T01:11:06.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:05 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:11:06.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:05 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:11:06.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:05 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:11:06.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:05 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:06.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:05 smithi169 ceph-mon[32759]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T01:11:06.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:05 smithi040 ceph-mon[32518]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:11:06.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:05 smithi040 ceph-mon[32518]: Detected new or changed devices on smithi169 2024-04-25T01:11:06.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:05 smithi040 ceph-mon[32518]: Adjusting osd_memory_target on smithi169 to 5696M 2024-04-25T01:11:06.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:05 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:11:06.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:05 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:11:06.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:05 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:11:06.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:05 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:06.201 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:05 smithi040 ceph-mon[32518]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T01:11:06.237 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:11:07.867 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:07 smithi031 ceph-mon[28053]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:07.868 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:07 smithi031 ceph-mon[28053]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T01:11:08.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:07 smithi169 ceph-mon[32759]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:08.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:07 smithi169 ceph-mon[32759]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T01:11:08.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:07 smithi040 ceph-mon[32518]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:08.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:07 smithi040 ceph-mon[32518]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T01:11:08.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.434+0000 7efc35c00700 1 -- 172.21.15.31:0/2282708252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc301034a0 msgr2=0x7efc30103900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:08.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.434+0000 7efc35c00700 1 --2- 172.21.15.31:0/2282708252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc301034a0 0x7efc30103900 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efc180099c0 tx=0x7efc18009cd0 comp rx=0 tx=0).stop 2024-04-25T01:11:08.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.435+0000 7efc35c00700 1 -- 172.21.15.31:0/2282708252 shutdown_connections 2024-04-25T01:11:08.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.435+0000 7efc35c00700 1 --2- 172.21.15.31:0/2282708252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc30103e40 0x7efc3010a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:08.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.435+0000 7efc35c00700 1 --2- 172.21.15.31:0/2282708252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc301022a0 0x7efc301026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:08.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.435+0000 7efc35c00700 1 --2- 172.21.15.31:0/2282708252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc301034a0 0x7efc30103900 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:08.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.435+0000 7efc35c00700 1 -- 172.21.15.31:0/2282708252 >> 172.21.15.31:0/2282708252 conn(0x7efc300fd800 msgr2=0x7efc300ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:08.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.435+0000 7efc35c00700 1 -- 172.21.15.31:0/2282708252 shutdown_connections 2024-04-25T01:11:08.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.435+0000 7efc35c00700 1 -- 172.21.15.31:0/2282708252 wait complete. 2024-04-25T01:11:08.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.436+0000 7efc35c00700 1 Processor -- start 2024-04-25T01:11:08.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.436+0000 7efc35c00700 1 -- start start 2024-04-25T01:11:08.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.436+0000 7efc35c00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc301022a0 0x7efc3019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:08.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.436+0000 7efc35c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc301034a0 0x7efc3019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:08.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.436+0000 7efc35c00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc30103e40 0x7efc301a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:08.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.436+0000 7efc35c00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efc3010dec0 con 0x7efc301022a0 2024-04-25T01:11:08.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.436+0000 7efc35c00700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efc3010dd40 con 0x7efc30103e40 2024-04-25T01:11:08.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.436+0000 7efc35c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efc3010e040 con 0x7efc301034a0 2024-04-25T01:11:08.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.437+0000 7efc2ea00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc301022a0 0x7efc3019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:08.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.437+0000 7efc2ea00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc301022a0 0x7efc3019a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41624/0 (socket says 172.21.15.31:41624) 2024-04-25T01:11:08.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.437+0000 7efc26000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc301034a0 0x7efc3019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:08.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.437+0000 7efc2ea00700 1 -- 172.21.15.31:0/2067823037 learned_addr learned my addr 172.21.15.31:0/2067823037 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:11:08.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.437+0000 7efc2f400700 1 --2- 172.21.15.31:0/2067823037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc30103e40 0x7efc301a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:08.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.437+0000 7efc2ea00700 1 -- 172.21.15.31:0/2067823037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc301034a0 msgr2=0x7efc3019ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:08.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.437+0000 7efc2ea00700 1 --2- 172.21.15.31:0/2067823037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc301034a0 0x7efc3019ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:08.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.437+0000 7efc2ea00700 1 -- 172.21.15.31:0/2067823037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc30103e40 msgr2=0x7efc301a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:08.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.437+0000 7efc2ea00700 1 --2- 172.21.15.31:0/2067823037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc30103e40 0x7efc301a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:08.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.437+0000 7efc2ea00700 1 -- 172.21.15.31:0/2067823037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc301a23b0 con 0x7efc301022a0 2024-04-25T01:11:08.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.437+0000 7efc2ea00700 1 --2- 172.21.15.31:0/2067823037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc301022a0 0x7efc3019a750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efc20007f10 tx=0x7efc2000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:08.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.438+0000 7efc27e00700 1 -- 172.21.15.31:0/2067823037 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2001a070 con 0x7efc301022a0 2024-04-25T01:11:08.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.438+0000 7efc27e00700 1 -- 172.21.15.31:0/2067823037 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc2000dd20 con 0x7efc301022a0 2024-04-25T01:11:08.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.438+0000 7efc35c00700 1 -- 172.21.15.31:0/2067823037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc301a26a0 con 0x7efc301022a0 2024-04-25T01:11:08.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.438+0000 7efc27e00700 1 -- 172.21.15.31:0/2067823037 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc20014690 con 0x7efc301022a0 2024-04-25T01:11:08.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.438+0000 7efc35c00700 1 -- 172.21.15.31:0/2067823037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc301a2c30 con 0x7efc301022a0 2024-04-25T01:11:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.439+0000 7efc35c00700 1 -- 172.21.15.31:0/2067823037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc30068f20 con 0x7efc301022a0 2024-04-25T01:11:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.441+0000 7efc27e00700 1 -- 172.21.15.31:0/2067823037 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efc2001d070 con 0x7efc301022a0 2024-04-25T01:11:08.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.442+0000 7efc27e00700 1 --2- 172.21.15.31:0/2067823037 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7efc140a2ca0 0x7efc140a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:08.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.443+0000 7efc27e00700 1 -- 172.21.15.31:0/2067823037 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5829+0+0 (secure 0 0 0) 0x7efc200ce080 con 0x7efc301022a0 2024-04-25T01:11:08.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.443+0000 7efc26000700 1 --2- 172.21.15.31:0/2067823037 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7efc140a2ca0 0x7efc140a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:08.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.443+0000 7efc26000700 1 --2- 172.21.15.31:0/2067823037 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7efc140a2ca0 0x7efc140a5160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efc18000c00 tx=0x7efc18005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:08.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.447+0000 7efc27e00700 1 -- 172.21.15.31:0/2067823037 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efc20010070 con 0x7efc301022a0 2024-04-25T01:11:08.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:08.690+0000 7efc35c00700 1 -- 172.21.15.31:0/2067823037 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7efc30075b80 con 0x7efc140a2ca0 2024-04-25T01:11:08.800 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:08 smithi031 ceph-mon[28053]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T01:11:08.801 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:08 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:11:08.801 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:08 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:11:09.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:08 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:11:09.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:08 smithi169 ceph-mon[32759]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T01:11:09.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:08 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:11:09.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:08 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:11:09.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:08 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:11:09.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:08 smithi040 ceph-mon[32518]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T01:11:09.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:08 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:11:09.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:08 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:11:09.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:08 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:11:09.974 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:09 smithi031 ceph-mon[28053]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:09.974 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:09 smithi031 ceph-mon[28053]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:11:10.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:09 smithi169 ceph-mon[32759]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:10.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:09 smithi169 ceph-mon[32759]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:11:10.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:09 smithi040 ceph-mon[32518]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:10.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:09 smithi040 ceph-mon[32518]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:11:12.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:11 smithi169 ceph-mon[32759]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:12.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:11 smithi040 ceph-mon[32518]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:12.258 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:11 smithi031 ceph-mon[28053]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:13.149 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:12 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2076549829' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7506a407-1960-49bc-9e38-7f2a8bf996ed"}]: dispatch 2024-04-25T01:11:13.149 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:12 smithi031 ceph-mon[28053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7506a407-1960-49bc-9e38-7f2a8bf996ed"}]: dispatch 2024-04-25T01:11:13.149 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:12 smithi031 ceph-mon[28053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7506a407-1960-49bc-9e38-7f2a8bf996ed"}]': finished 2024-04-25T01:11:13.149 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:12 smithi031 ceph-mon[28053]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T01:11:13.149 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:12 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:11:13.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:12 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2076549829' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7506a407-1960-49bc-9e38-7f2a8bf996ed"}]: dispatch 2024-04-25T01:11:13.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:12 smithi169 ceph-mon[32759]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7506a407-1960-49bc-9e38-7f2a8bf996ed"}]: dispatch 2024-04-25T01:11:13.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:12 smithi169 ceph-mon[32759]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7506a407-1960-49bc-9e38-7f2a8bf996ed"}]': finished 2024-04-25T01:11:13.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:12 smithi169 ceph-mon[32759]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T01:11:13.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:12 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:11:13.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:12 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2076549829' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7506a407-1960-49bc-9e38-7f2a8bf996ed"}]: dispatch 2024-04-25T01:11:13.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:12 smithi040 ceph-mon[32518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7506a407-1960-49bc-9e38-7f2a8bf996ed"}]: dispatch 2024-04-25T01:11:13.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:12 smithi040 ceph-mon[32518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7506a407-1960-49bc-9e38-7f2a8bf996ed"}]': finished 2024-04-25T01:11:13.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:12 smithi040 ceph-mon[32518]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T01:11:13.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:12 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:11:14.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:13 smithi169 ceph-mon[32759]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-25T01:11:14.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:13 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/687612960' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:11:14.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:13 smithi040 ceph-mon[32518]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-25T01:11:14.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:13 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/687612960' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:11:14.258 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:13 smithi031 ceph-mon[28053]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-25T01:11:14.258 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:13 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/687612960' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:11:16.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:15 smithi169 ceph-mon[32759]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T01:11:16.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:15 smithi040 ceph-mon[32518]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T01:11:16.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:15 smithi031 ceph-mon[28053]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T01:11:18.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:17 smithi169 ceph-mon[32759]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T01:11:18.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:17 smithi040 ceph-mon[32518]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T01:11:18.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:17 smithi031 ceph-mon[28053]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T01:11:20.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:19 smithi169 ceph-mon[32759]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T01:11:20.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:19 smithi040 ceph-mon[32518]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T01:11:20.248 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:19 smithi031 ceph-mon[28053]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T01:11:22.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:21 smithi169 ceph-mon[32759]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T01:11:22.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:21 smithi040 ceph-mon[32518]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T01:11:22.212 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:21 smithi031 ceph-mon[28053]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T01:11:23.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:22 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T01:11:23.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:22 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:11:23.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:22 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T01:11:23.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:22 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:11:23.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:22 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T01:11:23.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:22 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:11:23.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:23 smithi031 ceph-mon[28053]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:23.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:23 smithi031 ceph-mon[28053]: Deploying daemon osd.9 on smithi031 2024-04-25T01:11:24.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:23 smithi169 ceph-mon[32759]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:24.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:23 smithi169 ceph-mon[32759]: Deploying daemon osd.9 on smithi031 2024-04-25T01:11:24.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:23 smithi040 ceph-mon[32518]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:24.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:23 smithi040 ceph-mon[32518]: Deploying daemon osd.9 on smithi031 2024-04-25T01:11:26.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:25 smithi169 ceph-mon[32759]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:26.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:25 smithi040 ceph-mon[32518]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:26.252 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:25 smithi031 ceph-mon[28053]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:27.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:26 smithi169 ceph-mon[32759]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:27.254 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:26 smithi040 ceph-mon[32518]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:27.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:26 smithi031 ceph-mon[28053]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:29.447 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:29 smithi169 ceph-mon[32759]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:29.450 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:29 smithi040 ceph-mon[32518]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:29.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:29 smithi031 ceph-mon[28053]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:32.106 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:31 smithi031 ceph-mon[28053]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:32.107 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:31 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:32.107 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:31 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:32.107 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:31 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:11:32.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:31 smithi169 ceph-mon[32759]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:32.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:31 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:32.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:31 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:32.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:31 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:11:32.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:31 smithi040 ceph-mon[32518]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:32.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:31 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:32.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:31 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:32.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:31 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:11:34.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:33 smithi169 ceph-mon[32759]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:34.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:33 smithi040 ceph-mon[32518]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:34.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:33 smithi031 ceph-mon[28053]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:35.477 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 9 on host 'smithi031' 2024-04-25T01:11:35.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:35.488+0000 7efc27e00700 1 -- 172.21.15.31:0/2067823037 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efc30075b80 con 0x7efc140a2ca0 2024-04-25T01:11:35.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:35.491+0000 7efc35c00700 1 -- 172.21.15.31:0/2067823037 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7efc140a2ca0 msgr2=0x7efc140a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:35.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:35.491+0000 7efc35c00700 1 --2- 172.21.15.31:0/2067823037 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7efc140a2ca0 0x7efc140a5160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efc18000c00 tx=0x7efc18005fb0 comp rx=0 tx=0).stop 2024-04-25T01:11:35.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:35.492+0000 7efc35c00700 1 -- 172.21.15.31:0/2067823037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc301022a0 msgr2=0x7efc3019a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:35.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:35.492+0000 7efc35c00700 1 --2- 172.21.15.31:0/2067823037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc301022a0 0x7efc3019a750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efc20007f10 tx=0x7efc2000d3b0 comp rx=0 tx=0).stop 2024-04-25T01:11:35.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:35.492+0000 7efc35c00700 1 -- 172.21.15.31:0/2067823037 shutdown_connections 2024-04-25T01:11:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:35.492+0000 7efc35c00700 1 --2- 172.21.15.31:0/2067823037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc301022a0 0x7efc3019a750 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:35.492+0000 7efc35c00700 1 --2- 172.21.15.31:0/2067823037 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7efc140a2ca0 0x7efc140a5160 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:35.492+0000 7efc35c00700 1 --2- 172.21.15.31:0/2067823037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc30103e40 0x7efc301a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:35.492+0000 7efc35c00700 1 --2- 172.21.15.31:0/2067823037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc301034a0 0x7efc3019ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:35.492+0000 7efc35c00700 1 -- 172.21.15.31:0/2067823037 >> 172.21.15.31:0/2067823037 conn(0x7efc300fd800 msgr2=0x7efc30074460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:35.492+0000 7efc35c00700 1 -- 172.21.15.31:0/2067823037 shutdown_connections 2024-04-25T01:11:35.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:11:35.493+0000 7efc35c00700 1 -- 172.21.15.31:0/2067823037 wait complete. 2024-04-25T01:11:35.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:35 smithi031 ceph-mon[28053]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:35.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:35 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:35.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:35 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:35.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:35 smithi031 ceph-mon[28053]: from='osd.9 [v2:172.21.15.31:6826/1187207880,v1:172.21.15.31:6827/1187207880]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T01:11:35.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:35 smithi031 ceph-mon[28053]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T01:11:35.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:35 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:35.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:35 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:35.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:35 smithi169 ceph-mon[32759]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:35.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:35 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:35.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:35 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:35.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:35 smithi169 ceph-mon[32759]: from='osd.9 [v2:172.21.15.31:6826/1187207880,v1:172.21.15.31:6827/1187207880]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T01:11:35.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:35 smithi169 ceph-mon[32759]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T01:11:35.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:35 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:35.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:35 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:35.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:35 smithi040 ceph-mon[32518]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:35.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:35 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:35.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:35 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:35.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:35 smithi040 ceph-mon[32518]: from='osd.9 [v2:172.21.15.31:6826/1187207880,v1:172.21.15.31:6827/1187207880]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T01:11:35.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:35 smithi040 ceph-mon[32518]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T01:11:35.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:35 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:35.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:35 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:36.222 DEBUG:teuthology.orchestra.run.smithi031:osd.9> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T01:11:36.230 INFO:tasks.cephadm:Deploying osd.10 on smithi040 with /dev/vg_nvme/lv_1... 2024-04-25T01:11:36.230 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T01:11:36.486 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T01:11:36.608 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:36 smithi040 ceph-mon[32518]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T01:11:36.609 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:36 smithi040 ceph-mon[32518]: from='osd.9 [v2:172.21.15.31:6826/1187207880,v1:172.21.15.31:6827/1187207880]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T01:11:36.609 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:36 smithi040 ceph-mon[32518]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T01:11:36.609 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:36 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:11:36.609 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:36 smithi040 ceph-mon[32518]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T01:11:36.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:36 smithi031 ceph-mon[28053]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T01:11:36.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:36 smithi031 ceph-mon[28053]: from='osd.9 [v2:172.21.15.31:6826/1187207880,v1:172.21.15.31:6827/1187207880]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T01:11:36.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:36 smithi031 ceph-mon[28053]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T01:11:36.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:36 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:11:36.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:36 smithi031 ceph-mon[28053]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T01:11:36.758 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:11:36 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:11:36.518+0000 7fc705200700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T01:11:36.758 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:11:36 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:11:36.553+0000 7fc700a00700 -1 osd.9 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:11:36.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:36 smithi169 ceph-mon[32759]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T01:11:36.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:36 smithi169 ceph-mon[32759]: from='osd.9 [v2:172.21.15.31:6826/1187207880,v1:172.21.15.31:6827/1187207880]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T01:11:36.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:36 smithi169 ceph-mon[32759]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T01:11:36.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:36 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:11:36.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:36 smithi169 ceph-mon[32759]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T01:11:37.661 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:37 smithi040 ceph-mon[32518]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:37.661 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:37 smithi040 ceph-mon[32518]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T01:11:37.662 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:37 smithi040 ceph-mon[32518]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T01:11:37.662 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:37 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:11:37.662 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:37 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:11:37.676 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:37 smithi031 ceph-mon[28053]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:37.676 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:37 smithi031 ceph-mon[28053]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T01:11:37.676 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:37 smithi031 ceph-mon[28053]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T01:11:37.676 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:37 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:11:37.676 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:37 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:11:37.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:37 smithi169 ceph-mon[32759]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:37.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:37 smithi169 ceph-mon[32759]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T01:11:37.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:37 smithi169 ceph-mon[32759]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T01:11:37.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:37 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:11:37.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:37 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:11:38.501 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T01:11:38.502 INFO:teuthology.orchestra.run.smithi040.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T01:11:38.502 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10+0 records in 2024-04-25T01:11:38.502 INFO:teuthology.orchestra.run.smithi040.stderr:10+0 records out 2024-04-25T01:11:38.502 INFO:teuthology.orchestra.run.smithi040.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011187 s, 937 MB/s 2024-04-25T01:11:38.502 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping successful for: 2024-04-25T01:11:38.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:38 smithi031 ceph-mon[28053]: purged_snaps scrub starts 2024-04-25T01:11:38.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:38 smithi031 ceph-mon[28053]: purged_snaps scrub ok 2024-04-25T01:11:38.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:38 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:11:38.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:38 smithi031 ceph-mon[28053]: osd.9 [v2:172.21.15.31:6826/1187207880,v1:172.21.15.31:6827/1187207880] boot 2024-04-25T01:11:38.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:38 smithi031 ceph-mon[28053]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T01:11:38.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:38 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:11:38.811 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:38 smithi040 ceph-mon[32518]: purged_snaps scrub starts 2024-04-25T01:11:38.812 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:38 smithi040 ceph-mon[32518]: purged_snaps scrub ok 2024-04-25T01:11:38.812 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:38 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:11:38.812 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:38 smithi040 ceph-mon[32518]: osd.9 [v2:172.21.15.31:6826/1187207880,v1:172.21.15.31:6827/1187207880] boot 2024-04-25T01:11:38.812 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:38 smithi040 ceph-mon[32518]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T01:11:38.812 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:38 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:11:38.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:38 smithi169 ceph-mon[32759]: purged_snaps scrub starts 2024-04-25T01:11:38.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:38 smithi169 ceph-mon[32759]: purged_snaps scrub ok 2024-04-25T01:11:38.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:38 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:11:38.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:38 smithi169 ceph-mon[32759]: osd.9 [v2:172.21.15.31:6826/1187207880,v1:172.21.15.31:6827/1187207880] boot 2024-04-25T01:11:38.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:38 smithi169 ceph-mon[32759]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T01:11:38.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:38 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:11:39.112 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi040:vg_nvme/lv_1 2024-04-25T01:11:39.360 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T01:11:39.780 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:39 smithi040 ceph-mon[32518]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:39.780 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:39 smithi040 ceph-mon[32518]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T01:11:39.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:39 smithi169 ceph-mon[32759]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:39.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:39 smithi169 ceph-mon[32759]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T01:11:40.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:39 smithi031 ceph-mon[28053]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:11:40.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:39 smithi031 ceph-mon[28053]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T01:11:40.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.903+0000 7f0191223700 1 -- 172.21.15.40:0/3617006164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f018c103490 msgr2=0x7f018c1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:40.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.903+0000 7f0191223700 1 --2- 172.21.15.40:0/3617006164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f018c103490 0x7f018c1038f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f01800099c0 tx=0x7f0180009cd0 comp rx=0 tx=0).stop 2024-04-25T01:11:40.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.903+0000 7f0191223700 1 -- 172.21.15.40:0/3617006164 shutdown_connections 2024-04-25T01:11:40.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.903+0000 7f0191223700 1 --2- 172.21.15.40:0/3617006164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f018c102290 0x7f018c1026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:40.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.903+0000 7f0191223700 1 --2- 172.21.15.40:0/3617006164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f018c103e30 0x7f018c10a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:40.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.903+0000 7f0191223700 1 --2- 172.21.15.40:0/3617006164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f018c103490 0x7f018c1038f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:40.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.903+0000 7f0191223700 1 -- 172.21.15.40:0/3617006164 >> 172.21.15.40:0/3617006164 conn(0x7f018c0fd830 msgr2=0x7f018c0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:40.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.903+0000 7f0191223700 1 -- 172.21.15.40:0/3617006164 shutdown_connections 2024-04-25T01:11:40.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.903+0000 7f0191223700 1 -- 172.21.15.40:0/3617006164 wait complete. 2024-04-25T01:11:40.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.904+0000 7f0191223700 1 Processor -- start 2024-04-25T01:11:40.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.904+0000 7f0191223700 1 -- start start 2024-04-25T01:11:40.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.904+0000 7f0191223700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f018c102290 0x7f018c198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:40.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.904+0000 7f0191223700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f018c103490 0x7f018c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:40.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.905+0000 7f0191223700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f018c103e30 0x7f018c19fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:40.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.905+0000 7f0191223700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f018c10df10 con 0x7f018c103490 2024-04-25T01:11:40.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.905+0000 7f0191223700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f018c10dd90 con 0x7f018c103e30 2024-04-25T01:11:40.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.905+0000 7f0191223700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f018c10e090 con 0x7f018c102290 2024-04-25T01:11:40.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.905+0000 7f018b59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f018c103e30 0x7f018c19fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:40.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.905+0000 7f018b59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f018c103e30 0x7f018c19fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43008/0 (socket says 172.21.15.40:43008) 2024-04-25T01:11:40.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.905+0000 7f018b59e700 1 -- 172.21.15.40:0/1074738381 learned_addr learned my addr 172.21.15.40:0/1074738381 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-25T01:11:40.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.905+0000 7f018ad9d700 1 --2- 172.21.15.40:0/1074738381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f018c102290 0x7f018c198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:40.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.905+0000 7f018b59e700 1 -- 172.21.15.40:0/1074738381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f018c102290 msgr2=0x7f018c198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:40.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.905+0000 7f018b59e700 1 --2- 172.21.15.40:0/1074738381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f018c102290 0x7f018c198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:40.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.905+0000 7f018b59e700 1 -- 172.21.15.40:0/1074738381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f018c103490 msgr2=0x7f018c198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:40.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.905+0000 7f018b59e700 1 --2- 172.21.15.40:0/1074738381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f018c103490 0x7f018c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:40.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.905+0000 7f018b59e700 1 -- 172.21.15.40:0/1074738381 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f018c1a0160 con 0x7f018c103e30 2024-04-25T01:11:40.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.905+0000 7f018b59e700 1 --2- 172.21.15.40:0/1074738381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f018c103e30 0x7f018c19fac0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0184007ee0 tx=0x7f018400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:40.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.905+0000 7f0173fff700 1 -- 172.21.15.40:0/1074738381 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018400dbd0 con 0x7f018c103e30 2024-04-25T01:11:40.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.905+0000 7f0191223700 1 -- 172.21.15.40:0/1074738381 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f018c0744c0 con 0x7f018c103e30 2024-04-25T01:11:40.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.905+0000 7f0173fff700 1 -- 172.21.15.40:0/1074738381 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f018400dd70 con 0x7f018c103e30 2024-04-25T01:11:40.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.905+0000 7f0173fff700 1 -- 172.21.15.40:0/1074738381 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0184005a40 con 0x7f018c103e30 2024-04-25T01:11:40.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.906+0000 7f0191223700 1 -- 172.21.15.40:0/1074738381 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f018c074a50 con 0x7f018c103e30 2024-04-25T01:11:40.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.906+0000 7f0191223700 1 -- 172.21.15.40:0/1074738381 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f018c068f20 con 0x7f018c103e30 2024-04-25T01:11:40.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.908+0000 7f0173fff700 1 -- 172.21.15.40:0/1074738381 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f018401a020 con 0x7f018c103e30 2024-04-25T01:11:40.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.908+0000 7f0173fff700 1 --2- 172.21.15.40:0/1074738381 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f01740a2c20 0x7f01740a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:40.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.908+0000 7f0173fff700 1 -- 172.21.15.40:0/1074738381 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6261+0+0 (secure 0 0 0) 0x7f01840cdf00 con 0x7f018c103e30 2024-04-25T01:11:40.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.909+0000 7f018ad9d700 1 --2- 172.21.15.40:0/1074738381 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f01740a2c20 0x7f01740a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:40.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.910+0000 7f018ad9d700 1 --2- 172.21.15.40:0/1074738381 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f01740a2c20 0x7f01740a50e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f018c1032f0 tx=0x7f017c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:40.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:40.913+0000 7f0173fff700 1 -- 172.21.15.40:0/1074738381 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0184010070 con 0x7f018c103e30 2024-04-25T01:11:41.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:11:41.127+0000 7f0191223700 1 -- 172.21.15.40:0/1074738381 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f018c075ea0 con 0x7f01740a2c20 2024-04-25T01:11:41.770 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:41 smithi040 ceph-mon[32518]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:41.770 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:41.770 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:41.771 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:11:41.771 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:11:41.771 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:11:41.771 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:11:41.771 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:41.771 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:11:41.771 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:11:41.771 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:11:41.771 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:11:41.771 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:11:41.771 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:41.771 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:11:41.771 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:11:41.772 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:11:41.772 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:41 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:41.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:41 smithi169 ceph-mon[32759]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:41.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:41.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:41.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:11:41.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:11:41.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:11:41.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:11:41.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:41.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:11:41.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:11:41.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:11:41.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:11:41.948 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:11:41.948 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:41.948 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:11:41.948 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:11:41.948 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:11:41.948 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:41 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:42.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:41 smithi031 ceph-mon[28053]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:42.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:42.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:42.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:11:42.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:11:42.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:11:42.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:11:42.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:42.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:11:42.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:11:42.009 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:11:42.009 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:11:42.009 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:11:42.009 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:42.009 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:11:42.009 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:11:42.009 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:11:42.009 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:41 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:42.801 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:42 smithi040 ceph-mon[32518]: Detected new or changed devices on smithi031 2024-04-25T01:11:42.801 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:42 smithi040 ceph-mon[32518]: Adjusting osd_memory_target on smithi031 to 4273M 2024-04-25T01:11:42.801 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:42 smithi040 ceph-mon[32518]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:11:42.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:42 smithi169 ceph-mon[32759]: Detected new or changed devices on smithi031 2024-04-25T01:11:42.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:42 smithi169 ceph-mon[32759]: Adjusting osd_memory_target on smithi031 to 4273M 2024-04-25T01:11:42.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:42 smithi169 ceph-mon[32759]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:11:43.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:42 smithi031 ceph-mon[28053]: Detected new or changed devices on smithi031 2024-04-25T01:11:43.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:42 smithi031 ceph-mon[28053]: Adjusting osd_memory_target on smithi031 to 4273M 2024-04-25T01:11:43.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:42 smithi031 ceph-mon[28053]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:11:43.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:43 smithi169 ceph-mon[32759]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:43.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:43 smithi040 ceph-mon[32518]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:44.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:43 smithi031 ceph-mon[28053]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:44.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:44 smithi031 ceph-mon[28053]: from='client.? 172.21.15.40:0/39807001' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "831c48e2-b0d0-451f-959b-228e927542cd"}]: dispatch 2024-04-25T01:11:44.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:44 smithi031 ceph-mon[28053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "831c48e2-b0d0-451f-959b-228e927542cd"}]: dispatch 2024-04-25T01:11:44.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:44 smithi031 ceph-mon[28053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "831c48e2-b0d0-451f-959b-228e927542cd"}]': finished 2024-04-25T01:11:44.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:44 smithi031 ceph-mon[28053]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T01:11:44.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:44 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:11:44.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:44 smithi031 ceph-mon[28053]: from='client.? 172.21.15.40:0/1880612560' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:11:44.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:44 smithi169 ceph-mon[32759]: from='client.? 172.21.15.40:0/39807001' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "831c48e2-b0d0-451f-959b-228e927542cd"}]: dispatch 2024-04-25T01:11:44.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:44 smithi169 ceph-mon[32759]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "831c48e2-b0d0-451f-959b-228e927542cd"}]: dispatch 2024-04-25T01:11:44.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:44 smithi169 ceph-mon[32759]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "831c48e2-b0d0-451f-959b-228e927542cd"}]': finished 2024-04-25T01:11:44.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:44 smithi169 ceph-mon[32759]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T01:11:44.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:44 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:11:44.950 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:44 smithi169 ceph-mon[32759]: from='client.? 172.21.15.40:0/1880612560' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:11:44.951 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:44 smithi040 ceph-mon[32518]: from='client.? 172.21.15.40:0/39807001' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "831c48e2-b0d0-451f-959b-228e927542cd"}]: dispatch 2024-04-25T01:11:44.951 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:44 smithi040 ceph-mon[32518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "831c48e2-b0d0-451f-959b-228e927542cd"}]: dispatch 2024-04-25T01:11:44.951 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:44 smithi040 ceph-mon[32518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "831c48e2-b0d0-451f-959b-228e927542cd"}]': finished 2024-04-25T01:11:44.951 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:44 smithi040 ceph-mon[32518]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T01:11:44.951 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:44 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:11:44.951 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:44 smithi040 ceph-mon[32518]: from='client.? 172.21.15.40:0/1880612560' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:11:46.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:45 smithi169 ceph-mon[32759]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:46.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:45 smithi040 ceph-mon[32518]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:46.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:45 smithi031 ceph-mon[28053]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:47.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:47 smithi040 ceph-mon[32518]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:48.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:47 smithi169 ceph-mon[32759]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:48.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:47 smithi031 ceph-mon[28053]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:50.050 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:49 smithi040 ceph-mon[32518]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:50.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:49 smithi169 ceph-mon[32759]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:50.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:49 smithi031 ceph-mon[28053]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:52.063 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:51 smithi040 ceph-mon[32518]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:52.063 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:51 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T01:11:52.063 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:51 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:11:52.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:51 smithi169 ceph-mon[32759]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:52.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:51 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T01:11:52.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:51 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:11:52.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:51 smithi031 ceph-mon[28053]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:52.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:51 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T01:11:52.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:51 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:11:53.095 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:52 smithi040 ceph-mon[32518]: Deploying daemon osd.10 on smithi040 2024-04-25T01:11:53.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:52 smithi169 ceph-mon[32759]: Deploying daemon osd.10 on smithi040 2024-04-25T01:11:53.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:52 smithi031 ceph-mon[28053]: Deploying daemon osd.10 on smithi040 2024-04-25T01:11:53.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:53 smithi040 ceph-mon[32518]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:54.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:53 smithi169 ceph-mon[32759]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:54.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:53 smithi031 ceph-mon[28053]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:56.061 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:55 smithi031 ceph-mon[28053]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:56.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:55 smithi169 ceph-mon[32759]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:56.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:55 smithi040 ceph-mon[32518]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:58.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:57 smithi031 ceph-mon[28053]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:58.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:57 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:58.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:57 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:58.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:57 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:11:58.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:57 smithi169 ceph-mon[32759]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:58.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:57 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:58.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:57 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:58.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:57 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:11:58.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:57 smithi040 ceph-mon[32518]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:11:58.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:57 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:58.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:57 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:11:58.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:57 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:12:00.112 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:11:59 smithi040 ceph-mon[32518]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:12:00.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:11:59 smithi169 ceph-mon[32759]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:12:00.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:11:59 smithi031 ceph-mon[28053]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:12:00.761 INFO:teuthology.orchestra.run.smithi040.stdout:Created osd(s) 10 on host 'smithi040' 2024-04-25T01:12:00.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:12:00.772+0000 7f0173fff700 1 -- 172.21.15.40:0/1074738381 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f018c075ea0 con 0x7f01740a2c20 2024-04-25T01:12:00.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:12:00.775+0000 7f0191223700 1 -- 172.21.15.40:0/1074738381 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f01740a2c20 msgr2=0x7f01740a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:00.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:12:00.775+0000 7f0191223700 1 --2- 172.21.15.40:0/1074738381 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f01740a2c20 0x7f01740a50e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f018c1032f0 tx=0x7f017c009190 comp rx=0 tx=0).stop 2024-04-25T01:12:00.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:12:00.775+0000 7f0191223700 1 -- 172.21.15.40:0/1074738381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f018c103e30 msgr2=0x7f018c19fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:00.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:12:00.775+0000 7f0191223700 1 --2- 172.21.15.40:0/1074738381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f018c103e30 0x7f018c19fac0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0184007ee0 tx=0x7f018400d3b0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:12:00.775+0000 7f0191223700 1 -- 172.21.15.40:0/1074738381 shutdown_connections 2024-04-25T01:12:00.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:12:00.775+0000 7f0191223700 1 --2- 172.21.15.40:0/1074738381 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f01740a2c20 0x7f01740a50e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:12:00.775+0000 7f0191223700 1 --2- 172.21.15.40:0/1074738381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f018c103e30 0x7f018c19fac0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:12:00.775+0000 7f0191223700 1 --2- 172.21.15.40:0/1074738381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f018c102290 0x7f018c198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:12:00.775+0000 7f0191223700 1 --2- 172.21.15.40:0/1074738381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f018c103490 0x7f018c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:12:00.775+0000 7f0191223700 1 -- 172.21.15.40:0/1074738381 >> 172.21.15.40:0/1074738381 conn(0x7f018c0fd830 msgr2=0x7f018c106860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:00.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:12:00.775+0000 7f0191223700 1 -- 172.21.15.40:0/1074738381 shutdown_connections 2024-04-25T01:12:00.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-25T01:12:00.775+0000 7f0191223700 1 -- 172.21.15.40:0/1074738381 wait complete. 2024-04-25T01:12:01.439 DEBUG:teuthology.orchestra.run.smithi040:osd.10> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T01:12:01.441 INFO:tasks.cephadm:Deploying osd.11 on smithi169 with /dev/vg_nvme/lv_1... 2024-04-25T01:12:01.441 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T01:12:01.690 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:12:01.812 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:01 smithi169 ceph-mon[32759]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:12:01.812 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:01 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:01.812 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:01 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:01.812 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:01 smithi169 ceph-mon[32759]: from='osd.10 [v2:172.21.15.40:6824/3050148001,v1:172.21.15.40:6825/3050148001]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T01:12:01.812 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:01 smithi169 ceph-mon[32759]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T01:12:01.812 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:01 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:01.812 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:01 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:01.980 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:01 smithi040 ceph-mon[32518]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:12:01.980 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:01 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:01.980 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:01 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:01.980 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:01 smithi040 ceph-mon[32518]: from='osd.10 [v2:172.21.15.40:6824/3050148001,v1:172.21.15.40:6825/3050148001]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T01:12:01.980 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:01 smithi040 ceph-mon[32518]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T01:12:01.980 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:01 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:01.981 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:01 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:02.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:01 smithi031 ceph-mon[28053]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:12:02.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:01 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:02.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:01 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:02.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:01 smithi031 ceph-mon[28053]: from='osd.10 [v2:172.21.15.40:6824/3050148001,v1:172.21.15.40:6825/3050148001]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T01:12:02.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:01 smithi031 ceph-mon[28053]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T01:12:02.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:01 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:02.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:01 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:02.992 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:02 smithi040 ceph-mon[32518]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T01:12:02.992 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:02 smithi040 ceph-mon[32518]: from='osd.10 [v2:172.21.15.40:6824/3050148001,v1:172.21.15.40:6825/3050148001]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T01:12:02.992 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:02 smithi040 ceph-mon[32518]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T01:12:02.992 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:02 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:12:02.993 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:02 smithi040 ceph-mon[32518]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T01:12:03.120 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:02 smithi169 ceph-mon[32759]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T01:12:03.120 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:02 smithi169 ceph-mon[32759]: from='osd.10 [v2:172.21.15.40:6824/3050148001,v1:172.21.15.40:6825/3050148001]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T01:12:03.120 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:02 smithi169 ceph-mon[32759]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T01:12:03.120 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:02 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:12:03.120 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:02 smithi169 ceph-mon[32759]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T01:12:03.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:02 smithi031 ceph-mon[28053]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T01:12:03.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:02 smithi031 ceph-mon[28053]: from='osd.10 [v2:172.21.15.40:6824/3050148001,v1:172.21.15.40:6825/3050148001]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T01:12:03.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:02 smithi031 ceph-mon[28053]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T01:12:03.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:02 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:12:03.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:02 smithi031 ceph-mon[28053]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-25T01:12:03.684 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T01:12:03.685 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T01:12:03.685 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-04-25T01:12:03.685 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-04-25T01:12:03.685 INFO:teuthology.orchestra.run.smithi169.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115588 s, 907 MB/s 2024-04-25T01:12:03.685 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 2024-04-25T01:12:03.685 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-04-25T01:12:03.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:03 smithi040 ceph-mon[32518]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:12:03.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:03 smithi040 ceph-mon[32518]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-25T01:12:03.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:03 smithi040 ceph-mon[32518]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T01:12:03.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:03 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:12:03.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:03 smithi040 ceph-mon[32518]: from='osd.10 ' entity='osd.10' 2024-04-25T01:12:03.951 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:12:03 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:12:03.704+0000 7fc18ae79700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T01:12:03.951 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:12:03 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:12:03.725+0000 7fc183465700 -1 osd.10 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:12:04.027 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:03 smithi169 ceph-mon[32759]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:12:04.027 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:03 smithi169 ceph-mon[32759]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-25T01:12:04.027 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:03 smithi169 ceph-mon[32759]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T01:12:04.027 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:03 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:12:04.027 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:03 smithi169 ceph-mon[32759]: from='osd.10 ' entity='osd.10' 2024-04-25T01:12:04.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:03 smithi031 ceph-mon[28053]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:12:04.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:03 smithi031 ceph-mon[28053]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-25T01:12:04.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:03 smithi031 ceph-mon[28053]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T01:12:04.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:03 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:12:04.258 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:03 smithi031 ceph-mon[28053]: from='osd.10 ' entity='osd.10' 2024-04-25T01:12:04.334 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_1 2024-04-25T01:12:04.579 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:12:04.864 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:04 smithi169 ceph-mon[32759]: purged_snaps scrub starts 2024-04-25T01:12:04.864 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:04 smithi169 ceph-mon[32759]: purged_snaps scrub ok 2024-04-25T01:12:04.864 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:04 smithi169 ceph-mon[32759]: osd.10 [v2:172.21.15.40:6824/3050148001,v1:172.21.15.40:6825/3050148001] boot 2024-04-25T01:12:04.864 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:04 smithi169 ceph-mon[32759]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T01:12:04.864 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:04 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:12:04.864 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:04 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:04.864 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:04 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:04.864 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:04 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:12:04.865 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:04 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:12:04.865 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:04 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:12:04.865 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:04 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:12:04.865 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:04 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:04.865 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:04 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:04.865 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:04 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:12:04.865 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:04 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:04.865 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:04 smithi169 ceph-mon[32759]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T01:12:05.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:04 smithi040 ceph-mon[32518]: purged_snaps scrub starts 2024-04-25T01:12:05.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:04 smithi040 ceph-mon[32518]: purged_snaps scrub ok 2024-04-25T01:12:05.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:04 smithi040 ceph-mon[32518]: osd.10 [v2:172.21.15.40:6824/3050148001,v1:172.21.15.40:6825/3050148001] boot 2024-04-25T01:12:05.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:04 smithi040 ceph-mon[32518]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T01:12:05.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:04 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:12:05.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:04 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:05.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:04 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:05.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:04 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:12:05.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:04 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:12:05.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:04 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:12:05.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:04 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:12:05.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:04 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:05.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:04 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:05.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:04 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:12:05.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:04 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:05.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:04 smithi040 ceph-mon[32518]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T01:12:05.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:04 smithi031 ceph-mon[28053]: purged_snaps scrub starts 2024-04-25T01:12:05.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:04 smithi031 ceph-mon[28053]: purged_snaps scrub ok 2024-04-25T01:12:05.258 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:04 smithi031 ceph-mon[28053]: osd.10 [v2:172.21.15.40:6824/3050148001,v1:172.21.15.40:6825/3050148001] boot 2024-04-25T01:12:05.258 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:04 smithi031 ceph-mon[28053]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T01:12:05.258 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:04 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:12:05.258 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:04 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:05.258 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:04 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:05.258 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:04 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:12:05.258 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:04 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:12:05.258 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:04 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:12:05.258 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:04 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:12:05.258 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:04 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:05.258 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:04 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:05.258 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:04 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:12:05.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:04 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:05.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:04 smithi031 ceph-mon[28053]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T01:12:06.046 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.058+0000 7fef74e25700 1 -- 172.21.15.169:0/1444496106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef70100580 msgr2=0x7fef701009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:06.046 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.058+0000 7fef74e25700 1 --2- 172.21.15.169:0/1444496106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef70100580 0x7fef701009a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fef60009990 tx=0x7fef60009ca0 comp rx=0 tx=0).stop 2024-04-25T01:12:06.046 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.058+0000 7fef74e25700 1 -- 172.21.15.169:0/1444496106 shutdown_connections 2024-04-25T01:12:06.046 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.058+0000 7fef74e25700 1 --2- 172.21.15.169:0/1444496106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef70103f00 0x7fef7010a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:06.046 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.058+0000 7fef74e25700 1 --2- 172.21.15.169:0/1444496106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef70100580 0x7fef701009a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:06.046 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.058+0000 7fef74e25700 1 --2- 172.21.15.169:0/1444496106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef70103560 0x7fef701039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:06.046 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.058+0000 7fef74e25700 1 -- 172.21.15.169:0/1444496106 >> 172.21.15.169:0/1444496106 conn(0x7fef700fbee0 msgr2=0x7fef700fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:06.046 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.058+0000 7fef74e25700 1 -- 172.21.15.169:0/1444496106 shutdown_connections 2024-04-25T01:12:06.046 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.058+0000 7fef74e25700 1 -- 172.21.15.169:0/1444496106 wait complete. 2024-04-25T01:12:06.048 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.059+0000 7fef74e25700 1 Processor -- start 2024-04-25T01:12:06.048 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.059+0000 7fef74e25700 1 -- start start 2024-04-25T01:12:06.048 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.059+0000 7fef74e25700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef70100580 0x7fef70198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:06.048 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.059+0000 7fef74e25700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef70103560 0x7fef70198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:06.049 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.059+0000 7fef74e25700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef70103f00 0x7fef7019fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:06.049 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.059+0000 7fef74e25700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fef7010df60 con 0x7fef70103560 2024-04-25T01:12:06.049 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.059+0000 7fef74e25700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fef7010dde0 con 0x7fef70100580 2024-04-25T01:12:06.049 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.059+0000 7fef74e25700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fef7010e0e0 con 0x7fef70103f00 2024-04-25T01:12:06.049 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.059+0000 7fef6ed9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef70103f00 0x7fef7019fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:06.049 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.059+0000 7fef6ed9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef70103f00 0x7fef7019fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42354/0 (socket says 172.21.15.169:42354) 2024-04-25T01:12:06.049 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.059+0000 7fef6ed9d700 1 -- 172.21.15.169:0/391743478 learned_addr learned my addr 172.21.15.169:0/391743478 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T01:12:06.049 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.059+0000 7fef6e59c700 1 --2- 172.21.15.169:0/391743478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef70100580 0x7fef70198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:06.049 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.059+0000 7fef6ed9d700 1 -- 172.21.15.169:0/391743478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef70100580 msgr2=0x7fef70198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:06.049 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.059+0000 7fef6ed9d700 1 --2- 172.21.15.169:0/391743478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef70100580 0x7fef70198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:06.049 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.059+0000 7fef6ed9d700 1 -- 172.21.15.169:0/391743478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef70103560 msgr2=0x7fef70198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:06.049 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.059+0000 7fef6ed9d700 1 --2- 172.21.15.169:0/391743478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef70103560 0x7fef70198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:06.049 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.059+0000 7fef6ed9d700 1 -- 172.21.15.169:0/391743478 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef701a01a0 con 0x7fef70103f00 2024-04-25T01:12:06.050 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.060+0000 7fef6ed9d700 1 --2- 172.21.15.169:0/391743478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef70103f00 0x7fef7019fb00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fef68007f10 tx=0x7fef6800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:06.050 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.060+0000 7fef67fff700 1 -- 172.21.15.169:0/391743478 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68004bb0 con 0x7fef70103f00 2024-04-25T01:12:06.050 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.060+0000 7fef67fff700 1 -- 172.21.15.169:0/391743478 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef68004d50 con 0x7fef70103f00 2024-04-25T01:12:06.050 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.060+0000 7fef74e25700 1 -- 172.21.15.169:0/391743478 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef701a0490 con 0x7fef70103f00 2024-04-25T01:12:06.050 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.060+0000 7fef67fff700 1 -- 172.21.15.169:0/391743478 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68005a60 con 0x7fef70103f00 2024-04-25T01:12:06.050 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.060+0000 7fef74e25700 1 -- 172.21.15.169:0/391743478 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef701a0a50 con 0x7fef70103f00 2024-04-25T01:12:06.050 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.061+0000 7fef74e25700 1 -- 172.21.15.169:0/391743478 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef70068f20 con 0x7fef70103f00 2024-04-25T01:12:06.051 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.062+0000 7fef67fff700 1 -- 172.21.15.169:0/391743478 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fef6801a020 con 0x7fef70103f00 2024-04-25T01:12:06.051 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.062+0000 7fef67fff700 1 --2- 172.21.15.169:0/391743478 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fef5c0a2c20 0x7fef5c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:06.051 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.062+0000 7fef67fff700 1 -- 172.21.15.169:0/391743478 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6723+0+0 (secure 0 0 0) 0x7fef680ce1b0 con 0x7fef70103f00 2024-04-25T01:12:06.051 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.063+0000 7fef6e59c700 1 --2- 172.21.15.169:0/391743478 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fef5c0a2c20 0x7fef5c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:06.051 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.064+0000 7fef6e59c700 1 --2- 172.21.15.169:0/391743478 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fef5c0a2c20 0x7fef5c0a50e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fef60000c00 tx=0x7fef60005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:06.054 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.067+0000 7fef67fff700 1 -- 172.21.15.169:0/391743478 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fef68010070 con 0x7fef70103f00 2024-04-25T01:12:06.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:05 smithi169 ceph-mon[32759]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 547 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:12:06.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:05 smithi169 ceph-mon[32759]: Detected new or changed devices on smithi040 2024-04-25T01:12:06.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:05 smithi169 ceph-mon[32759]: Adjusting osd_memory_target on smithi040 to 4273M 2024-04-25T01:12:06.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:05 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:12:06.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:05 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:06.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:05 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:12:06.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:05 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:06.198 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:05 smithi169 ceph-mon[32759]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T01:12:06.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:05 smithi040 ceph-mon[32518]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 547 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:12:06.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:05 smithi040 ceph-mon[32518]: Detected new or changed devices on smithi040 2024-04-25T01:12:06.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:05 smithi040 ceph-mon[32518]: Adjusting osd_memory_target on smithi040 to 4273M 2024-04-25T01:12:06.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:05 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:12:06.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:05 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:06.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:05 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:12:06.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:05 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:06.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:05 smithi040 ceph-mon[32518]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T01:12:06.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:05 smithi031 ceph-mon[28053]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 547 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:12:06.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:05 smithi031 ceph-mon[28053]: Detected new or changed devices on smithi040 2024-04-25T01:12:06.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:05 smithi031 ceph-mon[28053]: Adjusting osd_memory_target on smithi040 to 4273M 2024-04-25T01:12:06.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:05 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:12:06.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:05 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:06.258 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:05 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:12:06.258 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:05 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:06.258 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:05 smithi031 ceph-mon[28053]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T01:12:06.262 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:06.275+0000 7fef74e25700 1 -- 172.21.15.169:0/391743478 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fef70062670 con 0x7fef5c0a2c20 2024-04-25T01:12:07.146 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:06 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:12:07.146 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:06 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:12:07.146 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:06 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:07.176 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:06 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:12:07.177 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:06 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:12:07.177 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:06 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:07.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:06 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:12:07.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:06 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:12:07.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:06 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:08.128 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:07 smithi169 ceph-mon[32759]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 547 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:12:08.128 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:07 smithi169 ceph-mon[32759]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:12:08.128 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:07 smithi169 ceph-mon[32759]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T01:12:08.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:07 smithi040 ceph-mon[32518]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 547 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:12:08.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:07 smithi040 ceph-mon[32518]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:12:08.200 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:07 smithi040 ceph-mon[32518]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T01:12:08.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:07 smithi031 ceph-mon[28053]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 547 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:12:08.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:07 smithi031 ceph-mon[28053]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:12:08.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:07 smithi031 ceph-mon[28053]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T01:12:09.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:08 smithi169 ceph-mon[32759]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T01:12:09.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:08 smithi040 ceph-mon[32518]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T01:12:09.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:08 smithi031 ceph-mon[28053]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T01:12:10.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:09 smithi169 ceph-mon[32759]: pgmap v266: 1 pgs: 1 remapped+peering; 577 KiB data, 548 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:12:10.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:09 smithi169 ceph-mon[32759]: from='client.? 172.21.15.169:0/3908041037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "144fe0aa-f757-4b3c-90ad-99775e17d57f"}]: dispatch 2024-04-25T01:12:10.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:09 smithi169 ceph-mon[32759]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "144fe0aa-f757-4b3c-90ad-99775e17d57f"}]: dispatch 2024-04-25T01:12:10.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:09 smithi169 ceph-mon[32759]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "144fe0aa-f757-4b3c-90ad-99775e17d57f"}]': finished 2024-04-25T01:12:10.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:09 smithi169 ceph-mon[32759]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T01:12:10.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:09 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:12:10.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:09 smithi169 ceph-mon[32759]: from='client.? 172.21.15.169:0/2260715416' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:12:10.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:09 smithi040 ceph-mon[32518]: pgmap v266: 1 pgs: 1 remapped+peering; 577 KiB data, 548 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:12:10.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:09 smithi040 ceph-mon[32518]: from='client.? 172.21.15.169:0/3908041037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "144fe0aa-f757-4b3c-90ad-99775e17d57f"}]: dispatch 2024-04-25T01:12:10.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:09 smithi040 ceph-mon[32518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "144fe0aa-f757-4b3c-90ad-99775e17d57f"}]: dispatch 2024-04-25T01:12:10.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:09 smithi040 ceph-mon[32518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "144fe0aa-f757-4b3c-90ad-99775e17d57f"}]': finished 2024-04-25T01:12:10.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:09 smithi040 ceph-mon[32518]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T01:12:10.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:09 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:12:10.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:09 smithi040 ceph-mon[32518]: from='client.? 172.21.15.169:0/2260715416' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:12:10.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:09 smithi031 ceph-mon[28053]: pgmap v266: 1 pgs: 1 remapped+peering; 577 KiB data, 548 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:12:10.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:09 smithi031 ceph-mon[28053]: from='client.? 172.21.15.169:0/3908041037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "144fe0aa-f757-4b3c-90ad-99775e17d57f"}]: dispatch 2024-04-25T01:12:10.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:09 smithi031 ceph-mon[28053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "144fe0aa-f757-4b3c-90ad-99775e17d57f"}]: dispatch 2024-04-25T01:12:10.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:09 smithi031 ceph-mon[28053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "144fe0aa-f757-4b3c-90ad-99775e17d57f"}]': finished 2024-04-25T01:12:10.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:09 smithi031 ceph-mon[28053]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T01:12:10.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:09 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:12:10.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:09 smithi031 ceph-mon[28053]: from='client.? 172.21.15.169:0/2260715416' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:12:11.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:10 smithi169 ceph-mon[32759]: pgmap v268: 1 pgs: 1 remapped+peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:12:11.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:10 smithi040 ceph-mon[32518]: pgmap v268: 1 pgs: 1 remapped+peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:12:11.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:10 smithi031 ceph-mon[28053]: pgmap v268: 1 pgs: 1 remapped+peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:12:13.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:13 smithi169 ceph-mon[32759]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T01:12:13.449 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:13 smithi040 ceph-mon[32518]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T01:12:13.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:13 smithi031 ceph-mon[28053]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T01:12:15.343 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:15 smithi169 ceph-mon[32759]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-25T01:12:15.449 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:15 smithi040 ceph-mon[32518]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-25T01:12:15.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:15 smithi031 ceph-mon[28053]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-25T01:12:17.317 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:17 smithi169 ceph-mon[32759]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-25T01:12:17.317 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T01:12:17.317 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:17.317 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:17 smithi169 ceph-mon[32759]: Deploying daemon osd.11 on smithi169 2024-04-25T01:12:17.449 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:17 smithi040 ceph-mon[32518]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-25T01:12:17.449 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T01:12:17.449 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:17.449 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:17 smithi040 ceph-mon[32518]: Deploying daemon osd.11 on smithi169 2024-04-25T01:12:17.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:17 smithi031 ceph-mon[28053]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-25T01:12:17.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T01:12:17.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:17.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:17 smithi031 ceph-mon[28053]: Deploying daemon osd.11 on smithi169 2024-04-25T01:12:19.135 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:19 smithi169 ceph-mon[32759]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T01:12:19.449 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:19 smithi040 ceph-mon[32518]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T01:12:19.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:19 smithi031 ceph-mon[28053]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T01:12:21.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:21 smithi169 ceph-mon[32759]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-04-25T01:12:21.449 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:21 smithi040 ceph-mon[32518]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-04-25T01:12:21.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:21 smithi031 ceph-mon[28053]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-04-25T01:12:22.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:22 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:22.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:22 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:12:22.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:22 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:22.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:22 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:22.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:22 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:12:22.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:22 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:23.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:22 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:23.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:22 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:12:23.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:22 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:23.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:23 smithi169 ceph-mon[32759]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T01:12:23.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:23 smithi040 ceph-mon[32518]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T01:12:24.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:23 smithi031 ceph-mon[28053]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T01:12:25.348 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:25.360+0000 7fef67fff700 1 -- 172.21.15.169:0/391743478 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fef70062670 con 0x7fef5c0a2c20 2024-04-25T01:12:25.348 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 11 on host 'smithi169' 2024-04-25T01:12:25.350 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:25.363+0000 7fef74e25700 1 -- 172.21.15.169:0/391743478 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fef5c0a2c20 msgr2=0x7fef5c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:25.350 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:25.363+0000 7fef74e25700 1 --2- 172.21.15.169:0/391743478 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fef5c0a2c20 0x7fef5c0a50e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fef60000c00 tx=0x7fef60005fb0 comp rx=0 tx=0).stop 2024-04-25T01:12:25.351 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:25.363+0000 7fef74e25700 1 -- 172.21.15.169:0/391743478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef70103f00 msgr2=0x7fef7019fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:25.351 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:25.363+0000 7fef74e25700 1 --2- 172.21.15.169:0/391743478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef70103f00 0x7fef7019fb00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fef68007f10 tx=0x7fef6800d3b0 comp rx=0 tx=0).stop 2024-04-25T01:12:25.351 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:25.363+0000 7fef74e25700 1 -- 172.21.15.169:0/391743478 shutdown_connections 2024-04-25T01:12:25.351 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:25.363+0000 7fef74e25700 1 --2- 172.21.15.169:0/391743478 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fef5c0a2c20 0x7fef5c0a50e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:25.351 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:25.363+0000 7fef74e25700 1 --2- 172.21.15.169:0/391743478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef70103f00 0x7fef7019fb00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:25.351 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:25.363+0000 7fef74e25700 1 --2- 172.21.15.169:0/391743478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef70100580 0x7fef70198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:25.351 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:25.363+0000 7fef74e25700 1 --2- 172.21.15.169:0/391743478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef70103560 0x7fef70198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:25.351 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:25.363+0000 7fef74e25700 1 -- 172.21.15.169:0/391743478 >> 172.21.15.169:0/391743478 conn(0x7fef700fbee0 msgr2=0x7fef70106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:25.351 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:25.363+0000 7fef74e25700 1 -- 172.21.15.169:0/391743478 shutdown_connections 2024-04-25T01:12:25.351 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:25.363+0000 7fef74e25700 1 -- 172.21.15.169:0/391743478 wait complete. 2024-04-25T01:12:25.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:25 smithi169 ceph-mon[32759]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:12:25.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:25.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:25.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:25.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:12:25.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:25.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:25.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:25.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:12:25.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:25 smithi040 ceph-mon[32518]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:12:25.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:25.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:25.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:25.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:12:25.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:25.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:25.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:25.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:12:26.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:25 smithi031 ceph-mon[28053]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:12:26.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:26.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:26.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:26.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:12:26.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:26.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:26.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:26.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:12:26.035 DEBUG:teuthology.orchestra.run.smithi169:osd.11> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T01:12:26.037 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T01:12:26.037 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T01:12:26.512 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:12:26.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:26 smithi169 ceph-mon[32759]: from='osd.11 [v2:172.21.15.169:6824/262401944,v1:172.21.15.169:6825/262401944]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T01:12:26.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:26 smithi169 ceph-mon[32759]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T01:12:26.818 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:26 smithi031 ceph-mon[28053]: from='osd.11 [v2:172.21.15.169:6824/262401944,v1:172.21.15.169:6825/262401944]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T01:12:26.819 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:26 smithi031 ceph-mon[28053]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T01:12:26.850 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:26 smithi040 ceph-mon[32518]: from='osd.11 [v2:172.21.15.169:6824/262401944,v1:172.21.15.169:6825/262401944]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T01:12:26.851 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:26 smithi040 ceph-mon[32518]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T01:12:27.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:27 smithi031 ceph-mon[28053]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:12:27.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:27 smithi031 ceph-mon[28053]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T01:12:27.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:27 smithi031 ceph-mon[28053]: from='osd.11 [v2:172.21.15.169:6824/262401944,v1:172.21.15.169:6825/262401944]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T01:12:27.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:27 smithi031 ceph-mon[28053]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T01:12:27.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:27 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:12:27.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:27 smithi031 ceph-mon[28053]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T01:12:27.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:27 smithi169 ceph-mon[32759]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:12:27.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:27 smithi169 ceph-mon[32759]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T01:12:27.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:27 smithi169 ceph-mon[32759]: from='osd.11 [v2:172.21.15.169:6824/262401944,v1:172.21.15.169:6825/262401944]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T01:12:27.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:27 smithi169 ceph-mon[32759]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T01:12:27.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:12:27.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:27 smithi169 ceph-mon[32759]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T01:12:27.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:27 smithi040 ceph-mon[32518]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:12:27.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:27 smithi040 ceph-mon[32518]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T01:12:27.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:27 smithi040 ceph-mon[32518]: from='osd.11 [v2:172.21.15.169:6824/262401944,v1:172.21.15.169:6825/262401944]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T01:12:27.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:27 smithi040 ceph-mon[32518]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T01:12:27.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:27 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:12:27.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:27 smithi040 ceph-mon[32518]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T01:12:28.538 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:12:28 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:12:28.383+0000 7f51d2ca1700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T01:12:28.538 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:12:28 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:12:28.404+0000 7f51cb28d700 -1 osd.11 81 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:12:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.741+0000 7f25e5400700 1 -- 172.21.15.31:0/4028637777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25e010a470 msgr2=0x7f25e0110d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.741+0000 7f25e5400700 1 --2- 172.21.15.31:0/4028637777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25e010a470 0x7f25e0110d20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f25d800b0d0 tx=0x7f25d800b3e0 comp rx=0 tx=0).stop 2024-04-25T01:12:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.741+0000 7f25e5400700 1 -- 172.21.15.31:0/4028637777 shutdown_connections 2024-04-25T01:12:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.741+0000 7f25e5400700 1 --2- 172.21.15.31:0/4028637777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e01088d0 0x7f25e0108cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.741+0000 7f25e5400700 1 --2- 172.21.15.31:0/4028637777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25e010a470 0x7f25e0110d20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.741+0000 7f25e5400700 1 --2- 172.21.15.31:0/4028637777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e0109ad0 0x7f25e0109f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.741+0000 7f25e5400700 1 -- 172.21.15.31:0/4028637777 >> 172.21.15.31:0/4028637777 conn(0x7f25e00778a0 msgr2=0x7f25e0079ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:28.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.741+0000 7f25e5400700 1 -- 172.21.15.31:0/4028637777 shutdown_connections 2024-04-25T01:12:28.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.741+0000 7f25e5400700 1 -- 172.21.15.31:0/4028637777 wait complete. 2024-04-25T01:12:28.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.742+0000 7f25e5400700 1 Processor -- start 2024-04-25T01:12:28.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.743+0000 7f25e5400700 1 -- start start 2024-04-25T01:12:28.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.743+0000 7f25e5400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e01088d0 0x7f25e019ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:28.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.743+0000 7f25e5400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e0109ad0 0x7f25e019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:28.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.743+0000 7f25e5400700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25e010a470 0x7f25e01a6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:28.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.743+0000 7f25e5400700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f25e01144f0 con 0x7f25e0109ad0 2024-04-25T01:12:28.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.743+0000 7f25e5400700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f25e0114370 con 0x7f25e010a470 2024-04-25T01:12:28.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.743+0000 7f25e5400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f25e0114670 con 0x7f25e01088d0 2024-04-25T01:12:28.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.743+0000 7f25dde00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e0109ad0 0x7f25e019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:28.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.743+0000 7f25dde00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e0109ad0 0x7f25e019ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57734/0 (socket says 172.21.15.31:57734) 2024-04-25T01:12:28.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.743+0000 7f25dde00700 1 -- 172.21.15.31:0/1837223140 learned_addr learned my addr 172.21.15.31:0/1837223140 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:12:28.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.743+0000 7f25de800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e01088d0 0x7f25e019ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:28.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.743+0000 7f25df200700 1 --2- 172.21.15.31:0/1837223140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25e010a470 0x7f25e01a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:28.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.744+0000 7f25de800700 1 -- 172.21.15.31:0/1837223140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25e010a470 msgr2=0x7f25e01a6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:28.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.744+0000 7f25de800700 1 --2- 172.21.15.31:0/1837223140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25e010a470 0x7f25e01a6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:28.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.744+0000 7f25de800700 1 -- 172.21.15.31:0/1837223140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e0109ad0 msgr2=0x7f25e019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:28.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.744+0000 7f25de800700 1 --2- 172.21.15.31:0/1837223140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e0109ad0 0x7f25e019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:28.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.744+0000 7f25de800700 1 -- 172.21.15.31:0/1837223140 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e01a66a0 con 0x7f25e01088d0 2024-04-25T01:12:28.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.744+0000 7f25dde00700 1 --2- 172.21.15.31:0/1837223140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e0109ad0 0x7f25e019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:12:28.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.744+0000 7f25de800700 1 --2- 172.21.15.31:0/1837223140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e01088d0 0x7f25e019ea40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f25d000e9f0 tx=0x7f25d000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:28.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.744+0000 7f25d7400700 1 -- 172.21.15.31:0/1837223140 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0004220 con 0x7f25e01088d0 2024-04-25T01:12:28.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.744+0000 7f25d7400700 1 -- 172.21.15.31:0/1837223140 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25d00043c0 con 0x7f25e01088d0 2024-04-25T01:12:28.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.744+0000 7f25d7400700 1 -- 172.21.15.31:0/1837223140 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0003cb0 con 0x7f25e01088d0 2024-04-25T01:12:28.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.744+0000 7f25e5400700 1 -- 172.21.15.31:0/1837223140 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e01a6990 con 0x7f25e01088d0 2024-04-25T01:12:28.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.745+0000 7f25e5400700 1 -- 172.21.15.31:0/1837223140 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e01a6e80 con 0x7f25e01088d0 2024-04-25T01:12:28.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.746+0000 7f25e5400700 1 -- 172.21.15.31:0/1837223140 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25e0068f20 con 0x7f25e01088d0 2024-04-25T01:12:28.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.748+0000 7f25d7400700 1 -- 172.21.15.31:0/1837223140 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f25d0043ae0 con 0x7f25e01088d0 2024-04-25T01:12:28.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.749+0000 7f25d7400700 1 --2- 172.21.15.31:0/1837223140 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f25c40a2ca0 0x7f25c40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:28.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.749+0000 7f25d7400700 1 -- 172.21.15.31:0/1837223140 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6847+0+0 (secure 0 0 0) 0x7f25d00cecc0 con 0x7f25e01088d0 2024-04-25T01:12:28.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.749+0000 7f25dde00700 1 --2- 172.21.15.31:0/1837223140 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f25c40a2ca0 0x7f25c40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:28.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.750+0000 7f25dde00700 1 --2- 172.21.15.31:0/1837223140 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f25c40a2ca0 0x7f25c40a5160 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f25cc011630 tx=0x7f25cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:28.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.751+0000 7f25d7400700 1 -- 172.21.15.31:0/1837223140 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f25d009c140 con 0x7f25e01088d0 2024-04-25T01:12:28.893 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:28 smithi169 ceph-mon[32759]: purged_snaps scrub starts 2024-04-25T01:12:28.893 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:28 smithi169 ceph-mon[32759]: purged_snaps scrub ok 2024-04-25T01:12:28.893 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:28 smithi169 ceph-mon[32759]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T01:12:28.893 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:28 smithi169 ceph-mon[32759]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T01:12:28.894 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:28 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:12:28.894 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:28 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:12:28.894 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:28 smithi169 ceph-mon[32759]: from='osd.11 ' entity='osd.11' 2024-04-25T01:12:28.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:28 smithi040 ceph-mon[32518]: purged_snaps scrub starts 2024-04-25T01:12:28.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:28 smithi040 ceph-mon[32518]: purged_snaps scrub ok 2024-04-25T01:12:28.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:28 smithi040 ceph-mon[32518]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T01:12:28.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:28 smithi040 ceph-mon[32518]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T01:12:28.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:28 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:12:28.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:28 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:12:28.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:28 smithi040 ceph-mon[32518]: from='osd.11 ' entity='osd.11' 2024-04-25T01:12:28.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.993+0000 7f25e5400700 1 -- 172.21.15.31:0/1837223140 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f25e002dbf0 con 0x7f25e01088d0 2024-04-25T01:12:28.982 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:28 smithi031 ceph-mon[28053]: purged_snaps scrub starts 2024-04-25T01:12:28.982 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:28 smithi031 ceph-mon[28053]: purged_snaps scrub ok 2024-04-25T01:12:28.982 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:28 smithi031 ceph-mon[28053]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T01:12:28.982 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:28 smithi031 ceph-mon[28053]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T01:12:28.982 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:28 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:12:28.982 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:28 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:12:28.982 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:28 smithi031 ceph-mon[28053]: from='osd.11 ' entity='osd.11' 2024-04-25T01:12:28.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:28.999+0000 7f25d7400700 1 -- 172.21.15.31:0/1837223140 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f25d009ba40 con 0x7f25e01088d0 2024-04-25T01:12:28.987 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:12:28.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:29.003+0000 7f25e5400700 1 -- 172.21.15.31:0/1837223140 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f25c40a2ca0 msgr2=0x7f25c40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:28.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:29.003+0000 7f25e5400700 1 --2- 172.21.15.31:0/1837223140 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f25c40a2ca0 0x7f25c40a5160 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f25cc011630 tx=0x7f25cc009260 comp rx=0 tx=0).stop 2024-04-25T01:12:28.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:29.003+0000 7f25e5400700 1 -- 172.21.15.31:0/1837223140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e01088d0 msgr2=0x7f25e019ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:28.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:29.003+0000 7f25e5400700 1 --2- 172.21.15.31:0/1837223140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e01088d0 0x7f25e019ea40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f25d000e9f0 tx=0x7f25d000ed00 comp rx=0 tx=0).stop 2024-04-25T01:12:28.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:29.003+0000 7f25e5400700 1 -- 172.21.15.31:0/1837223140 shutdown_connections 2024-04-25T01:12:28.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:29.003+0000 7f25e5400700 1 --2- 172.21.15.31:0/1837223140 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f25c40a2ca0 0x7f25c40a5160 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:28.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:29.003+0000 7f25e5400700 1 --2- 172.21.15.31:0/1837223140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25e010a470 0x7f25e01a6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:28.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:29.003+0000 7f25e5400700 1 --2- 172.21.15.31:0/1837223140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e01088d0 0x7f25e019ea40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:28.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:29.003+0000 7f25e5400700 1 --2- 172.21.15.31:0/1837223140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e0109ad0 0x7f25e019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:28.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:29.003+0000 7f25e5400700 1 -- 172.21.15.31:0/1837223140 >> 172.21.15.31:0/1837223140 conn(0x7f25e00778a0 msgr2=0x7f25e010cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:28.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:29.003+0000 7f25e5400700 1 -- 172.21.15.31:0/1837223140 shutdown_connections 2024-04-25T01:12:28.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:29.003+0000 7f25e5400700 1 -- 172.21.15.31:0/1837223140 wait complete. 2024-04-25T01:12:29.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:29 smithi031 ceph-mon[28053]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:12:29.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:29 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:12:29.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:29 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:29.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:29 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:29.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:29 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:12:29.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:29 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:12:29.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:29 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:12:29.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:29 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:12:29.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:29 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1837223140' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T01:12:29.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:29 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:29.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:29 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:29.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:29 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:12:29.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:29 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:29.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:29 smithi031 ceph-mon[28053]: osd.11 [v2:172.21.15.169:6824/262401944,v1:172.21.15.169:6825/262401944] boot 2024-04-25T01:12:29.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:29 smithi031 ceph-mon[28053]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T01:12:29.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:29 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:12:29.892 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":81,"num_osds":12,"num_up_osds":11,"osd_up_since":1714007523,"num_in_osds":12,"osd_in_since":1714007528,"num_remapped_pgs":0} 2024-04-25T01:12:29.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:29 smithi169 ceph-mon[32759]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:12:29.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:29 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:12:29.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:29 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:29.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:29 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:29.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:29 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:12:29.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:29 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:12:29.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:29 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:12:29.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:29 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:12:29.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:29 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1837223140' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T01:12:29.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:29 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:29.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:29 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:29.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:29 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:12:29.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:29 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:29.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:29 smithi169 ceph-mon[32759]: osd.11 [v2:172.21.15.169:6824/262401944,v1:172.21.15.169:6825/262401944] boot 2024-04-25T01:12:29.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:29 smithi169 ceph-mon[32759]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T01:12:29.947 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:29 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:12:29.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:29 smithi040 ceph-mon[32518]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:12:29.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:29 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:12:29.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:29 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:29.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:29 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:29.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:29 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:12:29.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:29 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:12:29.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:29 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:12:29.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:29 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:12:29.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:29 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1837223140' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T01:12:29.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:29 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:29.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:29 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:29.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:29 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:12:29.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:29 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:29.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:29 smithi040 ceph-mon[32518]: osd.11 [v2:172.21.15.169:6824/262401944,v1:172.21.15.169:6825/262401944] boot 2024-04-25T01:12:29.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:29 smithi040 ceph-mon[32518]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T01:12:29.950 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:29 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:12:30.893 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T01:12:30.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:30 smithi169 ceph-mon[32759]: Detected new or changed devices on smithi169 2024-04-25T01:12:30.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:30 smithi169 ceph-mon[32759]: Adjusting osd_memory_target on smithi169 to 4272M 2024-04-25T01:12:30.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:30 smithi040 ceph-mon[32518]: Detected new or changed devices on smithi169 2024-04-25T01:12:30.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:30 smithi040 ceph-mon[32518]: Adjusting osd_memory_target on smithi169 to 4272M 2024-04-25T01:12:30.977 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:30 smithi031 ceph-mon[28053]: Detected new or changed devices on smithi169 2024-04-25T01:12:30.977 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:30 smithi031 ceph-mon[28053]: Adjusting osd_memory_target on smithi169 to 4272M 2024-04-25T01:12:31.377 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:12:31.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:31 smithi169 ceph-mon[32759]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:31.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:31 smithi169 ceph-mon[32759]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T01:12:31.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:31 smithi040 ceph-mon[32518]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:31.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:31 smithi040 ceph-mon[32518]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T01:12:31.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:31 smithi031 ceph-mon[28053]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:31.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:31 smithi031 ceph-mon[28053]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T01:12:33.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.585+0000 7f88d3400700 1 -- 172.21.15.31:0/2748632331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88cc074c90 msgr2=0x7f88cc075110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:33.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.585+0000 7f88d3400700 1 --2- 172.21.15.31:0/2748632331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88cc074c90 0x7f88cc075110 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009990 tx=0x7f88b8009ca0 comp rx=0 tx=0).stop 2024-04-25T01:12:33.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.585+0000 7f88d3400700 1 -- 172.21.15.31:0/2748632331 shutdown_connections 2024-04-25T01:12:33.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.585+0000 7f88d3400700 1 --2- 172.21.15.31:0/2748632331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88cc10a170 0x7f88cc10c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:33.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.585+0000 7f88d3400700 1 --2- 172.21.15.31:0/2748632331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88cc076260 0x7f88cc0746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:33.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.585+0000 7f88d3400700 1 --2- 172.21.15.31:0/2748632331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88cc074c90 0x7f88cc075110 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:33.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.585+0000 7f88d3400700 1 -- 172.21.15.31:0/2748632331 >> 172.21.15.31:0/2748632331 conn(0x7f88cc0fd800 msgr2=0x7f88cc0ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:33.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.585+0000 7f88d3400700 1 -- 172.21.15.31:0/2748632331 shutdown_connections 2024-04-25T01:12:33.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.585+0000 7f88d3400700 1 -- 172.21.15.31:0/2748632331 wait complete. 2024-04-25T01:12:33.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.586+0000 7f88d3400700 1 Processor -- start 2024-04-25T01:12:33.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.586+0000 7f88d3400700 1 -- start start 2024-04-25T01:12:33.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.586+0000 7f88d3400700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88cc074c90 0x7f88cc19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:33.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.586+0000 7f88d3400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88cc076260 0x7f88cc19acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:33.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.587+0000 7f88d3400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88cc10a170 0x7f88cc1a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:33.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.587+0000 7f88d3400700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc110100 con 0x7f88cc10a170 2024-04-25T01:12:33.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.587+0000 7f88d3400700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc10ff80 con 0x7f88cc074c90 2024-04-25T01:12:33.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.587+0000 7f88d3400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc110280 con 0x7f88cc076260 2024-04-25T01:12:33.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.587+0000 7f88d0a00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88cc074c90 0x7f88cc19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:33.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.587+0000 7f88d0a00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88cc074c90 0x7f88cc19a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:54546/0 (socket says 172.21.15.31:54546) 2024-04-25T01:12:33.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.587+0000 7f88d0a00700 1 -- 172.21.15.31:0/4281387362 learned_addr learned my addr 172.21.15.31:0/4281387362 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:12:33.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.587+0000 7f88cbe00700 1 --2- 172.21.15.31:0/4281387362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88cc076260 0x7f88cc19acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:33.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.587+0000 7f88d1400700 1 --2- 172.21.15.31:0/4281387362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88cc10a170 0x7f88cc1a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:33.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.587+0000 7f88cbe00700 1 -- 172.21.15.31:0/4281387362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88cc074c90 msgr2=0x7f88cc19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:33.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.587+0000 7f88cbe00700 1 --2- 172.21.15.31:0/4281387362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88cc074c90 0x7f88cc19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:33.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.587+0000 7f88cbe00700 1 -- 172.21.15.31:0/4281387362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88cc10a170 msgr2=0x7f88cc1a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:33.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.588+0000 7f88cbe00700 1 --2- 172.21.15.31:0/4281387362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88cc10a170 0x7f88cc1a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:33.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.588+0000 7f88cbe00700 1 -- 172.21.15.31:0/4281387362 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88cc1a2390 con 0x7f88cc076260 2024-04-25T01:12:33.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.588+0000 7f88d1400700 1 --2- 172.21.15.31:0/4281387362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88cc10a170 0x7f88cc1a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:12:33.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.588+0000 7f88cbe00700 1 --2- 172.21.15.31:0/4281387362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88cc076260 0x7f88cc19acc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f88b8000c00 tx=0x7f88b800bb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:33.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.588+0000 7f88c9600700 1 -- 172.21.15.31:0/4281387362 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b8026070 con 0x7f88cc076260 2024-04-25T01:12:33.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.588+0000 7f88c9600700 1 -- 172.21.15.31:0/4281387362 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b8003d70 con 0x7f88cc076260 2024-04-25T01:12:33.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.588+0000 7f88d3400700 1 -- 172.21.15.31:0/4281387362 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88cc1a2620 con 0x7f88cc076260 2024-04-25T01:12:33.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.588+0000 7f88c9600700 1 -- 172.21.15.31:0/4281387362 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b8021410 con 0x7f88cc076260 2024-04-25T01:12:33.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.588+0000 7f88d3400700 1 -- 172.21.15.31:0/4281387362 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88cc1a2a70 con 0x7f88cc076260 2024-04-25T01:12:33.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.589+0000 7f88d3400700 1 -- 172.21.15.31:0/4281387362 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88cc068f20 con 0x7f88cc076260 2024-04-25T01:12:33.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.592+0000 7f88c9600700 1 -- 172.21.15.31:0/4281387362 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f88b8029070 con 0x7f88cc076260 2024-04-25T01:12:33.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.593+0000 7f88c9600700 1 --2- 172.21.15.31:0/4281387362 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f88b40a2c20 0x7f88b40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:33.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.593+0000 7f88c9600700 1 -- 172.21.15.31:0/4281387362 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f88b80cfcc0 con 0x7f88cc076260 2024-04-25T01:12:33.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.593+0000 7f88d0a00700 1 --2- 172.21.15.31:0/4281387362 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f88b40a2c20 0x7f88b40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:33.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.594+0000 7f88d0a00700 1 --2- 172.21.15.31:0/4281387362 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f88b40a2c20 0x7f88b40a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009d00 tx=0x7f88c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:33.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.597+0000 7f88c9600700 1 -- 172.21.15.31:0/4281387362 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f88b80d6080 con 0x7f88cc076260 2024-04-25T01:12:33.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:33 smithi031 ceph-mon[28053]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:33.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.833+0000 7f88d3400700 1 -- 172.21.15.31:0/4281387362 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f88cc062e40 con 0x7f88cc076260 2024-04-25T01:12:33.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.834+0000 7f88c9600700 1 -- 172.21.15.31:0/4281387362 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v83) ==== 74+0+133 (secure 0 0 0) 0x7f88b8013070 con 0x7f88cc076260 2024-04-25T01:12:33.822 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:12:33.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.838+0000 7f88d3400700 1 -- 172.21.15.31:0/4281387362 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f88b40a2c20 msgr2=0x7f88b40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:33.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.838+0000 7f88d3400700 1 --2- 172.21.15.31:0/4281387362 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f88b40a2c20 0x7f88b40a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009d00 tx=0x7f88c0009260 comp rx=0 tx=0).stop 2024-04-25T01:12:33.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.838+0000 7f88d3400700 1 -- 172.21.15.31:0/4281387362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88cc076260 msgr2=0x7f88cc19acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:33.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.838+0000 7f88d3400700 1 --2- 172.21.15.31:0/4281387362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88cc076260 0x7f88cc19acc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f88b8000c00 tx=0x7f88b800bb90 comp rx=0 tx=0).stop 2024-04-25T01:12:33.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.838+0000 7f88d3400700 1 -- 172.21.15.31:0/4281387362 shutdown_connections 2024-04-25T01:12:33.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.838+0000 7f88d3400700 1 --2- 172.21.15.31:0/4281387362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88cc10a170 0x7f88cc1a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:33.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.838+0000 7f88d3400700 1 --2- 172.21.15.31:0/4281387362 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f88b40a2c20 0x7f88b40a50e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:33.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.838+0000 7f88d3400700 1 --2- 172.21.15.31:0/4281387362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88cc074c90 0x7f88cc19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:33.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.838+0000 7f88d3400700 1 --2- 172.21.15.31:0/4281387362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88cc076260 0x7f88cc19acc0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:33.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.838+0000 7f88d3400700 1 -- 172.21.15.31:0/4281387362 >> 172.21.15.31:0/4281387362 conn(0x7f88cc0fd800 msgr2=0x7f88cc106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:33.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.838+0000 7f88d3400700 1 -- 172.21.15.31:0/4281387362 shutdown_connections 2024-04-25T01:12:33.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:33.838+0000 7f88d3400700 1 -- 172.21.15.31:0/4281387362 wait complete. 2024-04-25T01:12:33.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:33 smithi169 ceph-mon[32759]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:33.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:33 smithi040 ceph-mon[32518]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:34.686 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":83,"num_osds":12,"num_up_osds":12,"osd_up_since":1714007549,"num_in_osds":12,"osd_in_since":1714007528,"num_remapped_pgs":0} 2024-04-25T01:12:34.687 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T01:12:34.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:34 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/4281387362' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T01:12:34.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:34 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/4281387362' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T01:12:35.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:34 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/4281387362' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T01:12:35.144 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:12:35.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:35 smithi169 ceph-mon[32759]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:35.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:35 smithi040 ceph-mon[32518]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:36.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:35 smithi031 ceph-mon[28053]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:37.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.457+0000 7faa04600700 1 -- 172.21.15.31:0/755640602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9fc10a4e0 msgr2=0x7fa9fc10a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:37.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.457+0000 7faa04600700 1 --2- 172.21.15.31:0/755640602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9fc10a4e0 0x7fa9fc10a8c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009990 tx=0x7fa9f0009ca0 comp rx=0 tx=0).stop 2024-04-25T01:12:37.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.457+0000 7faa04600700 1 -- 172.21.15.31:0/755640602 shutdown_connections 2024-04-25T01:12:37.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.457+0000 7faa04600700 1 --2- 172.21.15.31:0/755640602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9fc104e10 0x7fa9fc111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:37.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.457+0000 7faa04600700 1 --2- 172.21.15.31:0/755640602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9fc10a4e0 0x7fa9fc10a8c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:37.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.457+0000 7faa04600700 1 --2- 172.21.15.31:0/755640602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9fc104450 0x7fa9fc1048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:37.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.457+0000 7faa04600700 1 -- 172.21.15.31:0/755640602 >> 172.21.15.31:0/755640602 conn(0x7fa9fc0fff40 msgr2=0x7fa9fc102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:37.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.458+0000 7faa04600700 1 -- 172.21.15.31:0/755640602 shutdown_connections 2024-04-25T01:12:37.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.458+0000 7faa04600700 1 -- 172.21.15.31:0/755640602 wait complete. 2024-04-25T01:12:37.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.459+0000 7faa04600700 1 Processor -- start 2024-04-25T01:12:37.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.459+0000 7faa04600700 1 -- start start 2024-04-25T01:12:37.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.460+0000 7faa04600700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9fc104450 0x7fa9fc1a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:37.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.460+0000 7faa04600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9fc104e10 0x7fa9fc1a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:37.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.460+0000 7faa04600700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9fc10a4e0 0x7fa9fc19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:37.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.460+0000 7faa04600700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa9fc1147b0 con 0x7fa9fc10a4e0 2024-04-25T01:12:37.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.460+0000 7faa04600700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa9fc114630 con 0x7fa9fc104450 2024-04-25T01:12:37.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.460+0000 7faa04600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa9fc114930 con 0x7fa9fc104e10 2024-04-25T01:12:37.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.460+0000 7faa01c00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9fc104450 0x7fa9fc1a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:37.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.460+0000 7faa02600700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9fc10a4e0 0x7fa9fc19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:37.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.460+0000 7faa01200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9fc104e10 0x7fa9fc1a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:37.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.460+0000 7faa01c00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9fc104450 0x7fa9fc1a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:58054/0 (socket says 172.21.15.31:58054) 2024-04-25T01:12:37.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.460+0000 7faa01c00700 1 -- 172.21.15.31:0/9077281 learned_addr learned my addr 172.21.15.31:0/9077281 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:12:37.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.461+0000 7faa01c00700 1 -- 172.21.15.31:0/9077281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9fc104e10 msgr2=0x7fa9fc1a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:37.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.461+0000 7faa01c00700 1 --2- 172.21.15.31:0/9077281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9fc104e10 0x7fa9fc1a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:37.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.461+0000 7faa01c00700 1 -- 172.21.15.31:0/9077281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9fc10a4e0 msgr2=0x7fa9fc19aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:37.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.461+0000 7faa01c00700 1 --2- 172.21.15.31:0/9077281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9fc10a4e0 0x7fa9fc19aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:37.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.461+0000 7faa01c00700 1 -- 172.21.15.31:0/9077281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9fc19b1b0 con 0x7fa9fc104450 2024-04-25T01:12:37.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.461+0000 7faa01c00700 1 --2- 172.21.15.31:0/9077281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9fc104450 0x7fa9fc1a0840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0000c00 tx=0x7fa9f000bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:37.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.461+0000 7fa9fa000700 1 -- 172.21.15.31:0/9077281 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0027070 con 0x7fa9fc104450 2024-04-25T01:12:37.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.461+0000 7fa9fa000700 1 -- 172.21.15.31:0/9077281 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9f00056d0 con 0x7fa9fc104450 2024-04-25T01:12:37.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.461+0000 7fa9fa000700 1 -- 172.21.15.31:0/9077281 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0022e10 con 0x7fa9fc104450 2024-04-25T01:12:37.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.461+0000 7faa04600700 1 -- 172.21.15.31:0/9077281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9fc19b440 con 0x7fa9fc104450 2024-04-25T01:12:37.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.462+0000 7faa04600700 1 -- 172.21.15.31:0/9077281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9fc1a7710 con 0x7fa9fc104450 2024-04-25T01:12:37.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.463+0000 7faa04600700 1 -- 172.21.15.31:0/9077281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9fc10c9e0 con 0x7fa9fc104450 2024-04-25T01:12:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.465+0000 7fa9fa000700 1 -- 172.21.15.31:0/9077281 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa9f002a070 con 0x7fa9fc104450 2024-04-25T01:12:37.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.466+0000 7fa9fa000700 1 --2- 172.21.15.31:0/9077281 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa9e40a2ca0 0x7fa9e40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:37.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.466+0000 7fa9fa000700 1 -- 172.21.15.31:0/9077281 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fa9f00cf820 con 0x7fa9fc104450 2024-04-25T01:12:37.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.466+0000 7faa01200700 1 --2- 172.21.15.31:0/9077281 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa9e40a2ca0 0x7fa9e40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:37.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.467+0000 7faa01200700 1 --2- 172.21.15.31:0/9077281 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa9e40a2ca0 0x7fa9e40a5160 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc109ae0 tx=0x7fa9e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:37.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.471+0000 7fa9fa000700 1 -- 172.21.15.31:0/9077281 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa9f0013070 con 0x7fa9fc104450 2024-04-25T01:12:37.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.704+0000 7faa04600700 1 -- 172.21.15.31:0/9077281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9fc19bf30 con 0x7fa9fc104450 2024-04-25T01:12:37.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.706+0000 7fa9fa000700 1 -- 172.21.15.31:0/9077281 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+18409 (secure 0 0 0) 0x7fa9f009cc10 con 0x7fa9fc104450 2024-04-25T01:12:37.694 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:12:37.695 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":83,"fsid":"95849b94-029f-11ef-bc93-c7b262605968","created":"2024-04-25T01:04:23.310856+0000","modified":"2024-04-25T01:12:30.554122+0000","last_up_change":"2024-04-25T01:12:29.393907+0000","last_in_change":"2024-04-25T01:12:08.978148+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:18.662391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa08e4c-fe6b-4116-bdf5-1adf5b465d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6803","nonce":788174091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6805","nonce":788174091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6809","nonce":788174091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6807","nonce":788174091}]},"public_addr":"172.21.15.31:6803/788174091","cluster_addr":"172.21.15.31:6805/788174091","heartbeat_back_addr":"172.21.15.31:6809/788174091","heartbeat_front_addr":"172.21.15.31:6807/788174091","state":["exists","up"]},{"osd":1,"uuid":"85339fe7-0dbc-4485-9334-2b039993c1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6801","nonce":3196394509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6803","nonce":3196394509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6807","nonce":3196394509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6805","nonce":3196394509}]},"public_addr":"172.21.15.40:6801/3196394509","cluster_addr":"172.21.15.40:6803/3196394509","heartbeat_back_addr":"172.21.15.40:6807/3196394509","heartbeat_front_addr":"172.21.15.40:6805/3196394509","state":["exists","up"]},{"osd":2,"uuid":"ab62ecb2-8aae-4987-9e2e-589e0f2fee5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6801","nonce":1955211197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6803","nonce":1955211197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6807","nonce":1955211197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6805","nonce":1955211197}]},"public_addr":"172.21.15.169:6801/1955211197","cluster_addr":"172.21.15.169:6803/1955211197","heartbeat_back_addr":"172.21.15.169:6807/1955211197","heartbeat_front_addr":"172.21.15.169:6805/1955211197","state":["exists","up"]},{"osd":3,"uuid":"8b5a7881-3401-495c-8ca4-c90685919d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6811","nonce":1228688643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6813","nonce":1228688643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6817","nonce":1228688643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6815","nonce":1228688643}]},"public_addr":"172.21.15.31:6811/1228688643","cluster_addr":"172.21.15.31:6813/1228688643","heartbeat_back_addr":"172.21.15.31:6817/1228688643","heartbeat_front_addr":"172.21.15.31:6815/1228688643","state":["exists","up"]},{"osd":4,"uuid":"56deb61d-a334-4cf2-8ac7-fbb6e6947092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6809","nonce":1913779091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6811","nonce":1913779091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6815","nonce":1913779091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6813","nonce":1913779091}]},"public_addr":"172.21.15.40:6809/1913779091","cluster_addr":"172.21.15.40:6811/1913779091","heartbeat_back_addr":"172.21.15.40:6815/1913779091","heartbeat_front_addr":"172.21.15.40:6813/1913779091","state":["exists","up"]},{"osd":5,"uuid":"8da55693-f9c7-49a7-99e0-165078f8001b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6809","nonce":2549650781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6811","nonce":2549650781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6815","nonce":2549650781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6813","nonce":2549650781}]},"public_addr":"172.21.15.169:6809/2549650781","cluster_addr":"172.21.15.169:6811/2549650781","heartbeat_back_addr":"172.21.15.169:6815/2549650781","heartbeat_front_addr":"172.21.15.169:6813/2549650781","state":["exists","up"]},{"osd":6,"uuid":"fc8350d2-1ad6-43cb-89f4-d336afc99ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6819","nonce":611507624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6821","nonce":611507624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6825","nonce":611507624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6823","nonce":611507624}]},"public_addr":"172.21.15.31:6819/611507624","cluster_addr":"172.21.15.31:6821/611507624","heartbeat_back_addr":"172.21.15.31:6825/611507624","heartbeat_front_addr":"172.21.15.31:6823/611507624","state":["exists","up"]},{"osd":7,"uuid":"3121b58b-8c94-40b5-9d48-f56552d7c39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6817","nonce":3776085849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6819","nonce":3776085849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6823","nonce":3776085849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6821","nonce":3776085849}]},"public_addr":"172.21.15.40:6817/3776085849","cluster_addr":"172.21.15.40:6819/3776085849","heartbeat_back_addr":"172.21.15.40:6823/3776085849","heartbeat_front_addr":"172.21.15.40:6821/3776085849","state":["exists","up"]},{"osd":8,"uuid":"f60407df-91f5-477a-9702-ddaef24faf82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6817","nonce":2736546122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6819","nonce":2736546122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6823","nonce":2736546122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6821","nonce":2736546122}]},"public_addr":"172.21.15.169:6817/2736546122","cluster_addr":"172.21.15.169:6819/2736546122","heartbeat_back_addr":"172.21.15.169:6823/2736546122","heartbeat_front_addr":"172.21.15.169:6821/2736546122","state":["exists","up"]},{"osd":9,"uuid":"7506a407-1960-49bc-9e38-7f2a8bf996ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6827","nonce":1187207880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6829","nonce":1187207880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6833","nonce":1187207880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6831","nonce":1187207880}]},"public_addr":"172.21.15.31:6827/1187207880","cluster_addr":"172.21.15.31:6829/1187207880","heartbeat_back_addr":"172.21.15.31:6833/1187207880","heartbeat_front_addr":"172.21.15.31:6831/1187207880","state":["exists","up"]},{"osd":10,"uuid":"831c48e2-b0d0-451f-959b-228e927542cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6825","nonce":3050148001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6827","nonce":3050148001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6831","nonce":3050148001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6829","nonce":3050148001}]},"public_addr":"172.21.15.40:6825/3050148001","cluster_addr":"172.21.15.40:6827/3050148001","heartbeat_back_addr":"172.21.15.40:6831/3050148001","heartbeat_front_addr":"172.21.15.40:6829/3050148001","state":["exists","up"]},{"osd":11,"uuid":"144fe0aa-f757-4b3c-90ad-99775e17d57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6825","nonce":262401944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6827","nonce":262401944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6831","nonce":262401944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6829","nonce":262401944}]},"public_addr":"172.21.15.169:6825/262401944","cluster_addr":"172.21.15.169:6827/262401944","heartbeat_back_addr":"172.21.15.169:6831/262401944","heartbeat_front_addr":"172.21.15.169:6829/262401944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:25.640995+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:51.036909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.635633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:48.467912+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:14.454971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:37.609315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:12.441807+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:38.239215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:02.230758+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:35.762553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:02.051405+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:26.569593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6801/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4004014066":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/1930101646":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/2638309002":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2238638505":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2902841144":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4190696323":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/480558521":"2024-04-26T01:04:57.149272+0000","172.21.15.31:6801/2638309002":"2024-04-26T01:04:57.149272+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:12:37.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.709+0000 7faa04600700 1 -- 172.21.15.31:0/9077281 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa9e40a2ca0 msgr2=0x7fa9e40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:37.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.709+0000 7faa04600700 1 --2- 172.21.15.31:0/9077281 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa9e40a2ca0 0x7fa9e40a5160 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc109ae0 tx=0x7fa9e8009260 comp rx=0 tx=0).stop 2024-04-25T01:12:37.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.709+0000 7faa04600700 1 -- 172.21.15.31:0/9077281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9fc104450 msgr2=0x7fa9fc1a0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:37.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.709+0000 7faa04600700 1 --2- 172.21.15.31:0/9077281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9fc104450 0x7fa9fc1a0840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0000c00 tx=0x7fa9f000bab0 comp rx=0 tx=0).stop 2024-04-25T01:12:37.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.709+0000 7faa04600700 1 -- 172.21.15.31:0/9077281 shutdown_connections 2024-04-25T01:12:37.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.709+0000 7faa04600700 1 --2- 172.21.15.31:0/9077281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9fc10a4e0 0x7fa9fc19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:37.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.709+0000 7faa04600700 1 --2- 172.21.15.31:0/9077281 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa9e40a2ca0 0x7fa9e40a5160 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:37.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.709+0000 7faa04600700 1 --2- 172.21.15.31:0/9077281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9fc104450 0x7fa9fc1a0840 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:37.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.709+0000 7faa04600700 1 --2- 172.21.15.31:0/9077281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9fc104e10 0x7fa9fc1a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:37.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.709+0000 7faa04600700 1 -- 172.21.15.31:0/9077281 >> 172.21.15.31:0/9077281 conn(0x7fa9fc0fff40 msgr2=0x7fa9fc100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:37.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.709+0000 7faa04600700 1 -- 172.21.15.31:0/9077281 shutdown_connections 2024-04-25T01:12:37.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:37.709+0000 7faa04600700 1 -- 172.21.15.31:0/9077281 wait complete. 2024-04-25T01:12:37.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:37 smithi169 ceph-mon[32759]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:37.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:37 smithi040 ceph-mon[32518]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:38.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:37 smithi031 ceph-mon[28053]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:38.476 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:18.662391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:12:38.476 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T01:12:38.945 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:12:38.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:38 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/9077281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:12:38.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:38 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/9077281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:12:39.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:38 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/9077281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:12:39.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:39 smithi031 ceph-mon[28053]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:39.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:39 smithi169 ceph-mon[32759]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:39.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:39 smithi040 ceph-mon[32518]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:41.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.093+0000 7fb0bbe00700 1 -- 172.21.15.31:0/1299421781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0bc1096a0 msgr2=0x7fb0bc111270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:41.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.093+0000 7fb0bbe00700 1 --2- 172.21.15.31:0/1299421781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0bc1096a0 0x7fb0bc111270 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400b0d0 tx=0x7fb0b400b3e0 comp rx=0 tx=0).stop 2024-04-25T01:12:41.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.093+0000 7fb0bbe00700 1 -- 172.21.15.31:0/1299421781 shutdown_connections 2024-04-25T01:12:41.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.093+0000 7fb0bbe00700 1 --2- 172.21.15.31:0/1299421781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0bc1096a0 0x7fb0bc111270 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:41.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.093+0000 7fb0bbe00700 1 --2- 172.21.15.31:0/1299421781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0bc1045c0 0x7fb0bc1049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:41.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.093+0000 7fb0bbe00700 1 --2- 172.21.15.31:0/1299421781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0bc104f70 0x7fb0bc108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:41.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.093+0000 7fb0bbe00700 1 -- 172.21.15.31:0/1299421781 >> 172.21.15.31:0/1299421781 conn(0x7fb0bc0ffe50 msgr2=0x7fb0bc102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:41.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.093+0000 7fb0bbe00700 1 -- 172.21.15.31:0/1299421781 shutdown_connections 2024-04-25T01:12:41.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.094+0000 7fb0bbe00700 1 -- 172.21.15.31:0/1299421781 wait complete. 2024-04-25T01:12:41.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.095+0000 7fb0bbe00700 1 Processor -- start 2024-04-25T01:12:41.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.095+0000 7fb0bbe00700 1 -- start start 2024-04-25T01:12:41.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.095+0000 7fb0bbe00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0bc1045c0 0x7fb0bc1a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:41.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.095+0000 7fb0bbe00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0bc104f70 0x7fb0bc1a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:41.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.095+0000 7fb0bbe00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0bc1096a0 0x7fb0bc19a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:41.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.095+0000 7fb0bbe00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc1146e0 con 0x7fb0bc104f70 2024-04-25T01:12:41.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.095+0000 7fb0bbe00700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc114560 con 0x7fb0bc1096a0 2024-04-25T01:12:41.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.095+0000 7fb0bbe00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc114860 con 0x7fb0bc1045c0 2024-04-25T01:12:41.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.096+0000 7fb0ba000700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0bc104f70 0x7fb0bc1a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:41.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.096+0000 7fb0baa00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0bc1045c0 0x7fb0bc1a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:41.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.096+0000 7fb0baa00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0bc1045c0 0x7fb0bc1a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:44568/0 (socket says 172.21.15.31:44568) 2024-04-25T01:12:41.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.096+0000 7fb0ba000700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0bc104f70 0x7fb0bc1a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49474/0 (socket says 172.21.15.31:49474) 2024-04-25T01:12:41.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.096+0000 7fb0baa00700 1 -- 172.21.15.31:0/1710560988 learned_addr learned my addr 172.21.15.31:0/1710560988 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:12:41.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.096+0000 7fb0bb400700 1 --2- 172.21.15.31:0/1710560988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0bc1096a0 0x7fb0bc19a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:41.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.096+0000 7fb0baa00700 1 -- 172.21.15.31:0/1710560988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0bc1096a0 msgr2=0x7fb0bc19a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:41.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.096+0000 7fb0baa00700 1 --2- 172.21.15.31:0/1710560988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0bc1096a0 0x7fb0bc19a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:41.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.096+0000 7fb0baa00700 1 -- 172.21.15.31:0/1710560988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0bc104f70 msgr2=0x7fb0bc1a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:41.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.096+0000 7fb0baa00700 1 --2- 172.21.15.31:0/1710560988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0bc104f70 0x7fb0bc1a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:41.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.096+0000 7fb0baa00700 1 -- 172.21.15.31:0/1710560988 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0bc19b020 con 0x7fb0bc1045c0 2024-04-25T01:12:41.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.096+0000 7fb0bb400700 1 --2- 172.21.15.31:0/1710560988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0bc1096a0 0x7fb0bc19a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:12:41.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.096+0000 7fb0baa00700 1 --2- 172.21.15.31:0/1710560988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0bc1045c0 0x7fb0bc1a0700 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00e9a0 tx=0x7fb0ac00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:41.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.097+0000 7fb0b3400700 1 -- 172.21.15.31:0/1710560988 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac00ca10 con 0x7fb0bc1045c0 2024-04-25T01:12:41.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.097+0000 7fb0b3400700 1 -- 172.21.15.31:0/1710560988 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0ac00cbb0 con 0x7fb0bc1045c0 2024-04-25T01:12:41.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.097+0000 7fb0bbe00700 1 -- 172.21.15.31:0/1710560988 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0bc19b310 con 0x7fb0bc1045c0 2024-04-25T01:12:41.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.097+0000 7fb0b3400700 1 -- 172.21.15.31:0/1710560988 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac018680 con 0x7fb0bc1045c0 2024-04-25T01:12:41.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.097+0000 7fb0bbe00700 1 -- 172.21.15.31:0/1710560988 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0bc1a75d0 con 0x7fb0bc1045c0 2024-04-25T01:12:41.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.098+0000 7fb0bbe00700 1 -- 172.21.15.31:0/1710560988 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0bc068f20 con 0x7fb0bc1045c0 2024-04-25T01:12:41.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.100+0000 7fb0b3400700 1 -- 172.21.15.31:0/1710560988 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb0ac010430 con 0x7fb0bc1045c0 2024-04-25T01:12:41.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.101+0000 7fb0b3400700 1 --2- 172.21.15.31:0/1710560988 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb0a00a2c20 0x7fb0a00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:41.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.102+0000 7fb0b3400700 1 -- 172.21.15.31:0/1710560988 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fb0ac0cf240 con 0x7fb0bc1045c0 2024-04-25T01:12:41.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.102+0000 7fb0ba000700 1 --2- 172.21.15.31:0/1710560988 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb0a00a2c20 0x7fb0a00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:41.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.102+0000 7fb0ba000700 1 --2- 172.21.15.31:0/1710560988 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb0a00a2c20 0x7fb0a00a50e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc103550 tx=0x7fb0a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:41.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.104+0000 7fb0b3400700 1 -- 172.21.15.31:0/1710560988 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb0ac09c5a0 con 0x7fb0bc1045c0 2024-04-25T01:12:41.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.330+0000 7fb0bbe00700 1 -- 172.21.15.31:0/1710560988 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb0bc02db30 con 0x7fb0bc1045c0 2024-04-25T01:12:41.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.331+0000 7fb0b3400700 1 -- 172.21.15.31:0/1710560988 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7fb0ac09bea0 con 0x7fb0bc1045c0 2024-04-25T01:12:41.319 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T01:12:41.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.334+0000 7fb0bbe00700 1 -- 172.21.15.31:0/1710560988 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb0a00a2c20 msgr2=0x7fb0a00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:41.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.334+0000 7fb0bbe00700 1 --2- 172.21.15.31:0/1710560988 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb0a00a2c20 0x7fb0a00a50e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc103550 tx=0x7fb0a8009260 comp rx=0 tx=0).stop 2024-04-25T01:12:41.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.334+0000 7fb0bbe00700 1 -- 172.21.15.31:0/1710560988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0bc1045c0 msgr2=0x7fb0bc1a0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:41.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.334+0000 7fb0bbe00700 1 --2- 172.21.15.31:0/1710560988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0bc1045c0 0x7fb0bc1a0700 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00e9a0 tx=0x7fb0ac00ecb0 comp rx=0 tx=0).stop 2024-04-25T01:12:41.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.334+0000 7fb0bbe00700 1 -- 172.21.15.31:0/1710560988 shutdown_connections 2024-04-25T01:12:41.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.334+0000 7fb0bbe00700 1 --2- 172.21.15.31:0/1710560988 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb0a00a2c20 0x7fb0a00a50e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:41.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.334+0000 7fb0bbe00700 1 --2- 172.21.15.31:0/1710560988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0bc1096a0 0x7fb0bc19a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:41.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.334+0000 7fb0bbe00700 1 --2- 172.21.15.31:0/1710560988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0bc1045c0 0x7fb0bc1a0700 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:41.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.334+0000 7fb0bbe00700 1 --2- 172.21.15.31:0/1710560988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0bc104f70 0x7fb0bc1a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:41.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.334+0000 7fb0bbe00700 1 -- 172.21.15.31:0/1710560988 >> 172.21.15.31:0/1710560988 conn(0x7fb0bc0ffe50 msgr2=0x7fb0bc100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:41.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.335+0000 7fb0bbe00700 1 -- 172.21.15.31:0/1710560988 shutdown_connections 2024-04-25T01:12:41.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:41.335+0000 7fb0bbe00700 1 -- 172.21.15.31:0/1710560988 wait complete. 2024-04-25T01:12:41.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:41 smithi169 ceph-mon[32759]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:41.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:41 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1710560988' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:12:41.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:41 smithi040 ceph-mon[32518]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:41.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:41 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1710560988' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:12:42.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:41 smithi031 ceph-mon[28053]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:42.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:41 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1710560988' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:12:42.216 INFO:tasks.cephadm:Adding mds.a on smithi031 2024-04-25T01:12:42.216 INFO:tasks.cephadm:Adding mds.d on smithi031 2024-04-25T01:12:42.216 INFO:tasks.cephadm:Adding mds.g on smithi031 2024-04-25T01:12:42.216 INFO:tasks.cephadm:Adding mds.j on smithi031 2024-04-25T01:12:42.216 INFO:tasks.cephadm:Adding mds.b on smithi040 2024-04-25T01:12:42.216 INFO:tasks.cephadm:Adding mds.e on smithi040 2024-04-25T01:12:42.216 INFO:tasks.cephadm:Adding mds.h on smithi040 2024-04-25T01:12:42.216 INFO:tasks.cephadm:Adding mds.k on smithi040 2024-04-25T01:12:42.216 INFO:tasks.cephadm:Adding mds.c on smithi169 2024-04-25T01:12:42.217 INFO:tasks.cephadm:Adding mds.f on smithi169 2024-04-25T01:12:42.217 INFO:tasks.cephadm:Adding mds.i on smithi169 2024-04-25T01:12:42.217 INFO:tasks.cephadm:Adding mds.l on smithi169 2024-04-25T01:12:42.217 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi040=b;smithi040=e;smithi040=h;smithi040=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l' 2024-04-25T01:12:42.470 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:12:43.775 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:43 smithi169 ceph-mon[32759]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:43.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:43 smithi040 ceph-mon[32518]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:43.983 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.994+0000 7f6712c69700 1 -- 172.21.15.169:0/3059675582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f670c100540 msgr2=0x7f670c100960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:43.983 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.994+0000 7f6712c69700 1 --2- 172.21.15.169:0/3059675582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f670c100540 0x7f670c100960 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6700009990 tx=0x7f6700009ca0 comp rx=0 tx=0).stop 2024-04-25T01:12:43.983 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.994+0000 7f6712c69700 1 -- 172.21.15.169:0/3059675582 shutdown_connections 2024-04-25T01:12:43.983 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.994+0000 7f6712c69700 1 --2- 172.21.15.169:0/3059675582 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f670c103ec0 0x7f670c10a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:43.983 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.994+0000 7f6712c69700 1 --2- 172.21.15.169:0/3059675582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f670c100540 0x7f670c100960 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:43.983 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.994+0000 7f6712c69700 1 --2- 172.21.15.169:0/3059675582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f670c103520 0x7f670c103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:43.983 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.994+0000 7f6712c69700 1 -- 172.21.15.169:0/3059675582 >> 172.21.15.169:0/3059675582 conn(0x7f670c0fbec0 msgr2=0x7f670c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:43.983 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.994+0000 7f6712c69700 1 -- 172.21.15.169:0/3059675582 shutdown_connections 2024-04-25T01:12:43.983 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.994+0000 7f6712c69700 1 -- 172.21.15.169:0/3059675582 wait complete. 2024-04-25T01:12:43.983 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.995+0000 7f6712c69700 1 Processor -- start 2024-04-25T01:12:43.984 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.995+0000 7f6712c69700 1 -- start start 2024-04-25T01:12:43.984 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.995+0000 7f6712c69700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f670c100540 0x7f670c198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:43.984 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.995+0000 7f6712c69700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f670c103520 0x7f670c198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:43.984 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.995+0000 7f6712c69700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f670c103ec0 0x7f670c19fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:43.984 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.995+0000 7f6712c69700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f670c10df30 con 0x7f670c100540 2024-04-25T01:12:43.984 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.995+0000 7f6712c69700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f670c10ddb0 con 0x7f670c103ec0 2024-04-25T01:12:43.984 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.995+0000 7f6712c69700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f670c10e0b0 con 0x7f670c103520 2024-04-25T01:12:43.984 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.995+0000 7f670bfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f670c103520 0x7f670c198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:43.984 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.995+0000 7f670bfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f670c103520 0x7f670c198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:34216/0 (socket says 172.21.15.169:34216) 2024-04-25T01:12:43.984 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.995+0000 7f670bfff700 1 -- 172.21.15.169:0/3516992527 learned_addr learned my addr 172.21.15.169:0/3516992527 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T01:12:43.984 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.995+0000 7f6711206700 1 --2- 172.21.15.169:0/3516992527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f670c103ec0 0x7f670c19fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:43.984 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.995+0000 7f670bfff700 1 -- 172.21.15.169:0/3516992527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f670c103ec0 msgr2=0x7f670c19fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:43.984 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.995+0000 7f670bfff700 1 --2- 172.21.15.169:0/3516992527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f670c103ec0 0x7f670c19fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:43.984 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.995+0000 7f670bfff700 1 -- 172.21.15.169:0/3516992527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f670c100540 msgr2=0x7f670c198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:43.985 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.995+0000 7f670bfff700 1 --2- 172.21.15.169:0/3516992527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f670c100540 0x7f670c198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:43.985 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.995+0000 7f670bfff700 1 -- 172.21.15.169:0/3516992527 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670c1a0260 con 0x7f670c103520 2024-04-25T01:12:43.985 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.995+0000 7f670bfff700 1 --2- 172.21.15.169:0/3516992527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f670c103520 0x7f670c198b40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f66f800b560 tx=0x7f66f800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:43.985 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.996+0000 7f6709ffb700 1 -- 172.21.15.169:0/3516992527 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f8015020 con 0x7f670c103520 2024-04-25T01:12:43.985 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.996+0000 7f6712c69700 1 -- 172.21.15.169:0/3516992527 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670c1a0550 con 0x7f670c103520 2024-04-25T01:12:43.985 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.996+0000 7f6709ffb700 1 -- 172.21.15.169:0/3516992527 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66f8011d10 con 0x7f670c103520 2024-04-25T01:12:43.985 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.996+0000 7f6709ffb700 1 -- 172.21.15.169:0/3516992527 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f8010e50 con 0x7f670c103520 2024-04-25T01:12:43.985 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.996+0000 7f6712c69700 1 -- 172.21.15.169:0/3516992527 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f670c1a0b60 con 0x7f670c103520 2024-04-25T01:12:43.986 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.997+0000 7f6712c69700 1 -- 172.21.15.169:0/3516992527 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f670c068f20 con 0x7f670c103520 2024-04-25T01:12:43.986 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.998+0000 7f6709ffb700 1 -- 172.21.15.169:0/3516992527 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f66f8018020 con 0x7f670c103520 2024-04-25T01:12:43.986 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.998+0000 7f6709ffb700 1 --2- 172.21.15.169:0/3516992527 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f66fc0a2dc0 0x7f66fc0a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:43.986 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.999+0000 7f6709ffb700 1 -- 172.21.15.169:0/3516992527 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f66f80ce310 con 0x7f670c103520 2024-04-25T01:12:43.986 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:43.999+0000 7f6710a05700 1 --2- 172.21.15.169:0/3516992527 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f66fc0a2dc0 0x7f66fc0a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:43.987 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:44.000+0000 7f6710a05700 1 --2- 172.21.15.169:0/3516992527 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f66fc0a2dc0 0x7f66fc0a5280 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6700005f50 tx=0x7f6700005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:43.990 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:44.003+0000 7f6709ffb700 1 -- 172.21.15.169:0/3516992527 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f66f801f020 con 0x7f670c103520 2024-04-25T01:12:44.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:43 smithi031 ceph-mon[28053]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:44.199 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:44.212+0000 7f6712c69700 1 -- 172.21.15.169:0/3516992527 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi040=b;smithi040=e;smithi040=h;smithi040=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l", "target": ["mon-mgr", ""]}) -- 0x7f670c062870 con 0x7f66fc0a2dc0 2024-04-25T01:12:44.215 INFO:teuthology.orchestra.run.smithi169.stdout:Scheduled mds.all update... 2024-04-25T01:12:44.215 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:44.228+0000 7f6709ffb700 1 -- 172.21.15.169:0/3516992527 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f670c062870 con 0x7f66fc0a2dc0 2024-04-25T01:12:44.218 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:44.230+0000 7f6712c69700 1 -- 172.21.15.169:0/3516992527 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f66fc0a2dc0 msgr2=0x7f66fc0a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:44.218 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:44.230+0000 7f6712c69700 1 --2- 172.21.15.169:0/3516992527 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f66fc0a2dc0 0x7f66fc0a5280 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6700005f50 tx=0x7f6700005e80 comp rx=0 tx=0).stop 2024-04-25T01:12:44.218 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:44.230+0000 7f6712c69700 1 -- 172.21.15.169:0/3516992527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f670c103520 msgr2=0x7f670c198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:44.218 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:44.230+0000 7f6712c69700 1 --2- 172.21.15.169:0/3516992527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f670c103520 0x7f670c198b40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f66f800b560 tx=0x7f66f800b870 comp rx=0 tx=0).stop 2024-04-25T01:12:44.218 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:44.231+0000 7f6712c69700 1 -- 172.21.15.169:0/3516992527 shutdown_connections 2024-04-25T01:12:44.218 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:44.231+0000 7f6712c69700 1 --2- 172.21.15.169:0/3516992527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f670c100540 0x7f670c198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:44.218 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:44.231+0000 7f6712c69700 1 --2- 172.21.15.169:0/3516992527 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f66fc0a2dc0 0x7f66fc0a5280 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:44.218 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:44.231+0000 7f6712c69700 1 --2- 172.21.15.169:0/3516992527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f670c103ec0 0x7f670c19fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:44.218 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:44.231+0000 7f6712c69700 1 --2- 172.21.15.169:0/3516992527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f670c103520 0x7f670c198b40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:44.218 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:44.231+0000 7f6712c69700 1 -- 172.21.15.169:0/3516992527 >> 172.21.15.169:0/3516992527 conn(0x7f670c0fbec0 msgr2=0x7f670c106890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:44.218 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:44.231+0000 7f6712c69700 1 -- 172.21.15.169:0/3516992527 shutdown_connections 2024-04-25T01:12:44.219 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T01:12:44.231+0000 7f6712c69700 1 -- 172.21.15.169:0/3516992527 wait complete. 2024-04-25T01:12:44.843 DEBUG:teuthology.orchestra.run.smithi031:mds.a> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T01:12:44.851 DEBUG:teuthology.orchestra.run.smithi031:mds.d> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T01:12:44.899 DEBUG:teuthology.orchestra.run.smithi031:mds.g> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T01:12:44.945 DEBUG:teuthology.orchestra.run.smithi031:mds.j> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T01:12:44.990 DEBUG:teuthology.orchestra.run.smithi040:mds.b> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T01:12:44.993 DEBUG:teuthology.orchestra.run.smithi040:mds.e> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T01:12:45.036 DEBUG:teuthology.orchestra.run.smithi040:mds.h> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T01:12:45.079 DEBUG:teuthology.orchestra.run.smithi040:mds.k> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T01:12:45.122 DEBUG:teuthology.orchestra.run.smithi169:mds.c> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T01:12:45.124 DEBUG:teuthology.orchestra.run.smithi169:mds.f> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T01:12:45.168 DEBUG:teuthology.orchestra.run.smithi169:mds.i> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T01:12:45.211 DEBUG:teuthology.orchestra.run.smithi169:mds.l> sudo journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T01:12:45.253 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T01:12:45.254 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:12:45.445 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:45 smithi169 ceph-mon[32759]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:45.446 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:45 smithi169 ceph-mon[32759]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi040=b;smithi040=e;smithi040=h;smithi040=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:12:45.446 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:45 smithi169 ceph-mon[32759]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi040=b;smithi040=e;smithi040=h;smithi040=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l;count:12 2024-04-25T01:12:45.446 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:45 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:45.446 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:45 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:12:45.448 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:45 smithi040 ceph-mon[32518]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:45.448 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:45 smithi040 ceph-mon[32518]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi040=b;smithi040=e;smithi040=h;smithi040=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:12:45.449 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:45 smithi040 ceph-mon[32518]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi040=b;smithi040=e;smithi040=h;smithi040=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l;count:12 2024-04-25T01:12:45.449 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:45 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:45.449 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:45 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:12:45.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.476+0000 7f6f27200640 1 -- 172.21.15.31:0/2071849940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f28101130 msgr2=0x7f6f281015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:45.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.476+0000 7f6f27200640 1 --2- 172.21.15.31:0/2071849940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f28101130 0x7f6f281015b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009860 tx=0x7f6f2002a340 comp rx=0 tx=0).stop 2024-04-25T01:12:45.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.476+0000 7f6f27200640 1 -- 172.21.15.31:0/2071849940 shutdown_connections 2024-04-25T01:12:45.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.476+0000 7f6f27200640 1 --2- 172.21.15.31:0/2071849940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f28101130 0x7f6f281015b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:45.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.476+0000 7f6f27200640 1 --2- 172.21.15.31:0/2071849940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f28100790 0x7f6f28100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:45.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.476+0000 7f6f27200640 1 --2- 172.21.15.31:0/2071849940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f280ffe30 0x7f6f28100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:45.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.476+0000 7f6f27200640 1 -- 172.21.15.31:0/2071849940 >> 172.21.15.31:0/2071849940 conn(0x7f6f280fba10 msgr2=0x7f6f280fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:45.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.477+0000 7f6f27200640 1 -- 172.21.15.31:0/2071849940 shutdown_connections 2024-04-25T01:12:45.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.477+0000 7f6f27200640 1 -- 172.21.15.31:0/2071849940 wait complete. 2024-04-25T01:12:45.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.478+0000 7f6f27200640 1 Processor -- start 2024-04-25T01:12:45.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.478+0000 7f6f27200640 1 -- start start 2024-04-25T01:12:45.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.478+0000 7f6f27200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f280ffe30 0x7f6f28199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:45.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.478+0000 7f6f27200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f28100790 0x7f6f28199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:45.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.478+0000 7f6f27200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f28101130 0x7f6f281a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:45.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.478+0000 7f6f27200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2810f410 con 0x7f6f280ffe30 2024-04-25T01:12:45.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.478+0000 7f6f27200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2810f290 con 0x7f6f28100790 2024-04-25T01:12:45.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.478+0000 7f6f27200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2810f590 con 0x7f6f28101130 2024-04-25T01:12:45.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.479+0000 7f6f25e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f280ffe30 0x7f6f28199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:45.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.479+0000 7f6f1d400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f28100790 0x7f6f28199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:45.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.479+0000 7f6f25e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f280ffe30 0x7f6f28199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49488/0 (socket says 172.21.15.31:49488) 2024-04-25T01:12:45.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.479+0000 7f6f25e00640 1 -- 172.21.15.31:0/4170949303 learned_addr learned my addr 172.21.15.31:0/4170949303 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:12:45.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.479+0000 7f6f26800640 1 --2- 172.21.15.31:0/4170949303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f28101130 0x7f6f281a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:45.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.479+0000 7f6f25e00640 1 -- 172.21.15.31:0/4170949303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f28101130 msgr2=0x7f6f281a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:45.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.479+0000 7f6f25e00640 1 --2- 172.21.15.31:0/4170949303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f28101130 0x7f6f281a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:45.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.479+0000 7f6f25e00640 1 -- 172.21.15.31:0/4170949303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f28100790 msgr2=0x7f6f28199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:45.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.479+0000 7f6f25e00640 1 --2- 172.21.15.31:0/4170949303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f28100790 0x7f6f28199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:45.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.479+0000 7f6f25e00640 1 -- 172.21.15.31:0/4170949303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f281a1670 con 0x7f6f280ffe30 2024-04-25T01:12:45.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.479+0000 7f6f25e00640 1 --2- 172.21.15.31:0/4170949303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f280ffe30 0x7f6f28199a50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800efb0 tx=0x7f6f1800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:45.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.480+0000 7f6f1f400640 1 -- 172.21.15.31:0/4170949303 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1801b070 con 0x7f6f280ffe30 2024-04-25T01:12:45.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.480+0000 7f6f1f400640 1 -- 172.21.15.31:0/4170949303 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f1800ba50 con 0x7f6f280ffe30 2024-04-25T01:12:45.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.480+0000 7f6f1f400640 1 -- 172.21.15.31:0/4170949303 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18016c00 con 0x7f6f280ffe30 2024-04-25T01:12:45.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.480+0000 7f6f27200640 1 -- 172.21.15.31:0/4170949303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f281a1960 con 0x7f6f280ffe30 2024-04-25T01:12:45.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.480+0000 7f6f27200640 1 -- 172.21.15.31:0/4170949303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f281a1f00 con 0x7f6f280ffe30 2024-04-25T01:12:45.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.481+0000 7f6f27200640 1 -- 172.21.15.31:0/4170949303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f28068480 con 0x7f6f280ffe30 2024-04-25T01:12:45.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.483+0000 7f6f1f400640 1 -- 172.21.15.31:0/4170949303 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6f1801e070 con 0x7f6f280ffe30 2024-04-25T01:12:45.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.484+0000 7f6f1f400640 1 --2- 172.21.15.31:0/4170949303 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6f0c0a2c10 0x7f6f0c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:45.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.485+0000 7f6f1f400640 1 -- 172.21.15.31:0/4170949303 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f6f180ceb20 con 0x7f6f280ffe30 2024-04-25T01:12:45.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.485+0000 7f6f1d400640 1 --2- 172.21.15.31:0/4170949303 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6f0c0a2c10 0x7f6f0c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:45.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.485+0000 7f6f1d400640 1 --2- 172.21.15.31:0/4170949303 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6f0c0a2c10 0x7f6f0c0a50d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6f2819af60 tx=0x7f6f10005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:45.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.489+0000 7f6f1f400640 1 -- 172.21.15.31:0/4170949303 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6f1809b830 con 0x7f6f280ffe30 2024-04-25T01:12:45.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:45 smithi031 ceph-mon[28053]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:45.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:45 smithi031 ceph-mon[28053]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi040=b;smithi040=e;smithi040=h;smithi040=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:12:45.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:45 smithi031 ceph-mon[28053]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi040=b;smithi040=e;smithi040=h;smithi040=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l;count:12 2024-04-25T01:12:45.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:45 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:45.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:45 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:12:45.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.700+0000 7f6f27200640 1 -- 172.21.15.31:0/4170949303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f28062340 con 0x7f6f280ffe30 2024-04-25T01:12:45.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.703+0000 7f6f1f400640 1 -- 172.21.15.31:0/4170949303 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+18409 (secure 0 0 0) 0x7f6f1809fb30 con 0x7f6f280ffe30 2024-04-25T01:12:45.691 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:12:45.691 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":83,"fsid":"95849b94-029f-11ef-bc93-c7b262605968","created":"2024-04-25T01:04:23.310856+0000","modified":"2024-04-25T01:12:30.554122+0000","last_up_change":"2024-04-25T01:12:29.393907+0000","last_in_change":"2024-04-25T01:12:08.978148+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:18.662391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa08e4c-fe6b-4116-bdf5-1adf5b465d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6803","nonce":788174091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6805","nonce":788174091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6809","nonce":788174091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6807","nonce":788174091}]},"public_addr":"172.21.15.31:6803/788174091","cluster_addr":"172.21.15.31:6805/788174091","heartbeat_back_addr":"172.21.15.31:6809/788174091","heartbeat_front_addr":"172.21.15.31:6807/788174091","state":["exists","up"]},{"osd":1,"uuid":"85339fe7-0dbc-4485-9334-2b039993c1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6801","nonce":3196394509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6803","nonce":3196394509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6807","nonce":3196394509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6805","nonce":3196394509}]},"public_addr":"172.21.15.40:6801/3196394509","cluster_addr":"172.21.15.40:6803/3196394509","heartbeat_back_addr":"172.21.15.40:6807/3196394509","heartbeat_front_addr":"172.21.15.40:6805/3196394509","state":["exists","up"]},{"osd":2,"uuid":"ab62ecb2-8aae-4987-9e2e-589e0f2fee5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6801","nonce":1955211197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6803","nonce":1955211197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6807","nonce":1955211197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6805","nonce":1955211197}]},"public_addr":"172.21.15.169:6801/1955211197","cluster_addr":"172.21.15.169:6803/1955211197","heartbeat_back_addr":"172.21.15.169:6807/1955211197","heartbeat_front_addr":"172.21.15.169:6805/1955211197","state":["exists","up"]},{"osd":3,"uuid":"8b5a7881-3401-495c-8ca4-c90685919d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6811","nonce":1228688643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6813","nonce":1228688643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6817","nonce":1228688643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6815","nonce":1228688643}]},"public_addr":"172.21.15.31:6811/1228688643","cluster_addr":"172.21.15.31:6813/1228688643","heartbeat_back_addr":"172.21.15.31:6817/1228688643","heartbeat_front_addr":"172.21.15.31:6815/1228688643","state":["exists","up"]},{"osd":4,"uuid":"56deb61d-a334-4cf2-8ac7-fbb6e6947092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6809","nonce":1913779091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6811","nonce":1913779091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6815","nonce":1913779091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6813","nonce":1913779091}]},"public_addr":"172.21.15.40:6809/1913779091","cluster_addr":"172.21.15.40:6811/1913779091","heartbeat_back_addr":"172.21.15.40:6815/1913779091","heartbeat_front_addr":"172.21.15.40:6813/1913779091","state":["exists","up"]},{"osd":5,"uuid":"8da55693-f9c7-49a7-99e0-165078f8001b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6809","nonce":2549650781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6811","nonce":2549650781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6815","nonce":2549650781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6813","nonce":2549650781}]},"public_addr":"172.21.15.169:6809/2549650781","cluster_addr":"172.21.15.169:6811/2549650781","heartbeat_back_addr":"172.21.15.169:6815/2549650781","heartbeat_front_addr":"172.21.15.169:6813/2549650781","state":["exists","up"]},{"osd":6,"uuid":"fc8350d2-1ad6-43cb-89f4-d336afc99ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6819","nonce":611507624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6821","nonce":611507624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6825","nonce":611507624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6823","nonce":611507624}]},"public_addr":"172.21.15.31:6819/611507624","cluster_addr":"172.21.15.31:6821/611507624","heartbeat_back_addr":"172.21.15.31:6825/611507624","heartbeat_front_addr":"172.21.15.31:6823/611507624","state":["exists","up"]},{"osd":7,"uuid":"3121b58b-8c94-40b5-9d48-f56552d7c39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6817","nonce":3776085849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6819","nonce":3776085849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6823","nonce":3776085849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6821","nonce":3776085849}]},"public_addr":"172.21.15.40:6817/3776085849","cluster_addr":"172.21.15.40:6819/3776085849","heartbeat_back_addr":"172.21.15.40:6823/3776085849","heartbeat_front_addr":"172.21.15.40:6821/3776085849","state":["exists","up"]},{"osd":8,"uuid":"f60407df-91f5-477a-9702-ddaef24faf82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6817","nonce":2736546122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6819","nonce":2736546122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6823","nonce":2736546122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6821","nonce":2736546122}]},"public_addr":"172.21.15.169:6817/2736546122","cluster_addr":"172.21.15.169:6819/2736546122","heartbeat_back_addr":"172.21.15.169:6823/2736546122","heartbeat_front_addr":"172.21.15.169:6821/2736546122","state":["exists","up"]},{"osd":9,"uuid":"7506a407-1960-49bc-9e38-7f2a8bf996ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6827","nonce":1187207880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6829","nonce":1187207880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6833","nonce":1187207880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6831","nonce":1187207880}]},"public_addr":"172.21.15.31:6827/1187207880","cluster_addr":"172.21.15.31:6829/1187207880","heartbeat_back_addr":"172.21.15.31:6833/1187207880","heartbeat_front_addr":"172.21.15.31:6831/1187207880","state":["exists","up"]},{"osd":10,"uuid":"831c48e2-b0d0-451f-959b-228e927542cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6825","nonce":3050148001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6827","nonce":3050148001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6831","nonce":3050148001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6829","nonce":3050148001}]},"public_addr":"172.21.15.40:6825/3050148001","cluster_addr":"172.21.15.40:6827/3050148001","heartbeat_back_addr":"172.21.15.40:6831/3050148001","heartbeat_front_addr":"172.21.15.40:6829/3050148001","state":["exists","up"]},{"osd":11,"uuid":"144fe0aa-f757-4b3c-90ad-99775e17d57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6825","nonce":262401944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6827","nonce":262401944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6831","nonce":262401944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6829","nonce":262401944}]},"public_addr":"172.21.15.169:6825/262401944","cluster_addr":"172.21.15.169:6827/262401944","heartbeat_back_addr":"172.21.15.169:6831/262401944","heartbeat_front_addr":"172.21.15.169:6829/262401944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:25.640995+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:51.036909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.635633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:48.467912+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:14.454971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:37.609315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:12.441807+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:38.239215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:02.230758+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:35.762553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:02.051405+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:26.569593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6801/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4004014066":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/1930101646":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/2638309002":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2238638505":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2902841144":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4190696323":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/480558521":"2024-04-26T01:04:57.149272+0000","172.21.15.31:6801/2638309002":"2024-04-26T01:04:57.149272+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:12:45.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.706+0000 7f6f27200640 1 -- 172.21.15.31:0/4170949303 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6f0c0a2c10 msgr2=0x7f6f0c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:45.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.706+0000 7f6f27200640 1 --2- 172.21.15.31:0/4170949303 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6f0c0a2c10 0x7f6f0c0a50d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6f2819af60 tx=0x7f6f10005f90 comp rx=0 tx=0).stop 2024-04-25T01:12:45.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.706+0000 7f6f27200640 1 -- 172.21.15.31:0/4170949303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f280ffe30 msgr2=0x7f6f28199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:45.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.706+0000 7f6f27200640 1 --2- 172.21.15.31:0/4170949303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f280ffe30 0x7f6f28199a50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800efb0 tx=0x7f6f1800c800 comp rx=0 tx=0).stop 2024-04-25T01:12:45.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.706+0000 7f6f27200640 1 -- 172.21.15.31:0/4170949303 shutdown_connections 2024-04-25T01:12:45.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.706+0000 7f6f27200640 1 --2- 172.21.15.31:0/4170949303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f28101130 0x7f6f281a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:45.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.706+0000 7f6f27200640 1 --2- 172.21.15.31:0/4170949303 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6f0c0a2c10 0x7f6f0c0a50d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:45.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.706+0000 7f6f27200640 1 --2- 172.21.15.31:0/4170949303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f28100790 0x7f6f28199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:45.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.706+0000 7f6f27200640 1 --2- 172.21.15.31:0/4170949303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f280ffe30 0x7f6f28199a50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:45.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.706+0000 7f6f27200640 1 -- 172.21.15.31:0/4170949303 >> 172.21.15.31:0/4170949303 conn(0x7f6f280fba10 msgr2=0x7f6f280fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:45.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.707+0000 7f6f27200640 1 -- 172.21.15.31:0/4170949303 shutdown_connections 2024-04-25T01:12:45.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.707+0000 7f6f27200640 1 -- 172.21.15.31:0/4170949303 wait complete. 2024-04-25T01:12:45.713 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:18.662391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:12:45.714 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:12:45.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.951+0000 7f2e51400640 1 -- 172.21.15.31:0/94847032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e4c077340 msgr2=0x7f2e4c0777a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:45.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.951+0000 7f2e51400640 1 --2- 172.21.15.31:0/94847032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e4c077340 0x7f2e4c0777a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009900 tx=0x7f2e3802a410 comp rx=0 tx=0).stop 2024-04-25T01:12:45.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.951+0000 7f2e51400640 1 -- 172.21.15.31:0/94847032 shutdown_connections 2024-04-25T01:12:45.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.951+0000 7f2e51400640 1 --2- 172.21.15.31:0/94847032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e4c077ce0 0x7f2e4c078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:45.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.951+0000 7f2e51400640 1 --2- 172.21.15.31:0/94847032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e4c077340 0x7f2e4c0777a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:45.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.951+0000 7f2e51400640 1 --2- 172.21.15.31:0/94847032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e4c076100 0x7f2e4c076520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:45.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.951+0000 7f2e51400640 1 -- 172.21.15.31:0/94847032 >> 172.21.15.31:0/94847032 conn(0x7f2e4c0fedf0 msgr2=0x7f2e4c101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:45.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.951+0000 7f2e51400640 1 -- 172.21.15.31:0/94847032 shutdown_connections 2024-04-25T01:12:45.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.951+0000 7f2e51400640 1 -- 172.21.15.31:0/94847032 wait complete. 2024-04-25T01:12:45.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.952+0000 7f2e51400640 1 Processor -- start 2024-04-25T01:12:45.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.952+0000 7f2e51400640 1 -- start start 2024-04-25T01:12:45.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.952+0000 7f2e51400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e4c076100 0x7f2e4c19df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:45.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.953+0000 7f2e51400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e4c077340 0x7f2e4c19e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:45.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.953+0000 7f2e51400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e4c077ce0 0x7f2e4c1a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:45.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.953+0000 7f2e51400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c1138d0 con 0x7f2e4c077ce0 2024-04-25T01:12:45.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.953+0000 7f2e51400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c113750 con 0x7f2e4c076100 2024-04-25T01:12:45.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.953+0000 7f2e51400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c113a50 con 0x7f2e4c077340 2024-04-25T01:12:45.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.953+0000 7f2e50a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e4c077ce0 0x7f2e4c1a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:45.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.953+0000 7f2e4be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e4c076100 0x7f2e4c19df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:45.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.953+0000 7f2e4be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e4c076100 0x7f2e4c19df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:58114/0 (socket says 172.21.15.31:58114) 2024-04-25T01:12:45.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.953+0000 7f2e4b400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e4c077340 0x7f2e4c19e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:45.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.953+0000 7f2e4be00640 1 -- 172.21.15.31:0/753194336 learned_addr learned my addr 172.21.15.31:0/753194336 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:12:45.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.953+0000 7f2e4be00640 1 -- 172.21.15.31:0/753194336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e4c077340 msgr2=0x7f2e4c19e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:45.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.953+0000 7f2e4be00640 1 --2- 172.21.15.31:0/753194336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e4c077340 0x7f2e4c19e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:45.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.953+0000 7f2e4be00640 1 -- 172.21.15.31:0/753194336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e4c077ce0 msgr2=0x7f2e4c1a5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:45.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.953+0000 7f2e4be00640 1 --2- 172.21.15.31:0/753194336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e4c077ce0 0x7f2e4c1a5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:45.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.953+0000 7f2e4be00640 1 -- 172.21.15.31:0/753194336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4c1a5ba0 con 0x7f2e4c076100 2024-04-25T01:12:45.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.953+0000 7f2e4b400640 1 --2- 172.21.15.31:0/753194336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e4c077340 0x7f2e4c19e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:12:45.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.953+0000 7f2e50a00640 1 --2- 172.21.15.31:0/753194336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e4c077ce0 0x7f2e4c1a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:12:45.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.954+0000 7f2e4be00640 1 --2- 172.21.15.31:0/753194336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e4c076100 0x7f2e4c19df80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c00e9f0 tx=0x7f2e3c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:45.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.954+0000 7f2e48c00640 1 -- 172.21.15.31:0/753194336 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c00cd80 con 0x7f2e4c076100 2024-04-25T01:12:45.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.954+0000 7f2e48c00640 1 -- 172.21.15.31:0/753194336 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e3c010910 con 0x7f2e4c076100 2024-04-25T01:12:45.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.954+0000 7f2e48c00640 1 -- 172.21.15.31:0/753194336 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c018990 con 0x7f2e4c076100 2024-04-25T01:12:45.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.954+0000 7f2e51400640 1 -- 172.21.15.31:0/753194336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4c1a5e90 con 0x7f2e4c076100 2024-04-25T01:12:45.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.954+0000 7f2e51400640 1 -- 172.21.15.31:0/753194336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e4c1a6300 con 0x7f2e4c076100 2024-04-25T01:12:45.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.955+0000 7f2e51400640 1 -- 172.21.15.31:0/753194336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e4c1101f0 con 0x7f2e4c076100 2024-04-25T01:12:45.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.957+0000 7f2e48c00640 1 -- 172.21.15.31:0/753194336 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2e3c010ac0 con 0x7f2e4c076100 2024-04-25T01:12:45.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.959+0000 7f2e48c00640 1 --2- 172.21.15.31:0/753194336 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f2e300a2c10 0x7f2e300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:45.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.959+0000 7f2e48c00640 1 -- 172.21.15.31:0/753194336 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f2e3c0cf450 con 0x7f2e4c076100 2024-04-25T01:12:45.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.959+0000 7f2e4b400640 1 --2- 172.21.15.31:0/753194336 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f2e300a2c10 0x7f2e300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.960+0000 7f2e4b400640 1 --2- 172.21.15.31:0/753194336 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f2e300a2c10 0x7f2e300a50d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2e380314e0 tx=0x7f2e38000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:45.961+0000 7f2e48c00640 1 -- 172.21.15.31:0/753194336 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2e3c09c160 con 0x7f2e4c076100 2024-04-25T01:12:46.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.166+0000 7f2e51400640 1 -- 172.21.15.31:0/753194336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2e4c02d3e0 con 0x7f2e4c076100 2024-04-25T01:12:46.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.167+0000 7f2e48c00640 1 -- 172.21.15.31:0/753194336 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f2e3c0a0460 con 0x7f2e4c076100 2024-04-25T01:12:46.154 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T01:12:46.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.170+0000 7f2e51400640 1 -- 172.21.15.31:0/753194336 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f2e300a2c10 msgr2=0x7f2e300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:46.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.170+0000 7f2e51400640 1 --2- 172.21.15.31:0/753194336 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f2e300a2c10 0x7f2e300a50d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2e380314e0 tx=0x7f2e38000bc0 comp rx=0 tx=0).stop 2024-04-25T01:12:46.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.170+0000 7f2e51400640 1 -- 172.21.15.31:0/753194336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e4c076100 msgr2=0x7f2e4c19df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:46.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.170+0000 7f2e51400640 1 --2- 172.21.15.31:0/753194336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e4c076100 0x7f2e4c19df80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c00e9f0 tx=0x7f2e3c00eeb0 comp rx=0 tx=0).stop 2024-04-25T01:12:46.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.170+0000 7f2e51400640 1 -- 172.21.15.31:0/753194336 shutdown_connections 2024-04-25T01:12:46.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.170+0000 7f2e51400640 1 --2- 172.21.15.31:0/753194336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e4c077ce0 0x7f2e4c1a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:46.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.170+0000 7f2e51400640 1 --2- 172.21.15.31:0/753194336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e4c077340 0x7f2e4c19e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:46.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.170+0000 7f2e51400640 1 --2- 172.21.15.31:0/753194336 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f2e300a2c10 0x7f2e300a50d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:46.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.170+0000 7f2e51400640 1 --2- 172.21.15.31:0/753194336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2e4c076100 0x7f2e4c19df80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:46.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.171+0000 7f2e51400640 1 -- 172.21.15.31:0/753194336 >> 172.21.15.31:0/753194336 conn(0x7f2e4c0fedf0 msgr2=0x7f2e4c1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:46.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.171+0000 7f2e51400640 1 -- 172.21.15.31:0/753194336 shutdown_connections 2024-04-25T01:12:46.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.171+0000 7f2e51400640 1 -- 172.21.15.31:0/753194336 wait complete. 2024-04-25T01:12:46.179 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:12:46.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.410+0000 7fb7db400640 1 -- 172.21.15.31:0/1055278248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d4104bb0 msgr2=0x7fb7d4105030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:46.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.410+0000 7fb7db400640 1 --2- 172.21.15.31:0/1055278248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d4104bb0 0x7fb7d4105030 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc009900 tx=0x7fb7bc02a3f0 comp rx=0 tx=0).stop 2024-04-25T01:12:46.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.410+0000 7fb7db400640 1 -- 172.21.15.31:0/1055278248 shutdown_connections 2024-04-25T01:12:46.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.410+0000 7fb7db400640 1 --2- 172.21.15.31:0/1055278248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7d4105570 0x7fb7d410be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:46.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.410+0000 7fb7db400640 1 --2- 172.21.15.31:0/1055278248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d4104bb0 0x7fb7d4105030 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:46.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.410+0000 7fb7db400640 1 --2- 172.21.15.31:0/1055278248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7d41039c0 0x7fb7d4103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:46.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.410+0000 7fb7db400640 1 -- 172.21.15.31:0/1055278248 >> 172.21.15.31:0/1055278248 conn(0x7fb7d40fef30 msgr2=0x7fb7d4101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:46.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.410+0000 7fb7db400640 1 -- 172.21.15.31:0/1055278248 shutdown_connections 2024-04-25T01:12:46.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.410+0000 7fb7db400640 1 -- 172.21.15.31:0/1055278248 wait complete. 2024-04-25T01:12:46.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.412+0000 7fb7db400640 1 Processor -- start 2024-04-25T01:12:46.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.412+0000 7fb7db400640 1 -- start start 2024-04-25T01:12:46.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.412+0000 7fb7db400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7d41039c0 0x7fb7d4199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:46.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.412+0000 7fb7db400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7d4104bb0 0x7fb7d419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:46.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.412+0000 7fb7db400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d4105570 0x7fb7d41a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:46.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.412+0000 7fb7db400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d410f670 con 0x7fb7d4104bb0 2024-04-25T01:12:46.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.412+0000 7fb7db400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d410f4f0 con 0x7fb7d4105570 2024-04-25T01:12:46.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.412+0000 7fb7db400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d410f7f0 con 0x7fb7d41039c0 2024-04-25T01:12:46.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.412+0000 7fb7d8a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7d41039c0 0x7fb7d4199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:46.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.412+0000 7fb7d8a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7d41039c0 0x7fb7d4199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:44614/0 (socket says 172.21.15.31:44614) 2024-04-25T01:12:46.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.412+0000 7fb7d8a00640 1 -- 172.21.15.31:0/3881412557 learned_addr learned my addr 172.21.15.31:0/3881412557 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:12:46.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.412+0000 7fb7cbe00640 1 --2- 172.21.15.31:0/3881412557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7d4104bb0 0x7fb7d419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:46.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.413+0000 7fb7d9400640 1 --2- 172.21.15.31:0/3881412557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d4105570 0x7fb7d41a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:46.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.413+0000 7fb7d8a00640 1 -- 172.21.15.31:0/3881412557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d4105570 msgr2=0x7fb7d41a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:46.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.413+0000 7fb7d8a00640 1 --2- 172.21.15.31:0/3881412557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d4105570 0x7fb7d41a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:46.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.413+0000 7fb7d8a00640 1 -- 172.21.15.31:0/3881412557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7d4104bb0 msgr2=0x7fb7d419a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:46.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.413+0000 7fb7d8a00640 1 --2- 172.21.15.31:0/3881412557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7d4104bb0 0x7fb7d419a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:46.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.413+0000 7fb7d8a00640 1 -- 172.21.15.31:0/3881412557 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d41a1940 con 0x7fb7d41039c0 2024-04-25T01:12:46.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.413+0000 7fb7d9400640 1 --2- 172.21.15.31:0/3881412557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d4105570 0x7fb7d41a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:12:46.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.413+0000 7fb7d8a00640 1 --2- 172.21.15.31:0/3881412557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7d41039c0 0x7fb7d4199d30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb7c400e9c0 tx=0x7fb7c400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:46.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.413+0000 7fb7d2000640 1 -- 172.21.15.31:0/3881412557 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c400cd80 con 0x7fb7d41039c0 2024-04-25T01:12:46.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.413+0000 7fb7d2000640 1 -- 172.21.15.31:0/3881412557 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7c40035f0 con 0x7fb7d41039c0 2024-04-25T01:12:46.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.413+0000 7fb7d2000640 1 -- 172.21.15.31:0/3881412557 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4010690 con 0x7fb7d41039c0 2024-04-25T01:12:46.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.414+0000 7fb7db400640 1 -- 172.21.15.31:0/3881412557 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7d41a1c30 con 0x7fb7d41039c0 2024-04-25T01:12:46.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.414+0000 7fb7db400640 1 -- 172.21.15.31:0/3881412557 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7d41a2130 con 0x7fb7d41039c0 2024-04-25T01:12:46.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.415+0000 7fb7db400640 1 -- 172.21.15.31:0/3881412557 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7d410ace0 con 0x7fb7d41039c0 2024-04-25T01:12:46.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.417+0000 7fb7d2000640 1 -- 172.21.15.31:0/3881412557 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb7c40037a0 con 0x7fb7d41039c0 2024-04-25T01:12:46.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.418+0000 7fb7d2000640 1 --2- 172.21.15.31:0/3881412557 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb7b80a2b90 0x7fb7b80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:46.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.418+0000 7fb7d2000640 1 -- 172.21.15.31:0/3881412557 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fb7c40cf140 con 0x7fb7d41039c0 2024-04-25T01:12:46.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.418+0000 7fb7cbe00640 1 --2- 172.21.15.31:0/3881412557 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb7b80a2b90 0x7fb7b80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:46.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.419+0000 7fb7cbe00640 1 --2- 172.21.15.31:0/3881412557 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb7b80a2b90 0x7fb7b80a5050 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc031440 tx=0x7fb7bc0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:46.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.422+0000 7fb7d2000640 1 -- 172.21.15.31:0/3881412557 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb7c409c4a0 con 0x7fb7d41039c0 2024-04-25T01:12:46.443 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:46 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:46.443 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:46 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:12:46.443 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:46 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:46.443 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:46 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:12:46.443 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:46 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:12:46.443 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:46 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:46.443 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:46 smithi169 ceph-mon[32759]: Deploying daemon mds.l on smithi169 2024-04-25T01:12:46.443 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:46 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/4170949303' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:12:46.443 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:46 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/753194336' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:12:46.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:46 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:46.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:46 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:12:46.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:46 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:46.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:46 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:12:46.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:46 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:12:46.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:46 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:46.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:46 smithi031 ceph-mon[28053]: Deploying daemon mds.l on smithi169 2024-04-25T01:12:46.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:46 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/4170949303' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:12:46.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:46 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/753194336' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:12:46.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.667+0000 7fb7db400640 1 -- 172.21.15.31:0/3881412557 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7d402d170 con 0x7fb7d41039c0 2024-04-25T01:12:46.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.668+0000 7fb7d2000640 1 -- 172.21.15.31:0/3881412557 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fb7c409bda0 con 0x7fb7d41039c0 2024-04-25T01:12:46.655 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:12:46.655 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T01:12:46.655 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 1 2024-04-25T01:12:46.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.671+0000 7fb7db400640 1 -- 172.21.15.31:0/3881412557 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb7b80a2b90 msgr2=0x7fb7b80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:46.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.671+0000 7fb7db400640 1 --2- 172.21.15.31:0/3881412557 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb7b80a2b90 0x7fb7b80a5050 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc031440 tx=0x7fb7bc0315b0 comp rx=0 tx=0).stop 2024-04-25T01:12:46.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.671+0000 7fb7db400640 1 -- 172.21.15.31:0/3881412557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7d41039c0 msgr2=0x7fb7d4199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:46.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.671+0000 7fb7db400640 1 --2- 172.21.15.31:0/3881412557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7d41039c0 0x7fb7d4199d30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb7c400e9c0 tx=0x7fb7c400ee80 comp rx=0 tx=0).stop 2024-04-25T01:12:46.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.671+0000 7fb7db400640 1 -- 172.21.15.31:0/3881412557 shutdown_connections 2024-04-25T01:12:46.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.671+0000 7fb7db400640 1 --2- 172.21.15.31:0/3881412557 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb7b80a2b90 0x7fb7b80a5050 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:46.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.671+0000 7fb7db400640 1 --2- 172.21.15.31:0/3881412557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d4105570 0x7fb7d41a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:46.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.671+0000 7fb7db400640 1 --2- 172.21.15.31:0/3881412557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7d4104bb0 0x7fb7d419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:46.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.671+0000 7fb7db400640 1 --2- 172.21.15.31:0/3881412557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7d41039c0 0x7fb7d4199d30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:46.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.671+0000 7fb7db400640 1 -- 172.21.15.31:0/3881412557 >> 172.21.15.31:0/3881412557 conn(0x7fb7d40fef30 msgr2=0x7fb7d4107eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:46.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.671+0000 7fb7db400640 1 -- 172.21.15.31:0/3881412557 shutdown_connections 2024-04-25T01:12:46.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:46.672+0000 7fb7db400640 1 -- 172.21.15.31:0/3881412557 wait complete. 2024-04-25T01:12:46.699 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:46 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:46.699 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:46 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:12:46.699 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:46 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:46.699 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:46 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:12:46.699 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:46 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:12:46.699 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:46 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:46.699 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:46 smithi040 ceph-mon[32518]: Deploying daemon mds.l on smithi169 2024-04-25T01:12:46.699 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:46 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/4170949303' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:12:46.699 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:46 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/753194336' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:12:47.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:47 smithi169 ceph-mon[32759]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:47.694 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:47 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3881412557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:12:47.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:47 smithi040 ceph-mon[32518]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:47.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:47 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3881412557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:12:48.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:47 smithi031 ceph-mon[28053]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:48.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:47 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3881412557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:12:48.446 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 01:12:48 smithi169 systemd[1]: Started Ceph mds.l for 95849b94-029f-11ef-bc93-c7b262605968. 2024-04-25T01:12:48.678 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:12:48.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.916+0000 7f896cc00640 1 -- 172.21.15.31:0/490211284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89680741a0 msgr2=0x7f8968074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:48.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.916+0000 7f896cc00640 1 --2- 172.21.15.31:0/490211284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89680741a0 0x7f8968074620 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8950009930 tx=0x7f895002a420 comp rx=0 tx=0).stop 2024-04-25T01:12:48.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.917+0000 7f896cc00640 1 -- 172.21.15.31:0/490211284 shutdown_connections 2024-04-25T01:12:48.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.917+0000 7f896cc00640 1 --2- 172.21.15.31:0/490211284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f896810dbb0 0x7f896810ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:48.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.917+0000 7f896cc00640 1 --2- 172.21.15.31:0/490211284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89680741a0 0x7f8968074620 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:48.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.917+0000 7f896cc00640 1 --2- 172.21.15.31:0/490211284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8968075780 0x7f8968073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:48.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.917+0000 7f896cc00640 1 -- 172.21.15.31:0/490211284 >> 172.21.15.31:0/490211284 conn(0x7f89680fcb50 msgr2=0x7f89680fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:48.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.917+0000 7f896cc00640 1 -- 172.21.15.31:0/490211284 shutdown_connections 2024-04-25T01:12:48.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.917+0000 7f896cc00640 1 -- 172.21.15.31:0/490211284 wait complete. 2024-04-25T01:12:48.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.918+0000 7f896cc00640 1 Processor -- start 2024-04-25T01:12:48.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.918+0000 7f896cc00640 1 -- start start 2024-04-25T01:12:48.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.918+0000 7f896cc00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89680741a0 0x7f896819df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:48.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.918+0000 7f896cc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8968075780 0x7f896819e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:48.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.918+0000 7f896cc00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f896810dbb0 0x7f89681a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.918+0000 7f896cc00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f89681138a0 con 0x7f896810dbb0 2024-04-25T01:12:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.918+0000 7f896cc00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8968113720 con 0x7f89680741a0 2024-04-25T01:12:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.918+0000 7f896cc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8968113a20 con 0x7f8968075780 2024-04-25T01:12:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.919+0000 7f8967e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f896810dbb0 0x7f89681a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.919+0000 7f8967400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89680741a0 0x7f896819df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.919+0000 7f8967400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89680741a0 0x7f896819df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:44102/0 (socket says 172.21.15.31:44102) 2024-04-25T01:12:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.919+0000 7f8967400640 1 -- 172.21.15.31:0/837546625 learned_addr learned my addr 172.21.15.31:0/837546625 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:12:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.919+0000 7f8966a00640 1 --2- 172.21.15.31:0/837546625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8968075780 0x7f896819e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.919+0000 7f8967400640 1 -- 172.21.15.31:0/837546625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8968075780 msgr2=0x7f896819e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.919+0000 7f8967400640 1 --2- 172.21.15.31:0/837546625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8968075780 0x7f896819e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.919+0000 7f8967400640 1 -- 172.21.15.31:0/837546625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f896810dbb0 msgr2=0x7f89681a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.919+0000 7f8967400640 1 --2- 172.21.15.31:0/837546625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f896810dbb0 0x7f89681a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:48.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.919+0000 7f8967400640 1 -- 172.21.15.31:0/837546625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89681a5b30 con 0x7f89680741a0 2024-04-25T01:12:48.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.919+0000 7f8967400640 1 --2- 172.21.15.31:0/837546625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89680741a0 0x7f896819df10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f895800e9f0 tx=0x7f895800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:48.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.920+0000 7f895fe00640 1 -- 172.21.15.31:0/837546625 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895800cd80 con 0x7f89680741a0 2024-04-25T01:12:48.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.920+0000 7f895fe00640 1 -- 172.21.15.31:0/837546625 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8958010910 con 0x7f89680741a0 2024-04-25T01:12:48.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.920+0000 7f896cc00640 1 -- 172.21.15.31:0/837546625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89681a5e20 con 0x7f89680741a0 2024-04-25T01:12:48.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.920+0000 7f895fe00640 1 -- 172.21.15.31:0/837546625 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8958018990 con 0x7f89680741a0 2024-04-25T01:12:48.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.920+0000 7f896cc00640 1 -- 172.21.15.31:0/837546625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89681a6280 con 0x7f89680741a0 2024-04-25T01:12:48.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.921+0000 7f896cc00640 1 -- 172.21.15.31:0/837546625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8968103d80 con 0x7f89680741a0 2024-04-25T01:12:48.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.923+0000 7f895fe00640 1 -- 172.21.15.31:0/837546625 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8958010ac0 con 0x7f89680741a0 2024-04-25T01:12:48.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.925+0000 7f895fe00640 1 --2- 172.21.15.31:0/837546625 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f89480a2b90 0x7f89480a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:48.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.925+0000 7f895fe00640 1 -- 172.21.15.31:0/837546625 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f89580cef30 con 0x7f89680741a0 2024-04-25T01:12:48.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.925+0000 7f8966a00640 1 --2- 172.21.15.31:0/837546625 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f89480a2b90 0x7f89480a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:48.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.926+0000 7f8966a00640 1 --2- 172.21.15.31:0/837546625 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f89480a2b90 0x7f89480a5050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8950009900 tx=0x7f89500315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:48.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:48.929+0000 7f895fe00640 1 -- 172.21.15.31:0/837546625 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f895809bc40 con 0x7f89680741a0 2024-04-25T01:12:49.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:49.172+0000 7f896cc00640 1 -- 172.21.15.31:0/837546625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8968062410 con 0x7f89680741a0 2024-04-25T01:12:49.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:49.173+0000 7f895fe00640 1 -- 172.21.15.31:0/837546625 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7f895809ff40 con 0x7f89680741a0 2024-04-25T01:12:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:12:49.160 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/420362823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":420362823},{"type":"v1","addr":"172.21.15.169:6833","nonce":420362823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T01:12:49.160 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 2 2024-04-25T01:12:49.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:49.176+0000 7f896cc00640 1 -- 172.21.15.31:0/837546625 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f89480a2b90 msgr2=0x7f89480a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:49.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:49.176+0000 7f896cc00640 1 --2- 172.21.15.31:0/837546625 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f89480a2b90 0x7f89480a5050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8950009900 tx=0x7f89500315b0 comp rx=0 tx=0).stop 2024-04-25T01:12:49.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:49.176+0000 7f896cc00640 1 -- 172.21.15.31:0/837546625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89680741a0 msgr2=0x7f896819df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:49.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:49.176+0000 7f896cc00640 1 --2- 172.21.15.31:0/837546625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89680741a0 0x7f896819df10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f895800e9f0 tx=0x7f895800eeb0 comp rx=0 tx=0).stop 2024-04-25T01:12:49.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:49.176+0000 7f896cc00640 1 -- 172.21.15.31:0/837546625 shutdown_connections 2024-04-25T01:12:49.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:49.176+0000 7f896cc00640 1 --2- 172.21.15.31:0/837546625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f896810dbb0 0x7f89681a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:49.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:49.176+0000 7f896cc00640 1 --2- 172.21.15.31:0/837546625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8968075780 0x7f896819e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:49.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:49.176+0000 7f896cc00640 1 --2- 172.21.15.31:0/837546625 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f89480a2b90 0x7f89480a5050 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:49.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:49.176+0000 7f896cc00640 1 --2- 172.21.15.31:0/837546625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89680741a0 0x7f896819df10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:49.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:49.177+0000 7f896cc00640 1 -- 172.21.15.31:0/837546625 >> 172.21.15.31:0/837546625 conn(0x7f89680fcb50 msgr2=0x7f8968100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:49.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:49.177+0000 7f896cc00640 1 -- 172.21.15.31:0/837546625 shutdown_connections 2024-04-25T01:12:49.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:49.177+0000 7f896cc00640 1 -- 172.21.15.31:0/837546625 wait complete. 2024-04-25T01:12:49.202 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:49 smithi040 ceph-mon[32518]: mds.? [v2:172.21.15.169:6832/420362823,v1:172.21.15.169:6833/420362823] up:boot 2024-04-25T01:12:49.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:49 smithi040 ceph-mon[32518]: fsmap 1 up:standby 2024-04-25T01:12:49.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:49 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T01:12:49.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:49 smithi040 ceph-mon[32518]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:49.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:49 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:49.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:49 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:49.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:49 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:49.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:49 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:12:49.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:49 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:12:49.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:49 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:49.203 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:49 smithi040 ceph-mon[32518]: Deploying daemon mds.k on smithi040 2024-04-25T01:12:49.446 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:49 smithi169 ceph-mon[32759]: mds.? [v2:172.21.15.169:6832/420362823,v1:172.21.15.169:6833/420362823] up:boot 2024-04-25T01:12:49.446 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:49 smithi169 ceph-mon[32759]: fsmap 1 up:standby 2024-04-25T01:12:49.446 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:49 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T01:12:49.446 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:49 smithi169 ceph-mon[32759]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:49.446 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:49 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:49.447 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:49 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:49.447 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:49 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:49.447 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:49 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:12:49.447 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:49 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:12:49.447 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:49 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:49.447 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:49 smithi169 ceph-mon[32759]: Deploying daemon mds.k on smithi040 2024-04-25T01:12:49.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:49 smithi031 ceph-mon[28053]: mds.? [v2:172.21.15.169:6832/420362823,v1:172.21.15.169:6833/420362823] up:boot 2024-04-25T01:12:49.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:49 smithi031 ceph-mon[28053]: fsmap 1 up:standby 2024-04-25T01:12:49.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:49 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T01:12:49.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:49 smithi031 ceph-mon[28053]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:49.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:49 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:49.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:49 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:49.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:49 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:49.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:49 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:12:49.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:49 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:12:49.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:49 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:49.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:49 smithi031 ceph-mon[28053]: Deploying daemon mds.k on smithi040 2024-04-25T01:12:50.356 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:50 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/837546625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:12:50.445 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:50 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/837546625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:12:50.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:50 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/837546625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:12:50.699 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 01:12:50 smithi040 systemd[1]: Starting Ceph mds.k for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:12:51.026 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 01:12:50 smithi040 podman[62271]: 2024-04-25 01:12:50.687805795 +0000 UTC m=+0.018177363 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:12:51.027 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 01:12:50 smithi040 podman[62271]: 2024-04-25 01:12:50.903864606 +0000 UTC m=+0.234236172 container create b6248292272ea2abf2921c4efede399990005aef985fbc328e1be1309488a7ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-k, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:12:51.183 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:12:51.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.422+0000 7f752c000640 1 -- 172.21.15.31:0/1781314028 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7524077340 msgr2=0x7f75240777a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:51.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.422+0000 7f752c000640 1 --2- 172.21.15.31:0/1781314028 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7524077340 0x7f75240777a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7510009860 tx=0x7f751002a340 comp rx=0 tx=0).stop 2024-04-25T01:12:51.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.422+0000 7f752c000640 1 -- 172.21.15.31:0/1781314028 shutdown_connections 2024-04-25T01:12:51.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.422+0000 7f752c000640 1 --2- 172.21.15.31:0/1781314028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7524077ce0 0x7f7524078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:51.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.422+0000 7f752c000640 1 --2- 172.21.15.31:0/1781314028 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7524077340 0x7f75240777a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:51.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.422+0000 7f752c000640 1 --2- 172.21.15.31:0/1781314028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7524076100 0x7f7524076520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:51.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.422+0000 7f752c000640 1 -- 172.21.15.31:0/1781314028 >> 172.21.15.31:0/1781314028 conn(0x7f75240fef30 msgr2=0x7f7524101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:51.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.422+0000 7f752c000640 1 -- 172.21.15.31:0/1781314028 shutdown_connections 2024-04-25T01:12:51.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.422+0000 7f752c000640 1 -- 172.21.15.31:0/1781314028 wait complete. 2024-04-25T01:12:51.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.423+0000 7f752c000640 1 Processor -- start 2024-04-25T01:12:51.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.423+0000 7f752c000640 1 -- start start 2024-04-25T01:12:51.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.424+0000 7f752c000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7524076100 0x7f752419e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:51.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.424+0000 7f752c000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7524077340 0x7f752419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:51.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.424+0000 7f752c000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7524077ce0 0x7f75241a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:51.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.424+0000 7f752c000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7524113a30 con 0x7f7524077340 2024-04-25T01:12:51.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.424+0000 7f752c000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f75241138b0 con 0x7f7524077ce0 2024-04-25T01:12:51.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.424+0000 7f752c000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7524113bb0 con 0x7f7524076100 2024-04-25T01:12:51.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.424+0000 7f7528c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7524077340 0x7f752419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:51.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.424+0000 7f7528c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7524077340 0x7f752419e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49746/0 (socket says 172.21.15.31:49746) 2024-04-25T01:12:51.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.424+0000 7f752a000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7524077ce0 0x7f75241a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:51.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.424+0000 7f7528c00640 1 -- 172.21.15.31:0/3700957448 learned_addr learned my addr 172.21.15.31:0/3700957448 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:12:51.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.424+0000 7f7529600640 1 --2- 172.21.15.31:0/3700957448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7524076100 0x7f752419e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:51.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.424+0000 7f7528c00640 1 -- 172.21.15.31:0/3700957448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7524076100 msgr2=0x7f752419e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:51.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.424+0000 7f7528c00640 1 --2- 172.21.15.31:0/3700957448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7524076100 0x7f752419e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:51.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.424+0000 7f7528c00640 1 -- 172.21.15.31:0/3700957448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7524077ce0 msgr2=0x7f75241a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:51.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.424+0000 7f7528c00640 1 --2- 172.21.15.31:0/3700957448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7524077ce0 0x7f75241a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:51.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.424+0000 7f7528c00640 1 -- 172.21.15.31:0/3700957448 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75241a5d10 con 0x7f7524077340 2024-04-25T01:12:51.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.424+0000 7f752a000640 1 --2- 172.21.15.31:0/3700957448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7524077ce0 0x7f75241a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:12:51.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.424+0000 7f7529600640 1 --2- 172.21.15.31:0/3700957448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7524076100 0x7f752419e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:12:51.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.425+0000 7f7528c00640 1 --2- 172.21.15.31:0/3700957448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7524077340 0x7f752419e630 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7510035040 tx=0x7f7510033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:51.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.425+0000 7f7522000640 1 -- 172.21.15.31:0/3700957448 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510037040 con 0x7f7524077340 2024-04-25T01:12:51.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.425+0000 7f752c000640 1 -- 172.21.15.31:0/3700957448 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75241a5fa0 con 0x7f7524077340 2024-04-25T01:12:51.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.425+0000 7f7522000640 1 -- 172.21.15.31:0/3700957448 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7510030850 con 0x7f7524077340 2024-04-25T01:12:51.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.425+0000 7f7522000640 1 -- 172.21.15.31:0/3700957448 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510004920 con 0x7f7524077340 2024-04-25T01:12:51.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.425+0000 7f752c000640 1 -- 172.21.15.31:0/3700957448 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75241a6410 con 0x7f7524077340 2024-04-25T01:12:51.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.427+0000 7f752c000640 1 -- 172.21.15.31:0/3700957448 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7524110540 con 0x7f7524077340 2024-04-25T01:12:51.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.429+0000 7f7522000640 1 -- 172.21.15.31:0/3700957448 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7510004050 con 0x7f7524077340 2024-04-25T01:12:51.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.430+0000 7f7522000640 1 --2- 172.21.15.31:0/3700957448 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f750c0a2c10 0x7f750c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:51.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.430+0000 7f7522000640 1 -- 172.21.15.31:0/3700957448 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f75100eeb50 con 0x7f7524077340 2024-04-25T01:12:51.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.431+0000 7f7529600640 1 --2- 172.21.15.31:0/3700957448 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f750c0a2c10 0x7f750c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:51.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.431+0000 7f7529600640 1 --2- 172.21.15.31:0/3700957448 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f750c0a2c10 0x7f750c0a50d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7518009c20 tx=0x7f7518005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:51.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.435+0000 7f7522000640 1 -- 172.21.15.31:0/3700957448 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f75100bbeb0 con 0x7f7524077340 2024-04-25T01:12:51.445 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:51 smithi169 ceph-mon[32759]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:51.448 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:51 smithi040 ceph-mon[32518]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:51.449 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 01:12:51 smithi040 podman[62271]: 2024-04-25 01:12:51.139791061 +0000 UTC m=+0.470162615 container init b6248292272ea2abf2921c4efede399990005aef985fbc328e1be1309488a7ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-k, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:12:51.449 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 01:12:51 smithi040 podman[62271]: 2024-04-25 01:12:51.142872021 +0000 UTC m=+0.473243583 container start b6248292272ea2abf2921c4efede399990005aef985fbc328e1be1309488a7ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T01:12:51.449 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 01:12:51 smithi040 ceph-mds[62290]: -- 172.21.15.40:0/2428245468 <== mon.2 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b32b25da40 con 0x55b32b230800 2024-04-25T01:12:51.449 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 01:12:51 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-k[62286]: starting mds.k at 2024-04-25T01:12:51.449 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 01:12:51 smithi040 bash[62271]: b6248292272ea2abf2921c4efede399990005aef985fbc328e1be1309488a7ab 2024-04-25T01:12:51.449 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 01:12:51 smithi040 systemd[1]: Started Ceph mds.k for 95849b94-029f-11ef-bc93-c7b262605968. 2024-04-25T01:12:51.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:51 smithi031 ceph-mon[28053]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:51.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.686+0000 7f752c000640 1 -- 172.21.15.31:0/3700957448 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75240623e0 con 0x7f7524077340 2024-04-25T01:12:51.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.687+0000 7f7522000640 1 -- 172.21.15.31:0/3700957448 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7f75100bb7b0 con 0x7f7524077340 2024-04-25T01:12:51.675 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:12:51.675 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/420362823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":420362823},{"type":"v1","addr":"172.21.15.169:6833","nonce":420362823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T01:12:51.675 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 2 2024-04-25T01:12:51.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.690+0000 7f752c000640 1 -- 172.21.15.31:0/3700957448 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f750c0a2c10 msgr2=0x7f750c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:51.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.690+0000 7f752c000640 1 --2- 172.21.15.31:0/3700957448 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f750c0a2c10 0x7f750c0a50d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7518009c20 tx=0x7f7518005f90 comp rx=0 tx=0).stop 2024-04-25T01:12:51.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.690+0000 7f752c000640 1 -- 172.21.15.31:0/3700957448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7524077340 msgr2=0x7f752419e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:51.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.690+0000 7f752c000640 1 --2- 172.21.15.31:0/3700957448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7524077340 0x7f752419e630 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7510035040 tx=0x7f7510033d90 comp rx=0 tx=0).stop 2024-04-25T01:12:51.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.690+0000 7f752c000640 1 -- 172.21.15.31:0/3700957448 shutdown_connections 2024-04-25T01:12:51.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.690+0000 7f752c000640 1 --2- 172.21.15.31:0/3700957448 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f750c0a2c10 0x7f750c0a50d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:51.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.690+0000 7f752c000640 1 --2- 172.21.15.31:0/3700957448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7524077ce0 0x7f75241a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:51.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.690+0000 7f752c000640 1 --2- 172.21.15.31:0/3700957448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7524077340 0x7f752419e630 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:51.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.690+0000 7f752c000640 1 --2- 172.21.15.31:0/3700957448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7524076100 0x7f752419e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:51.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.690+0000 7f752c000640 1 -- 172.21.15.31:0/3700957448 >> 172.21.15.31:0/3700957448 conn(0x7f75240fef30 msgr2=0x7f75241005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:51.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.690+0000 7f752c000640 1 -- 172.21.15.31:0/3700957448 shutdown_connections 2024-04-25T01:12:51.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:51.691+0000 7f752c000640 1 -- 172.21.15.31:0/3700957448 wait complete. 2024-04-25T01:12:52.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:52 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:52.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:52 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:52.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:52 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:52.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:52 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:12:52.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:52 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:12:52.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:52 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:52.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:52 smithi031 ceph-mon[28053]: Deploying daemon mds.j on smithi031 2024-04-25T01:12:52.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:52 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3700957448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:12:52.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:52 smithi031 ceph-mon[28053]: mds.? [v2:172.21.15.40:6832/1943710438,v1:172.21.15.40:6833/1943710438] up:boot 2024-04-25T01:12:52.519 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:52 smithi031 ceph-mon[28053]: fsmap 2 up:standby 2024-04-25T01:12:52.519 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:52 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T01:12:52.547 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:52 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:52.547 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:52 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:52.547 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:52 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:52.547 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:52 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:12:52.547 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:52 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:12:52.547 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:52 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:52.547 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:52 smithi040 ceph-mon[32518]: Deploying daemon mds.j on smithi031 2024-04-25T01:12:52.547 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:52 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3700957448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:12:52.547 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:52 smithi040 ceph-mon[32518]: mds.? [v2:172.21.15.40:6832/1943710438,v1:172.21.15.40:6833/1943710438] up:boot 2024-04-25T01:12:52.547 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:52 smithi040 ceph-mon[32518]: fsmap 2 up:standby 2024-04-25T01:12:52.548 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:52 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T01:12:52.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:52 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:52.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:52 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:52.697 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:52 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:52.697 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:52 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:12:52.697 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:52 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:12:52.697 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:52 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:52.697 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:52 smithi169 ceph-mon[32759]: Deploying daemon mds.j on smithi031 2024-04-25T01:12:52.697 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:52 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3700957448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:12:52.697 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:52 smithi169 ceph-mon[32759]: mds.? [v2:172.21.15.40:6832/1943710438,v1:172.21.15.40:6833/1943710438] up:boot 2024-04-25T01:12:52.697 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:52 smithi169 ceph-mon[32759]: fsmap 2 up:standby 2024-04-25T01:12:52.697 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:52 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T01:12:53.462 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:53 smithi031 ceph-mon[28053]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:53.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:53 smithi169 ceph-mon[32759]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:53.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:53 smithi040 ceph-mon[32518]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:53.700 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:12:54.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.103+0000 7f7d67a00640 1 -- 172.21.15.31:0/2477074173 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d601089b0 msgr2=0x7f7d6010ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:54.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.103+0000 7f7d67a00640 1 --2- 172.21.15.31:0/2477074173 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d601089b0 0x7f7d6010ada0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7d480074e0 tx=0x7f7d48029f80 comp rx=0 tx=0).stop 2024-04-25T01:12:54.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.103+0000 7f7d67a00640 1 -- 172.21.15.31:0/2477074173 shutdown_connections 2024-04-25T01:12:54.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.103+0000 7f7d67a00640 1 --2- 172.21.15.31:0/2477074173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d6010de50 0x7f7d60110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:54.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.103+0000 7f7d67a00640 1 --2- 172.21.15.31:0/2477074173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6010b370 0x7f7d6010d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.103+0000 7f7d67a00640 1 --2- 172.21.15.31:0/2477074173 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d601089b0 0x7f7d6010ada0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.103+0000 7f7d67a00640 1 -- 172.21.15.31:0/2477074173 >> 172.21.15.31:0/2477074173 conn(0x7f7d6006e5e0 msgr2=0x7f7d60070a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.103+0000 7f7d67a00640 1 -- 172.21.15.31:0/2477074173 shutdown_connections 2024-04-25T01:12:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.104+0000 7f7d67a00640 1 -- 172.21.15.31:0/2477074173 wait complete. 2024-04-25T01:12:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.105+0000 7f7d67a00640 1 Processor -- start 2024-04-25T01:12:54.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.106+0000 7f7d67a00640 1 -- start start 2024-04-25T01:12:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.106+0000 7f7d67a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d601089b0 0x7f7d601a2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.106+0000 7f7d67a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6010b370 0x7f7d601a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.106+0000 7f7d67a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d6010de50 0x7f7d601a9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.106+0000 7f7d67a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7d60113a80 con 0x7f7d601089b0 2024-04-25T01:12:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.106+0000 7f7d67a00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7d60113900 con 0x7f7d6010de50 2024-04-25T01:12:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.106+0000 7f7d67a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7d60113c00 con 0x7f7d6010b370 2024-04-25T01:12:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.106+0000 7f7d65000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d601089b0 0x7f7d601a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.106+0000 7f7d5fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6010b370 0x7f7d601a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.106+0000 7f7d65000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d601089b0 0x7f7d601a2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49764/0 (socket says 172.21.15.31:49764) 2024-04-25T01:12:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.106+0000 7f7d65000640 1 -- 172.21.15.31:0/170617216 learned_addr learned my addr 172.21.15.31:0/170617216 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:12:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.107+0000 7f7d65a00640 1 --2- 172.21.15.31:0/170617216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d6010de50 0x7f7d601a9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.107+0000 7f7d65000640 1 -- 172.21.15.31:0/170617216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6010b370 msgr2=0x7f7d601a2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.107+0000 7f7d65000640 1 --2- 172.21.15.31:0/170617216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6010b370 0x7f7d601a2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.107+0000 7f7d65000640 1 -- 172.21.15.31:0/170617216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d6010de50 msgr2=0x7f7d601a9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.107+0000 7f7d65000640 1 --2- 172.21.15.31:0/170617216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d6010de50 0x7f7d601a9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.107+0000 7f7d65000640 1 -- 172.21.15.31:0/170617216 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d601aa130 con 0x7f7d601089b0 2024-04-25T01:12:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.107+0000 7f7d5fe00640 1 --2- 172.21.15.31:0/170617216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6010b370 0x7f7d601a2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:12:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.107+0000 7f7d65000640 1 --2- 172.21.15.31:0/170617216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d601089b0 0x7f7d601a2510 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7d48004880 tx=0x7f7d4800d020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:54.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.108+0000 7f7d5d600640 1 -- 172.21.15.31:0/170617216 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4802abd0 con 0x7f7d601089b0 2024-04-25T01:12:54.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.108+0000 7f7d5d600640 1 -- 172.21.15.31:0/170617216 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d48040070 con 0x7f7d601089b0 2024-04-25T01:12:54.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.108+0000 7f7d67a00640 1 -- 172.21.15.31:0/170617216 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d601aa3c0 con 0x7f7d601089b0 2024-04-25T01:12:54.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.108+0000 7f7d5d600640 1 -- 172.21.15.31:0/170617216 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4803b8a0 con 0x7f7d601089b0 2024-04-25T01:12:54.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.108+0000 7f7d67a00640 1 -- 172.21.15.31:0/170617216 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d601aa8b0 con 0x7f7d601089b0 2024-04-25T01:12:54.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.109+0000 7f7d67a00640 1 -- 172.21.15.31:0/170617216 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d600710e0 con 0x7f7d601089b0 2024-04-25T01:12:54.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.111+0000 7f7d5d600640 1 -- 172.21.15.31:0/170617216 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7d4800a640 con 0x7f7d601089b0 2024-04-25T01:12:54.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.113+0000 7f7d5d600640 1 --2- 172.21.15.31:0/170617216 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7d440a2b90 0x7f7d440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:54.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.114+0000 7f7d5d600640 1 -- 172.21.15.31:0/170617216 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f7d480eed40 con 0x7f7d601089b0 2024-04-25T01:12:54.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.114+0000 7f7d5fe00640 1 --2- 172.21.15.31:0/170617216 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7d440a2b90 0x7f7d440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:54.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.114+0000 7f7d5fe00640 1 --2- 172.21.15.31:0/170617216 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7d440a2b90 0x7f7d440a5050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7d601a3a20 tx=0x7f7d500075b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:54.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.117+0000 7f7d5d600640 1 -- 172.21.15.31:0/170617216 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7d480bc0a0 con 0x7f7d601089b0 2024-04-25T01:12:54.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.370+0000 7f7d67a00640 1 -- 172.21.15.31:0/170617216 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d6002d170 con 0x7f7d601089b0 2024-04-25T01:12:54.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.371+0000 7f7d5d600640 1 -- 172.21.15.31:0/170617216 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7f7d480bb9a0 con 0x7f7d601089b0 2024-04-25T01:12:54.358 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:12:54.359 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/420362823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":420362823},{"type":"v1","addr":"172.21.15.169:6833","nonce":420362823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1943710438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1943710438},{"type":"v1","addr":"172.21.15.40:6833","nonce":1943710438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T01:12:54.359 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 3 2024-04-25T01:12:54.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.374+0000 7f7d67a00640 1 -- 172.21.15.31:0/170617216 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7d440a2b90 msgr2=0x7f7d440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:54.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.374+0000 7f7d67a00640 1 --2- 172.21.15.31:0/170617216 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7d440a2b90 0x7f7d440a5050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7d601a3a20 tx=0x7f7d500075b0 comp rx=0 tx=0).stop 2024-04-25T01:12:54.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.374+0000 7f7d67a00640 1 -- 172.21.15.31:0/170617216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d601089b0 msgr2=0x7f7d601a2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:54.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.374+0000 7f7d67a00640 1 --2- 172.21.15.31:0/170617216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d601089b0 0x7f7d601a2510 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7d48004880 tx=0x7f7d4800d020 comp rx=0 tx=0).stop 2024-04-25T01:12:54.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.374+0000 7f7d67a00640 1 -- 172.21.15.31:0/170617216 shutdown_connections 2024-04-25T01:12:54.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.374+0000 7f7d67a00640 1 --2- 172.21.15.31:0/170617216 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7d440a2b90 0x7f7d440a5050 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:54.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.374+0000 7f7d67a00640 1 --2- 172.21.15.31:0/170617216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d6010de50 0x7f7d601a9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:54.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.374+0000 7f7d67a00640 1 --2- 172.21.15.31:0/170617216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6010b370 0x7f7d601a2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:54.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.374+0000 7f7d67a00640 1 --2- 172.21.15.31:0/170617216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d601089b0 0x7f7d601a2510 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:54.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.374+0000 7f7d67a00640 1 -- 172.21.15.31:0/170617216 >> 172.21.15.31:0/170617216 conn(0x7f7d6006e5e0 msgr2=0x7f7d60108d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:54.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.374+0000 7f7d67a00640 1 -- 172.21.15.31:0/170617216 shutdown_connections 2024-04-25T01:12:54.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:54.374+0000 7f7d67a00640 1 -- 172.21.15.31:0/170617216 wait complete. 2024-04-25T01:12:55.007 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 01:12:54 smithi031 podman[68342]: 2024-04-25 01:12:54.761169349 +0000 UTC m=+0.197666786 container create 8ed92e04f5832057f65651f2e67214486e99ab5608eae12727beb0ebc2e13460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-j, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-25T01:12:55.322 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 01:12:55 smithi031 podman[68342]: 2024-04-25 01:12:55.048510166 +0000 UTC m=+0.485007646 container init 8ed92e04f5832057f65651f2e67214486e99ab5608eae12727beb0ebc2e13460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-j, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424) 2024-04-25T01:12:55.322 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 01:12:55 smithi031 podman[68342]: 2024-04-25 01:12:55.05860098 +0000 UTC m=+0.495098409 container start 8ed92e04f5832057f65651f2e67214486e99ab5608eae12727beb0ebc2e13460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9) 2024-04-25T01:12:55.322 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 01:12:55 smithi031 ceph-mds[68360]: -- 172.21.15.31:0/2287143503 <== mon.2 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5606e8fb5a40 con 0x5606e8f88c00 2024-04-25T01:12:55.322 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 01:12:55 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-j[68356]: starting mds.j at 2024-04-25T01:12:55.322 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 01:12:55 smithi031 bash[68342]: 8ed92e04f5832057f65651f2e67214486e99ab5608eae12727beb0ebc2e13460 2024-04-25T01:12:55.322 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 01:12:55 smithi031 systemd[1]: Started Ceph mds.j for 95849b94-029f-11ef-bc93-c7b262605968. 2024-04-25T01:12:55.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:55 smithi169 ceph-mon[32759]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:55.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:55 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/170617216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:12:55.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:55 smithi040 ceph-mon[32518]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:55.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:55 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/170617216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:12:55.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:55 smithi031 ceph-mon[28053]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:55.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:55 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/170617216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:12:56.381 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:12:56.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.607+0000 7f3491a00640 1 -- 172.21.15.31:0/168877105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c1027a0 msgr2=0x7f348c104c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:56.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.607+0000 7f3491a00640 1 --2- 172.21.15.31:0/168877105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c1027a0 0x7f348c104c10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3478009900 tx=0x7f347802a3f0 comp rx=0 tx=0).stop 2024-04-25T01:12:56.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.607+0000 7f3491a00640 1 -- 172.21.15.31:0/168877105 shutdown_connections 2024-04-25T01:12:56.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.607+0000 7f3491a00640 1 --2- 172.21.15.31:0/168877105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f348c105150 0x7f348c1055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:56.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.607+0000 7f3491a00640 1 --2- 172.21.15.31:0/168877105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c1027a0 0x7f348c104c10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:56.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.607+0000 7f3491a00640 1 --2- 172.21.15.31:0/168877105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348c0ffe30 0x7f348c102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:56.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.607+0000 7f3491a00640 1 -- 172.21.15.31:0/168877105 >> 172.21.15.31:0/168877105 conn(0x7f348c0fba10 msgr2=0x7f348c0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:56.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.607+0000 7f3491a00640 1 -- 172.21.15.31:0/168877105 shutdown_connections 2024-04-25T01:12:56.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.607+0000 7f3491a00640 1 -- 172.21.15.31:0/168877105 wait complete. 2024-04-25T01:12:56.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.608+0000 7f3491a00640 1 Processor -- start 2024-04-25T01:12:56.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.608+0000 7f3491a00640 1 -- start start 2024-04-25T01:12:56.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.608+0000 7f3491a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348c0ffe30 0x7f348c19de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.608+0000 7f3491a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f348c1027a0 0x7f348c19e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.608+0000 7f3491a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c105150 0x7f348c1a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.608+0000 7f3491a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f348c113860 con 0x7f348c105150 2024-04-25T01:12:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.608+0000 7f3491a00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f348c1136e0 con 0x7f348c0ffe30 2024-04-25T01:12:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.608+0000 7f3491a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f348c1139e0 con 0x7f348c1027a0 2024-04-25T01:12:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.609+0000 7f348be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348c0ffe30 0x7f348c19de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.609+0000 7f3491000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c105150 0x7f348c1a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.609+0000 7f348be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348c0ffe30 0x7f348c19de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:44192/0 (socket says 172.21.15.31:44192) 2024-04-25T01:12:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.609+0000 7f348be00640 1 -- 172.21.15.31:0/3295188153 learned_addr learned my addr 172.21.15.31:0/3295188153 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:12:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.609+0000 7f348b400640 1 --2- 172.21.15.31:0/3295188153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f348c1027a0 0x7f348c19e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.609+0000 7f348be00640 1 -- 172.21.15.31:0/3295188153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f348c1027a0 msgr2=0x7f348c19e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.609+0000 7f348be00640 1 --2- 172.21.15.31:0/3295188153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f348c1027a0 0x7f348c19e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.609+0000 7f348be00640 1 -- 172.21.15.31:0/3295188153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c105150 msgr2=0x7f348c1a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:56.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.609+0000 7f348be00640 1 --2- 172.21.15.31:0/3295188153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c105150 0x7f348c1a5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:56.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.609+0000 7f348be00640 1 -- 172.21.15.31:0/3295188153 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f348c1a5a70 con 0x7f348c0ffe30 2024-04-25T01:12:56.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.609+0000 7f3491000640 1 --2- 172.21.15.31:0/3295188153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c105150 0x7f348c1a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:12:56.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.610+0000 7f348be00640 1 --2- 172.21.15.31:0/3295188153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348c0ffe30 0x7f348c19de50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f347c00e9f0 tx=0x7f347c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:56.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.610+0000 7f3488c00640 1 -- 172.21.15.31:0/3295188153 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c00ccc0 con 0x7f348c0ffe30 2024-04-25T01:12:56.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.610+0000 7f3488c00640 1 -- 172.21.15.31:0/3295188153 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f347c010910 con 0x7f348c0ffe30 2024-04-25T01:12:56.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.610+0000 7f3488c00640 1 -- 172.21.15.31:0/3295188153 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c0188a0 con 0x7f348c0ffe30 2024-04-25T01:12:56.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.610+0000 7f3491a00640 1 -- 172.21.15.31:0/3295188153 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f348c1a5d60 con 0x7f348c0ffe30 2024-04-25T01:12:56.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.610+0000 7f3491a00640 1 -- 172.21.15.31:0/3295188153 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f348c1a61d0 con 0x7f348c0ffe30 2024-04-25T01:12:56.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.611+0000 7f3491a00640 1 -- 172.21.15.31:0/3295188153 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f348c0fd500 con 0x7f348c0ffe30 2024-04-25T01:12:56.599 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:56 smithi040 ceph-mon[32518]: mds.? [v2:172.21.15.31:6834/498218714,v1:172.21.15.31:6835/498218714] up:boot 2024-04-25T01:12:56.599 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:56 smithi040 ceph-mon[32518]: fsmap 3 up:standby 2024-04-25T01:12:56.599 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:56 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T01:12:56.599 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:56 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:56.599 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:56 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:56.599 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:56 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:56.599 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:56 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:12:56.600 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:56 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:12:56.600 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:56 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:56.600 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:56 smithi040 ceph-mon[32518]: Deploying daemon mds.d on smithi031 2024-04-25T01:12:56.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.613+0000 7f3488c00640 1 -- 172.21.15.31:0/3295188153 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f347c010ac0 con 0x7f348c0ffe30 2024-04-25T01:12:56.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.614+0000 7f3488c00640 1 --2- 172.21.15.31:0/3295188153 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f34700a2c10 0x7f34700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:56.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.615+0000 7f3488c00640 1 -- 172.21.15.31:0/3295188153 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f347c0cefd0 con 0x7f348c0ffe30 2024-04-25T01:12:56.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.615+0000 7f348b400640 1 --2- 172.21.15.31:0/3295188153 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f34700a2c10 0x7f34700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:56.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.615+0000 7f348b400640 1 --2- 172.21.15.31:0/3295188153 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f34700a2c10 0x7f34700a50d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3478031440 tx=0x7f34780315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:56.604 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:56 smithi031 ceph-mon[28053]: mds.? [v2:172.21.15.31:6834/498218714,v1:172.21.15.31:6835/498218714] up:boot 2024-04-25T01:12:56.605 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:56 smithi031 ceph-mon[28053]: fsmap 3 up:standby 2024-04-25T01:12:56.605 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:56 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T01:12:56.605 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:56 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:56.605 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:56 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:56.605 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:56 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:56.605 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:56 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:12:56.605 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:56 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:12:56.605 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:56 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:56.605 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:56 smithi031 ceph-mon[28053]: Deploying daemon mds.d on smithi031 2024-04-25T01:12:56.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.619+0000 7f3488c00640 1 -- 172.21.15.31:0/3295188153 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f347c09c330 con 0x7f348c0ffe30 2024-04-25T01:12:56.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:56 smithi169 ceph-mon[32759]: mds.? [v2:172.21.15.31:6834/498218714,v1:172.21.15.31:6835/498218714] up:boot 2024-04-25T01:12:56.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:56 smithi169 ceph-mon[32759]: fsmap 3 up:standby 2024-04-25T01:12:56.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:56 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T01:12:56.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:56 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:56.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:56 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:56.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:56 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:56.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:56 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:12:56.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:56 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:12:56.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:56 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:56.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:56 smithi169 ceph-mon[32759]: Deploying daemon mds.d on smithi031 2024-04-25T01:12:56.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.876+0000 7f3491a00640 1 -- 172.21.15.31:0/3295188153 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f348c02d170 con 0x7f348c0ffe30 2024-04-25T01:12:56.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.877+0000 7f3488c00640 1 -- 172.21.15.31:0/3295188153 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3040 (secure 0 0 0) 0x7f348c02d170 con 0x7f348c0ffe30 2024-04-25T01:12:56.863 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:12:56.864 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/498218714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":498218714},{"type":"v1","addr":"172.21.15.31:6835","nonce":498218714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/420362823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":420362823},{"type":"v1","addr":"172.21.15.169:6833","nonce":420362823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1943710438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1943710438},{"type":"v1","addr":"172.21.15.40:6833","nonce":1943710438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T01:12:56.864 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 4 2024-04-25T01:12:56.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.880+0000 7f3491a00640 1 -- 172.21.15.31:0/3295188153 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f34700a2c10 msgr2=0x7f34700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:56.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.880+0000 7f3491a00640 1 --2- 172.21.15.31:0/3295188153 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f34700a2c10 0x7f34700a50d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3478031440 tx=0x7f34780315b0 comp rx=0 tx=0).stop 2024-04-25T01:12:56.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.880+0000 7f3491a00640 1 -- 172.21.15.31:0/3295188153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348c0ffe30 msgr2=0x7f348c19de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:56.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.880+0000 7f3491a00640 1 --2- 172.21.15.31:0/3295188153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348c0ffe30 0x7f348c19de50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f347c00e9f0 tx=0x7f347c00edf0 comp rx=0 tx=0).stop 2024-04-25T01:12:56.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.880+0000 7f3491a00640 1 -- 172.21.15.31:0/3295188153 shutdown_connections 2024-04-25T01:12:56.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.880+0000 7f3491a00640 1 --2- 172.21.15.31:0/3295188153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c105150 0x7f348c1a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:56.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.880+0000 7f3491a00640 1 --2- 172.21.15.31:0/3295188153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f348c1027a0 0x7f348c19e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:56.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.880+0000 7f3491a00640 1 --2- 172.21.15.31:0/3295188153 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f34700a2c10 0x7f34700a50d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:56.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.880+0000 7f3491a00640 1 --2- 172.21.15.31:0/3295188153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348c0ffe30 0x7f348c19de50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:56.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.880+0000 7f3491a00640 1 -- 172.21.15.31:0/3295188153 >> 172.21.15.31:0/3295188153 conn(0x7f348c0fba10 msgr2=0x7f348c100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:56.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.880+0000 7f3491a00640 1 -- 172.21.15.31:0/3295188153 shutdown_connections 2024-04-25T01:12:56.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:56.880+0000 7f3491a00640 1 -- 172.21.15.31:0/3295188153 wait complete. 2024-04-25T01:12:57.592 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:57 smithi031 ceph-mon[28053]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:57.592 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:57 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3295188153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:12:57.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:57 smithi169 ceph-mon[32759]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:57.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:57 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3295188153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:12:57.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:57 smithi040 ceph-mon[32518]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:57.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:57 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3295188153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:12:58.757 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 01:12:58 smithi031 systemd[1]: Starting Ceph mds.d for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:12:58.889 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:12:59.099 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 01:12:58 smithi031 podman[68691]: 2024-04-25 01:12:58.727394574 +0000 UTC m=+0.036532697 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:12:59.099 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 01:12:58 smithi031 podman[68691]: 2024-04-25 01:12:58.840801827 +0000 UTC m=+0.149939987 container create 6d1e33ce69ac2e1912d54b8469f71bb32535411a9a88aac04ccfe450ab331fd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-d, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9) 2024-04-25T01:12:59.099 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 01:12:59 smithi031 podman[68691]: 2024-04-25 01:12:59.112265527 +0000 UTC m=+0.421403694 container init 6d1e33ce69ac2e1912d54b8469f71bb32535411a9a88aac04ccfe450ab331fd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-04-25T01:12:59.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.124+0000 7f0be3200640 1 -- 172.21.15.31:0/505293260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bdc1049b0 msgr2=0x7f0bdc104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:59.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.124+0000 7f0be3200640 1 --2- 172.21.15.31:0/505293260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bdc1049b0 0x7f0bdc104e30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009900 tx=0x7f0bcc02a3f0 comp rx=0 tx=0).stop 2024-04-25T01:12:59.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.125+0000 7f0be3200640 1 -- 172.21.15.31:0/505293260 shutdown_connections 2024-04-25T01:12:59.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.125+0000 7f0be3200640 1 --2- 172.21.15.31:0/505293260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bdc105370 0x7f0bdc10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:59.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.125+0000 7f0be3200640 1 --2- 172.21.15.31:0/505293260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bdc1049b0 0x7f0bdc104e30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:59.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.125+0000 7f0be3200640 1 --2- 172.21.15.31:0/505293260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bdc1037c0 0x7f0bdc103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:59.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.125+0000 7f0be3200640 1 -- 172.21.15.31:0/505293260 >> 172.21.15.31:0/505293260 conn(0x7f0bdc0fed50 msgr2=0x7f0bdc101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:59.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.125+0000 7f0be3200640 1 -- 172.21.15.31:0/505293260 shutdown_connections 2024-04-25T01:12:59.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.125+0000 7f0be3200640 1 -- 172.21.15.31:0/505293260 wait complete. 2024-04-25T01:12:59.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.126+0000 7f0be3200640 1 Processor -- start 2024-04-25T01:12:59.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.126+0000 7f0be3200640 1 -- start start 2024-04-25T01:12:59.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.126+0000 7f0be3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bdc1037c0 0x7f0bdc199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:59.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.126+0000 7f0be3200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bdc1049b0 0x7f0bdc19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:59.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.126+0000 7f0be3200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bdc105370 0x7f0bdc1a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:59.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.126+0000 7f0be3200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc10f470 con 0x7f0bdc105370 2024-04-25T01:12:59.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.126+0000 7f0be3200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc10f2f0 con 0x7f0bdc1049b0 2024-04-25T01:12:59.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.126+0000 7f0be3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc10f5f0 con 0x7f0bdc1037c0 2024-04-25T01:12:59.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.127+0000 7f0be1400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bdc1049b0 0x7f0bdc19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:59.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.127+0000 7f0be2800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bdc105370 0x7f0bdc1a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:59.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.127+0000 7f0be1e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bdc1037c0 0x7f0bdc199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:59.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.127+0000 7f0be1400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bdc1049b0 0x7f0bdc19a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:41052/0 (socket says 172.21.15.31:41052) 2024-04-25T01:12:59.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.127+0000 7f0be1400640 1 -- 172.21.15.31:0/1652860997 learned_addr learned my addr 172.21.15.31:0/1652860997 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:12:59.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.127+0000 7f0be2800640 1 -- 172.21.15.31:0/1652860997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bdc1037c0 msgr2=0x7f0bdc199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:59.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.127+0000 7f0be2800640 1 --2- 172.21.15.31:0/1652860997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bdc1037c0 0x7f0bdc199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:59.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.127+0000 7f0be2800640 1 -- 172.21.15.31:0/1652860997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bdc1049b0 msgr2=0x7f0bdc19a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:59.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.127+0000 7f0be2800640 1 --2- 172.21.15.31:0/1652860997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bdc1049b0 0x7f0bdc19a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:59.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.127+0000 7f0be2800640 1 -- 172.21.15.31:0/1652860997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bdc1a1700 con 0x7f0bdc105370 2024-04-25T01:12:59.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.127+0000 7f0be2800640 1 --2- 172.21.15.31:0/1652860997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bdc105370 0x7f0bdc1a1090 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800ec50 tx=0x7f0bd800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:59.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.128+0000 7f0bd6a00640 1 -- 172.21.15.31:0/1652860997 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8003b70 con 0x7f0bdc105370 2024-04-25T01:12:59.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.128+0000 7f0be3200640 1 -- 172.21.15.31:0/1652860997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bdc1a19f0 con 0x7f0bdc105370 2024-04-25T01:12:59.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.128+0000 7f0bd6a00640 1 -- 172.21.15.31:0/1652860997 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bd8003d10 con 0x7f0bdc105370 2024-04-25T01:12:59.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.128+0000 7f0bd6a00640 1 -- 172.21.15.31:0/1652860997 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8010890 con 0x7f0bdc105370 2024-04-25T01:12:59.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.128+0000 7f0be3200640 1 -- 172.21.15.31:0/1652860997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bdc1a1f40 con 0x7f0bdc105370 2024-04-25T01:12:59.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.129+0000 7f0be3200640 1 -- 172.21.15.31:0/1652860997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bdc10ab60 con 0x7f0bdc105370 2024-04-25T01:12:59.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.133+0000 7f0bd6a00640 1 -- 172.21.15.31:0/1652860997 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0bd8004b40 con 0x7f0bdc105370 2024-04-25T01:12:59.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.135+0000 7f0bd6a00640 1 --2- 172.21.15.31:0/1652860997 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0bc40a2b90 0x7f0bc40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:59.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.135+0000 7f0bd6a00640 1 -- 172.21.15.31:0/1652860997 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f0bd80cf260 con 0x7f0bdc105370 2024-04-25T01:12:59.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.135+0000 7f0be1e00640 1 --2- 172.21.15.31:0/1652860997 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0bc40a2b90 0x7f0bc40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:59.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.136+0000 7f0be1e00640 1 --2- 172.21.15.31:0/1652860997 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0bc40a2b90 0x7f0bc40a5050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc104810 tx=0x7f0bd0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:59.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.137+0000 7f0bd6a00640 1 -- 172.21.15.31:0/1652860997 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0bd809c690 con 0x7f0bdc105370 2024-04-25T01:12:59.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.394+0000 7f0be3200640 1 -- 172.21.15.31:0/1652860997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0bdc02d170 con 0x7f0bdc105370 2024-04-25T01:12:59.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.395+0000 7f0bd6a00640 1 -- 172.21.15.31:0/1652860997 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3040 (secure 0 0 0) 0x7f0bd809bf90 con 0x7f0bdc105370 2024-04-25T01:12:59.382 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:12:59.382 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/498218714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":498218714},{"type":"v1","addr":"172.21.15.31:6835","nonce":498218714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/420362823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":420362823},{"type":"v1","addr":"172.21.15.169:6833","nonce":420362823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1943710438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1943710438},{"type":"v1","addr":"172.21.15.40:6833","nonce":1943710438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T01:12:59.382 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 4 2024-04-25T01:12:59.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.397+0000 7f0be3200640 1 -- 172.21.15.31:0/1652860997 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0bc40a2b90 msgr2=0x7f0bc40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:59.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.397+0000 7f0be3200640 1 --2- 172.21.15.31:0/1652860997 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0bc40a2b90 0x7f0bc40a5050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc104810 tx=0x7f0bd0005f90 comp rx=0 tx=0).stop 2024-04-25T01:12:59.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.398+0000 7f0be3200640 1 -- 172.21.15.31:0/1652860997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bdc105370 msgr2=0x7f0bdc1a1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:59.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.398+0000 7f0be3200640 1 --2- 172.21.15.31:0/1652860997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bdc105370 0x7f0bdc1a1090 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800ec50 tx=0x7f0bd800c6a0 comp rx=0 tx=0).stop 2024-04-25T01:12:59.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.398+0000 7f0be3200640 1 -- 172.21.15.31:0/1652860997 shutdown_connections 2024-04-25T01:12:59.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.398+0000 7f0be3200640 1 --2- 172.21.15.31:0/1652860997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bdc105370 0x7f0bdc1a1090 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:59.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.398+0000 7f0be3200640 1 --2- 172.21.15.31:0/1652860997 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0bc40a2b90 0x7f0bc40a5050 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:59.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.398+0000 7f0be3200640 1 --2- 172.21.15.31:0/1652860997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bdc1049b0 0x7f0bdc19a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:59.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.398+0000 7f0be3200640 1 --2- 172.21.15.31:0/1652860997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bdc1037c0 0x7f0bdc199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:59.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.398+0000 7f0be3200640 1 -- 172.21.15.31:0/1652860997 >> 172.21.15.31:0/1652860997 conn(0x7f0bdc0fed50 msgr2=0x7f0bdc107d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:59.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.398+0000 7f0be3200640 1 -- 172.21.15.31:0/1652860997 shutdown_connections 2024-04-25T01:12:59.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:12:59.398+0000 7f0be3200640 1 -- 172.21.15.31:0/1652860997 wait complete. 2024-04-25T01:12:59.400 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 01:12:59 smithi031 podman[68691]: 2024-04-25 01:12:59.123408095 +0000 UTC m=+0.432546218 container start 6d1e33ce69ac2e1912d54b8469f71bb32535411a9a88aac04ccfe450ab331fd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T01:12:59.400 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 01:12:59 smithi031 ceph-mds[68740]: -- 172.21.15.31:0/4072486192 <== mon.2 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562883f7fa40 con 0x562883f52c00 2024-04-25T01:12:59.401 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 01:12:59 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-d[68722]: starting mds.d at 2024-04-25T01:12:59.401 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 01:12:59 smithi031 bash[68691]: 6d1e33ce69ac2e1912d54b8469f71bb32535411a9a88aac04ccfe450ab331fd3 2024-04-25T01:12:59.401 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 01:12:59 smithi031 systemd[1]: Started Ceph mds.d for 95849b94-029f-11ef-bc93-c7b262605968. 2024-04-25T01:12:59.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:59 smithi169 ceph-mon[32759]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:59.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:59 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:59.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:59 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:59.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:59 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:59.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:59 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:12:59.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:59 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:12:59.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:59 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:59.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:12:59 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1652860997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:12:59.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:59 smithi040 ceph-mon[32518]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:59.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:59 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:59.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:59 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:59.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:59 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:59.699 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:59 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:12:59.699 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:59 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:12:59.699 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:59 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:59.699 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:12:59 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1652860997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:12:59.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:59 smithi031 ceph-mon[28053]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:12:59.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:59 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:59.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:59 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:59.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:59 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:12:59.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:59 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:12:59.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:59 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:12:59.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:59 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:12:59.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:12:59 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1652860997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:00.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:00 smithi169 ceph-mon[32759]: Deploying daemon mds.e on smithi040 2024-04-25T01:13:00.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:00 smithi169 ceph-mon[32759]: mds.? [v2:172.21.15.31:6836/4191344115,v1:172.21.15.31:6837/4191344115] up:boot 2024-04-25T01:13:00.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:00 smithi169 ceph-mon[32759]: fsmap 4 up:standby 2024-04-25T01:13:00.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:00 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T01:13:00.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:00 smithi040 ceph-mon[32518]: Deploying daemon mds.e on smithi040 2024-04-25T01:13:00.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:00 smithi040 ceph-mon[32518]: mds.? [v2:172.21.15.31:6836/4191344115,v1:172.21.15.31:6837/4191344115] up:boot 2024-04-25T01:13:00.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:00 smithi040 ceph-mon[32518]: fsmap 4 up:standby 2024-04-25T01:13:00.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:00 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T01:13:00.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:00 smithi031 ceph-mon[28053]: Deploying daemon mds.e on smithi040 2024-04-25T01:13:00.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:00 smithi031 ceph-mon[28053]: mds.? [v2:172.21.15.31:6836/4191344115,v1:172.21.15.31:6837/4191344115] up:boot 2024-04-25T01:13:00.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:00 smithi031 ceph-mon[28053]: fsmap 4 up:standby 2024-04-25T01:13:00.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:00 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T01:13:01.404 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:13:01.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.634+0000 7f68bbe00640 1 -- 172.21.15.31:0/4083728918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68bc077340 msgr2=0x7f68bc0777a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:01.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.634+0000 7f68bbe00640 1 --2- 172.21.15.31:0/4083728918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68bc077340 0x7f68bc0777a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f68a4009900 tx=0x7f68a402a3f0 comp rx=0 tx=0).stop 2024-04-25T01:13:01.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.634+0000 7f68bbe00640 1 -- 172.21.15.31:0/4083728918 shutdown_connections 2024-04-25T01:13:01.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.634+0000 7f68bbe00640 1 --2- 172.21.15.31:0/4083728918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68bc077ce0 0x7f68bc078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:01.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.634+0000 7f68bbe00640 1 --2- 172.21.15.31:0/4083728918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68bc077340 0x7f68bc0777a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:01.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.634+0000 7f68bbe00640 1 --2- 172.21.15.31:0/4083728918 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68bc076100 0x7f68bc076520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:01.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.634+0000 7f68bbe00640 1 -- 172.21.15.31:0/4083728918 >> 172.21.15.31:0/4083728918 conn(0x7f68bc0fed50 msgr2=0x7f68bc101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:01.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.635+0000 7f68bbe00640 1 -- 172.21.15.31:0/4083728918 shutdown_connections 2024-04-25T01:13:01.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.635+0000 7f68bbe00640 1 -- 172.21.15.31:0/4083728918 wait complete. 2024-04-25T01:13:01.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.636+0000 7f68bbe00640 1 Processor -- start 2024-04-25T01:13:01.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.636+0000 7f68bbe00640 1 -- start start 2024-04-25T01:13:01.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.636+0000 7f68bbe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68bc076100 0x7f68bc19dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:01.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.636+0000 7f68bbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68bc077340 0x7f68bc19e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:01.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.636+0000 7f68bbe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68bc077ce0 0x7f68bc1a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:01.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.637+0000 7f68bbe00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc113850 con 0x7f68bc077ce0 2024-04-25T01:13:01.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.637+0000 7f68bbe00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc1136d0 con 0x7f68bc076100 2024-04-25T01:13:01.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.637+0000 7f68bbe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc1139d0 con 0x7f68bc077340 2024-04-25T01:13:01.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.637+0000 7f68baa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68bc076100 0x7f68bc19dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:01.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.637+0000 7f68bb400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68bc077ce0 0x7f68bc1a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:01.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.637+0000 7f68baa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68bc076100 0x7f68bc19dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:41094/0 (socket says 172.21.15.31:41094) 2024-04-25T01:13:01.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.637+0000 7f68baa00640 1 -- 172.21.15.31:0/1616795929 learned_addr learned my addr 172.21.15.31:0/1616795929 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:01.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.637+0000 7f68b3e00640 1 --2- 172.21.15.31:0/1616795929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68bc077340 0x7f68bc19e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:01.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.637+0000 7f68baa00640 1 -- 172.21.15.31:0/1616795929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68bc077340 msgr2=0x7f68bc19e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:01.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.637+0000 7f68baa00640 1 --2- 172.21.15.31:0/1616795929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68bc077340 0x7f68bc19e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:01.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.637+0000 7f68baa00640 1 -- 172.21.15.31:0/1616795929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68bc077ce0 msgr2=0x7f68bc1a5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:01.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.637+0000 7f68baa00640 1 --2- 172.21.15.31:0/1616795929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68bc077ce0 0x7f68bc1a5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:01.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.637+0000 7f68baa00640 1 -- 172.21.15.31:0/1616795929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68bc1a5b00 con 0x7f68bc076100 2024-04-25T01:13:01.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.638+0000 7f68baa00640 1 --2- 172.21.15.31:0/1616795929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68bc076100 0x7f68bc19dee0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00e9f0 tx=0x7f68ac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:01.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.638+0000 7f68b3400640 1 -- 172.21.15.31:0/1616795929 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac00cd80 con 0x7f68bc076100 2024-04-25T01:13:01.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.638+0000 7f68b3400640 1 -- 172.21.15.31:0/1616795929 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68ac010910 con 0x7f68bc076100 2024-04-25T01:13:01.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.638+0000 7f68bbe00640 1 -- 172.21.15.31:0/1616795929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68bc1a5df0 con 0x7f68bc076100 2024-04-25T01:13:01.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.638+0000 7f68b3400640 1 -- 172.21.15.31:0/1616795929 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac018990 con 0x7f68bc076100 2024-04-25T01:13:01.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.638+0000 7f68bbe00640 1 -- 172.21.15.31:0/1616795929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68bc1a6260 con 0x7f68bc076100 2024-04-25T01:13:01.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.639+0000 7f68bbe00640 1 -- 172.21.15.31:0/1616795929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68bc110320 con 0x7f68bc076100 2024-04-25T01:13:01.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.641+0000 7f68b3400640 1 -- 172.21.15.31:0/1616795929 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f68ac010ac0 con 0x7f68bc076100 2024-04-25T01:13:01.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.643+0000 7f68b3400640 1 --2- 172.21.15.31:0/1616795929 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f68a00a2c10 0x7f68a00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:01.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.643+0000 7f68b3400640 1 -- 172.21.15.31:0/1616795929 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f68ac0ceeb0 con 0x7f68bc076100 2024-04-25T01:13:01.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.643+0000 7f68b3e00640 1 --2- 172.21.15.31:0/1616795929 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f68a00a2c10 0x7f68a00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:01.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.644+0000 7f68b3e00640 1 --2- 172.21.15.31:0/1616795929 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f68a00a2c10 0x7f68a00a50d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f68a40314e0 tx=0x7f68a4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:01.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.647+0000 7f68b3400640 1 -- 172.21.15.31:0/1616795929 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f68ac09bbc0 con 0x7f68bc076100 2024-04-25T01:13:01.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:01 smithi169 ceph-mon[32759]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:01.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:01 smithi040 ceph-mon[32518]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:01.698 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 01:13:01 smithi040 systemd[1]: Starting Ceph mds.e for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:13:01.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:01 smithi031 ceph-mon[28053]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:01.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.890+0000 7f68bbe00640 1 -- 172.21.15.31:0/1616795929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68bc062440 con 0x7f68bc076100 2024-04-25T01:13:01.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.891+0000 7f68b3400640 1 -- 172.21.15.31:0/1616795929 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3865 (secure 0 0 0) 0x7f68ac09fec0 con 0x7f68bc076100 2024-04-25T01:13:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:13:01.878 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/498218714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":498218714},{"type":"v1","addr":"172.21.15.31:6835","nonce":498218714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/420362823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":420362823},{"type":"v1","addr":"172.21.15.169:6833","nonce":420362823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1943710438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1943710438},{"type":"v1","addr":"172.21.15.40:6833","nonce":1943710438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/4191344115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":4191344115},{"type":"v1","addr":"172.21.15.31:6837","nonce":4191344115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-04-25T01:13:01.878 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 5 2024-04-25T01:13:01.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.894+0000 7f68bbe00640 1 -- 172.21.15.31:0/1616795929 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f68a00a2c10 msgr2=0x7f68a00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:01.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.894+0000 7f68bbe00640 1 --2- 172.21.15.31:0/1616795929 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f68a00a2c10 0x7f68a00a50d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f68a40314e0 tx=0x7f68a4000bc0 comp rx=0 tx=0).stop 2024-04-25T01:13:01.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.894+0000 7f68bbe00640 1 -- 172.21.15.31:0/1616795929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68bc076100 msgr2=0x7f68bc19dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:01.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.894+0000 7f68bbe00640 1 --2- 172.21.15.31:0/1616795929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68bc076100 0x7f68bc19dee0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00e9f0 tx=0x7f68ac00eeb0 comp rx=0 tx=0).stop 2024-04-25T01:13:01.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.894+0000 7f68bbe00640 1 -- 172.21.15.31:0/1616795929 shutdown_connections 2024-04-25T01:13:01.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.894+0000 7f68bbe00640 1 --2- 172.21.15.31:0/1616795929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68bc077ce0 0x7f68bc1a5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:01.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.894+0000 7f68bbe00640 1 --2- 172.21.15.31:0/1616795929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68bc077340 0x7f68bc19e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:01.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.894+0000 7f68bbe00640 1 --2- 172.21.15.31:0/1616795929 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f68a00a2c10 0x7f68a00a50d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:01.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.894+0000 7f68bbe00640 1 --2- 172.21.15.31:0/1616795929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68bc076100 0x7f68bc19dee0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:01.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.894+0000 7f68bbe00640 1 -- 172.21.15.31:0/1616795929 >> 172.21.15.31:0/1616795929 conn(0x7f68bc0fed50 msgr2=0x7f68bc1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:01.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.894+0000 7f68bbe00640 1 -- 172.21.15.31:0/1616795929 shutdown_connections 2024-04-25T01:13:01.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:01.894+0000 7f68bbe00640 1 -- 172.21.15.31:0/1616795929 wait complete. 2024-04-25T01:13:01.971 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 01:13:01 smithi040 podman[62591]: 2024-04-25 01:13:01.615596592 +0000 UTC m=+0.018600304 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:13:01.971 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 01:13:01 smithi040 podman[62591]: 2024-04-25 01:13:01.714357822 +0000 UTC m=+0.117361526 container create 69a334d0808fae89763a7f6ce362f012cd85f1779a5c65557fd01d30b503ce75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T01:13:01.971 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 01:13:01 smithi040 podman[62591]: 2024-04-25 01:13:01.900461784 +0000 UTC m=+0.303465501 container init 69a334d0808fae89763a7f6ce362f012cd85f1779a5c65557fd01d30b503ce75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-25T01:13:01.971 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 01:13:01 smithi040 podman[62591]: 2024-04-25 01:13:01.903248433 +0000 UTC m=+0.306252154 container start 69a334d0808fae89763a7f6ce362f012cd85f1779a5c65557fd01d30b503ce75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-e, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T01:13:01.971 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 01:13:01 smithi040 ceph-mds[62610]: -- 172.21.15.40:0/4028716264 <== mon.2 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56453c795a40 con 0x56453c769000 2024-04-25T01:13:01.971 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 01:13:01 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-e[62606]: starting mds.e at 2024-04-25T01:13:01.971 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 01:13:01 smithi040 bash[62591]: 69a334d0808fae89763a7f6ce362f012cd85f1779a5c65557fd01d30b503ce75 2024-04-25T01:13:02.439 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 01:13:01 smithi040 systemd[1]: Started Ceph mds.e for 95849b94-029f-11ef-bc93-c7b262605968. 2024-04-25T01:13:02.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:02 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1616795929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:02.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:02 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:02.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:02 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:02.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:02 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:02.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:02 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:13:02.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:02 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:13:02.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:02 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:02.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:02 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1616795929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:02.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:02 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:02.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:02 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:02.699 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:02 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:02.699 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:02 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:13:02.699 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:02 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:13:02.699 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:02 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:02.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:02 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1616795929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:02.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:02 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:02.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:02 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:02.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:02 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:02.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:02 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:13:02.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:02 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:13:02.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:02 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:03.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:03 smithi169 ceph-mon[32759]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:03.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:03 smithi169 ceph-mon[32759]: Deploying daemon mds.f on smithi169 2024-04-25T01:13:03.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:03 smithi169 ceph-mon[32759]: mds.? [v2:172.21.15.40:6834/2916305801,v1:172.21.15.40:6835/2916305801] up:boot 2024-04-25T01:13:03.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:03 smithi169 ceph-mon[32759]: fsmap 5 up:standby 2024-04-25T01:13:03.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:03 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T01:13:03.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:03 smithi040 ceph-mon[32518]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:03.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:03 smithi040 ceph-mon[32518]: Deploying daemon mds.f on smithi169 2024-04-25T01:13:03.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:03 smithi040 ceph-mon[32518]: mds.? [v2:172.21.15.40:6834/2916305801,v1:172.21.15.40:6835/2916305801] up:boot 2024-04-25T01:13:03.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:03 smithi040 ceph-mon[32518]: fsmap 5 up:standby 2024-04-25T01:13:03.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:03 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T01:13:03.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:03 smithi031 ceph-mon[28053]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:03.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:03 smithi031 ceph-mon[28053]: Deploying daemon mds.f on smithi169 2024-04-25T01:13:03.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:03 smithi031 ceph-mon[28053]: mds.? [v2:172.21.15.40:6834/2916305801,v1:172.21.15.40:6835/2916305801] up:boot 2024-04-25T01:13:03.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:03 smithi031 ceph-mon[28053]: fsmap 5 up:standby 2024-04-25T01:13:03.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:03 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T01:13:03.900 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:13:04.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.139+0000 7fa5c3200640 1 -- 172.21.15.31:0/3018467137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5c4077340 msgr2=0x7fa5c40777a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:04.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.139+0000 7fa5c3200640 1 --2- 172.21.15.31:0/3018467137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5c4077340 0x7fa5c40777a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac009be0 tx=0x7fa5ac02a6c0 comp rx=0 tx=0).stop 2024-04-25T01:13:04.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.140+0000 7fa5c3200640 1 -- 172.21.15.31:0/3018467137 shutdown_connections 2024-04-25T01:13:04.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.140+0000 7fa5c3200640 1 --2- 172.21.15.31:0/3018467137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5c4077ce0 0x7fa5c4078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:04.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.140+0000 7fa5c3200640 1 --2- 172.21.15.31:0/3018467137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5c4077340 0x7fa5c40777a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:04.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.140+0000 7fa5c3200640 1 --2- 172.21.15.31:0/3018467137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5c4076100 0x7fa5c4076520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:04.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.140+0000 7fa5c3200640 1 -- 172.21.15.31:0/3018467137 >> 172.21.15.31:0/3018467137 conn(0x7fa5c40fed50 msgr2=0x7fa5c4101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:04.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.140+0000 7fa5c3200640 1 -- 172.21.15.31:0/3018467137 shutdown_connections 2024-04-25T01:13:04.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.140+0000 7fa5c3200640 1 -- 172.21.15.31:0/3018467137 wait complete. 2024-04-25T01:13:04.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.141+0000 7fa5c3200640 1 Processor -- start 2024-04-25T01:13:04.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.141+0000 7fa5c3200640 1 -- start start 2024-04-25T01:13:04.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.141+0000 7fa5c3200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5c4076100 0x7fa5c419deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:04.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.141+0000 7fa5c3200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5c4077340 0x7fa5c419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:04.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.142+0000 7fa5c3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5c4077ce0 0x7fa5c41a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:04.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.142+0000 7fa5c3200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c4113850 con 0x7fa5c4076100 2024-04-25T01:13:04.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.142+0000 7fa5c3200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c41136d0 con 0x7fa5c4077340 2024-04-25T01:13:04.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.142+0000 7fa5c3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c41139d0 con 0x7fa5c4077ce0 2024-04-25T01:13:04.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.142+0000 7fa5c1e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5c4076100 0x7fa5c419deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:04.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.142+0000 7fa5c2800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5c4077ce0 0x7fa5c41a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:04.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.142+0000 7fa5c2800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5c4077ce0 0x7fa5c41a5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:32822/0 (socket says 172.21.15.31:32822) 2024-04-25T01:13:04.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.142+0000 7fa5c1e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5c4076100 0x7fa5c419deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49706/0 (socket says 172.21.15.31:49706) 2024-04-25T01:13:04.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.142+0000 7fa5c2800640 1 -- 172.21.15.31:0/2789081433 learned_addr learned my addr 172.21.15.31:0/2789081433 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:04.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.142+0000 7fa5c1400640 1 --2- 172.21.15.31:0/2789081433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5c4077340 0x7fa5c419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:04.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.142+0000 7fa5c2800640 1 -- 172.21.15.31:0/2789081433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5c4077340 msgr2=0x7fa5c419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:04.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.142+0000 7fa5c2800640 1 --2- 172.21.15.31:0/2789081433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5c4077340 0x7fa5c419e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:04.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.142+0000 7fa5c2800640 1 -- 172.21.15.31:0/2789081433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5c4076100 msgr2=0x7fa5c419deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:04.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.142+0000 7fa5c2800640 1 --2- 172.21.15.31:0/2789081433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5c4076100 0x7fa5c419deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:04.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.142+0000 7fa5c2800640 1 -- 172.21.15.31:0/2789081433 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c41a5ad0 con 0x7fa5c4077ce0 2024-04-25T01:13:04.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.143+0000 7fa5c2800640 1 --2- 172.21.15.31:0/2789081433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5c4077ce0 0x7fa5c41a5460 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00e970 tx=0x7fa5bc00ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:04.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.143+0000 7fa5baa00640 1 -- 172.21.15.31:0/2789081433 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc00ccc0 con 0x7fa5c4077ce0 2024-04-25T01:13:04.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.143+0000 7fa5baa00640 1 -- 172.21.15.31:0/2789081433 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5bc005500 con 0x7fa5c4077ce0 2024-04-25T01:13:04.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.143+0000 7fa5c3200640 1 -- 172.21.15.31:0/2789081433 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c41a5dc0 con 0x7fa5c4077ce0 2024-04-25T01:13:04.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.143+0000 7fa5baa00640 1 -- 172.21.15.31:0/2789081433 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc010690 con 0x7fa5c4077ce0 2024-04-25T01:13:04.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.143+0000 7fa5c3200640 1 -- 172.21.15.31:0/2789081433 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c41a62e0 con 0x7fa5c4077ce0 2024-04-25T01:13:04.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.144+0000 7fa5c3200640 1 -- 172.21.15.31:0/2789081433 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c41102e0 con 0x7fa5c4077ce0 2024-04-25T01:13:04.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.146+0000 7fa5baa00640 1 -- 172.21.15.31:0/2789081433 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa5bc004e50 con 0x7fa5c4077ce0 2024-04-25T01:13:04.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.148+0000 7fa5baa00640 1 --2- 172.21.15.31:0/2789081433 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa5a40a2c10 0x7fa5a40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:04.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.148+0000 7fa5baa00640 1 -- 172.21.15.31:0/2789081433 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fa5bc0cef60 con 0x7fa5c4077ce0 2024-04-25T01:13:04.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.148+0000 7fa5c1e00640 1 --2- 172.21.15.31:0/2789081433 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa5a40a2c10 0x7fa5a40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:04.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.149+0000 7fa5c1e00640 1 --2- 172.21.15.31:0/2789081433 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa5a40a2c10 0x7fa5a40a50d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4005d00 tx=0x7fa5b400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:04.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.156+0000 7fa5baa00640 1 -- 172.21.15.31:0/2789081433 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa5bc09c2c0 con 0x7fa5c4077ce0 2024-04-25T01:13:04.385 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 01:13:04 smithi169 systemd[1]: Starting Ceph mds.f for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:13:04.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.399+0000 7fa5c3200640 1 -- 172.21.15.31:0/2789081433 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5c402d170 con 0x7fa5c4077ce0 2024-04-25T01:13:04.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.399+0000 7fa5baa00640 1 -- 172.21.15.31:0/2789081433 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4690 (secure 0 0 0) 0x7fa5bc09bbc0 con 0x7fa5c4077ce0 2024-04-25T01:13:04.386 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:13:04.386 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/498218714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":498218714},{"type":"v1","addr":"172.21.15.31:6835","nonce":498218714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/420362823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":420362823},{"type":"v1","addr":"172.21.15.169:6833","nonce":420362823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1943710438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1943710438},{"type":"v1","addr":"172.21.15.40:6833","nonce":1943710438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/4191344115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":4191344115},{"type":"v1","addr":"172.21.15.31:6837","nonce":4191344115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2916305801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2916305801},{"type":"v1","addr":"172.21.15.40:6835","nonce":2916305801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T01:13:04.386 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 6 2024-04-25T01:13:04.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.402+0000 7fa5c3200640 1 -- 172.21.15.31:0/2789081433 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa5a40a2c10 msgr2=0x7fa5a40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:04.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.402+0000 7fa5c3200640 1 --2- 172.21.15.31:0/2789081433 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa5a40a2c10 0x7fa5a40a50d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4005d00 tx=0x7fa5b400b410 comp rx=0 tx=0).stop 2024-04-25T01:13:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.402+0000 7fa5c3200640 1 -- 172.21.15.31:0/2789081433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5c4077ce0 msgr2=0x7fa5c41a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.402+0000 7fa5c3200640 1 --2- 172.21.15.31:0/2789081433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5c4077ce0 0x7fa5c41a5460 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00e970 tx=0x7fa5bc00ee30 comp rx=0 tx=0).stop 2024-04-25T01:13:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.402+0000 7fa5c3200640 1 -- 172.21.15.31:0/2789081433 shutdown_connections 2024-04-25T01:13:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.402+0000 7fa5c3200640 1 --2- 172.21.15.31:0/2789081433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5c4077ce0 0x7fa5c41a5460 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.402+0000 7fa5c3200640 1 --2- 172.21.15.31:0/2789081433 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa5a40a2c10 0x7fa5a40a50d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.402+0000 7fa5c3200640 1 --2- 172.21.15.31:0/2789081433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5c4077340 0x7fa5c419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.402+0000 7fa5c3200640 1 --2- 172.21.15.31:0/2789081433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5c4076100 0x7fa5c419deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.402+0000 7fa5c3200640 1 -- 172.21.15.31:0/2789081433 >> 172.21.15.31:0/2789081433 conn(0x7fa5c40fed50 msgr2=0x7fa5c4100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.402+0000 7fa5c3200640 1 -- 172.21.15.31:0/2789081433 shutdown_connections 2024-04-25T01:13:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:04.402+0000 7fa5c3200640 1 -- 172.21.15.31:0/2789081433 wait complete. 2024-04-25T01:13:04.637 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:04 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2789081433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:04.637 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 01:13:04 smithi169 podman[62918]: 2024-04-25 01:13:04.39840155 +0000 UTC m=+0.108523927 container create b6396cd0d5014a2a1aa4667f226c71c4cd019f3dc0fc335636fab72325fc4c15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T01:13:04.637 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 01:13:04 smithi169 podman[62918]: 2024-04-25 01:13:04.307976362 +0000 UTC m=+0.018098752 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:13:04.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:04 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2789081433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:04.945 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 01:13:04 smithi169 podman[62918]: 2024-04-25 01:13:04.651175221 +0000 UTC m=+0.361297602 container init b6396cd0d5014a2a1aa4667f226c71c4cd019f3dc0fc335636fab72325fc4c15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-25T01:13:04.945 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 01:13:04 smithi169 podman[62918]: 2024-04-25 01:13:04.653948762 +0000 UTC m=+0.364071141 container start b6396cd0d5014a2a1aa4667f226c71c4cd019f3dc0fc335636fab72325fc4c15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:13:04.945 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 01:13:04 smithi169 ceph-mds[62937]: -- 172.21.15.169:0/1431466838 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ae02ab5a40 con 0x55ae02a88800 2024-04-25T01:13:04.946 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 01:13:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-f[62933]: starting mds.f at 2024-04-25T01:13:04.946 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 01:13:04 smithi169 bash[62918]: b6396cd0d5014a2a1aa4667f226c71c4cd019f3dc0fc335636fab72325fc4c15 2024-04-25T01:13:04.946 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 01:13:04 smithi169 systemd[1]: Started Ceph mds.f for 95849b94-029f-11ef-bc93-c7b262605968. 2024-04-25T01:13:04.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:04 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2789081433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:06.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:05 smithi169 ceph-mon[32759]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:06.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:05 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:06.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:05 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:06.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:05 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:06.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:05 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:13:06.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:05 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:13:06.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:05 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:06.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:05 smithi169 ceph-mon[32759]: Deploying daemon mds.c on smithi169 2024-04-25T01:13:06.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:05 smithi169 ceph-mon[32759]: mds.? [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] up:boot 2024-04-25T01:13:06.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:05 smithi169 ceph-mon[32759]: fsmap 6 up:standby 2024-04-25T01:13:06.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:05 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T01:13:06.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:05 smithi040 ceph-mon[32518]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:06.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:05 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:06.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:05 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:06.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:05 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:06.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:05 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:13:06.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:05 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:13:06.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:05 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:06.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:05 smithi040 ceph-mon[32518]: Deploying daemon mds.c on smithi169 2024-04-25T01:13:06.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:05 smithi040 ceph-mon[32518]: mds.? [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] up:boot 2024-04-25T01:13:06.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:05 smithi040 ceph-mon[32518]: fsmap 6 up:standby 2024-04-25T01:13:06.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:05 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T01:13:06.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:05 smithi031 ceph-mon[28053]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:06.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:05 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:06.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:05 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:06.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:05 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:06.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:05 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:13:06.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:05 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:13:06.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:05 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:06.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:05 smithi031 ceph-mon[28053]: Deploying daemon mds.c on smithi169 2024-04-25T01:13:06.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:05 smithi031 ceph-mon[28053]: mds.? [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] up:boot 2024-04-25T01:13:06.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:05 smithi031 ceph-mon[28053]: fsmap 6 up:standby 2024-04-25T01:13:06.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:05 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T01:13:06.409 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:13:06.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.646+0000 7f93aa200640 1 -- 172.21.15.31:0/2669327923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93a4103160 msgr2=0x7f93a41099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:06.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.646+0000 7f93aa200640 1 --2- 172.21.15.31:0/2669327923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93a4103160 0x7f93a41099f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f939c009ca0 tx=0x7f939c02a8c0 comp rx=0 tx=0).stop 2024-04-25T01:13:06.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.646+0000 7f93aa200640 1 -- 172.21.15.31:0/2669327923 shutdown_connections 2024-04-25T01:13:06.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.646+0000 7f93aa200640 1 --2- 172.21.15.31:0/2669327923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93a4103160 0x7f93a41099f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:06.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.646+0000 7f93aa200640 1 --2- 172.21.15.31:0/2669327923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a41027a0 0x7f93a4102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:06.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.646+0000 7f93aa200640 1 --2- 172.21.15.31:0/2669327923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93a41015b0 0x7f93a41019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:06.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.646+0000 7f93aa200640 1 -- 172.21.15.31:0/2669327923 >> 172.21.15.31:0/2669327923 conn(0x7f93a40fcb20 msgr2=0x7f93a40fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:06.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.646+0000 7f93aa200640 1 -- 172.21.15.31:0/2669327923 shutdown_connections 2024-04-25T01:13:06.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.646+0000 7f93aa200640 1 -- 172.21.15.31:0/2669327923 wait complete. 2024-04-25T01:13:06.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.647+0000 7f93aa200640 1 Processor -- start 2024-04-25T01:13:06.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.647+0000 7f93aa200640 1 -- start start 2024-04-25T01:13:06.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.648+0000 7f93aa200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93a41015b0 0x7f93a41978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:06.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.648+0000 7f93aa200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93a41027a0 0x7f93a4197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:06.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.648+0000 7f93aa200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a4103160 0x7f93a419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:06.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.648+0000 7f93aa200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f93a410d280 con 0x7f93a4103160 2024-04-25T01:13:06.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.648+0000 7f93aa200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f93a410d100 con 0x7f93a41015b0 2024-04-25T01:13:06.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.648+0000 7f93aa200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f93a410d400 con 0x7f93a41027a0 2024-04-25T01:13:06.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.648+0000 7f93a3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93a41027a0 0x7f93a4197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:06.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.648+0000 7f93a8e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93a41015b0 0x7f93a41978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:06.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.648+0000 7f93a3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93a41027a0 0x7f93a4197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:32842/0 (socket says 172.21.15.31:32842) 2024-04-25T01:13:06.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.648+0000 7f93a3e00640 1 -- 172.21.15.31:0/2128060311 learned_addr learned my addr 172.21.15.31:0/2128060311 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:06.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.648+0000 7f93a9800640 1 --2- 172.21.15.31:0/2128060311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a4103160 0x7f93a419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:06.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.649+0000 7f93a3e00640 1 -- 172.21.15.31:0/2128060311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93a41015b0 msgr2=0x7f93a41978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:06.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.649+0000 7f93a3e00640 1 --2- 172.21.15.31:0/2128060311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93a41015b0 0x7f93a41978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:06.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.649+0000 7f93a3e00640 1 -- 172.21.15.31:0/2128060311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a4103160 msgr2=0x7f93a419ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:06.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.649+0000 7f93a3e00640 1 --2- 172.21.15.31:0/2128060311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a4103160 0x7f93a419ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:06.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.649+0000 7f93a3e00640 1 -- 172.21.15.31:0/2128060311 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a419f4c0 con 0x7f93a41027a0 2024-04-25T01:13:06.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.649+0000 7f93a3e00640 1 --2- 172.21.15.31:0/2128060311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93a41027a0 0x7f93a4197de0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f939000e9f0 tx=0x7f939000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:06.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.649+0000 7f93a1600640 1 -- 172.21.15.31:0/2128060311 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939000cd80 con 0x7f93a41027a0 2024-04-25T01:13:06.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.649+0000 7f93a1600640 1 -- 172.21.15.31:0/2128060311 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9390010910 con 0x7f93a41027a0 2024-04-25T01:13:06.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.649+0000 7f93aa200640 1 -- 172.21.15.31:0/2128060311 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93a419f7b0 con 0x7f93a41027a0 2024-04-25T01:13:06.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.649+0000 7f93a1600640 1 -- 172.21.15.31:0/2128060311 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9390018990 con 0x7f93a41027a0 2024-04-25T01:13:06.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.649+0000 7f93aa200640 1 -- 172.21.15.31:0/2128060311 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93a419fd50 con 0x7f93a41027a0 2024-04-25T01:13:06.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.651+0000 7f93aa200640 1 -- 172.21.15.31:0/2128060311 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93a41088d0 con 0x7f93a41027a0 2024-04-25T01:13:06.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.653+0000 7f93a1600640 1 -- 172.21.15.31:0/2128060311 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9390010ac0 con 0x7f93a41027a0 2024-04-25T01:13:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.655+0000 7f93a1600640 1 --2- 172.21.15.31:0/2128060311 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f938c0a2c10 0x7f938c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.655+0000 7f93a1600640 1 -- 172.21.15.31:0/2128060311 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f93900ceeb0 con 0x7f93a41027a0 2024-04-25T01:13:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.655+0000 7f93a8e00640 1 --2- 172.21.15.31:0/2128060311 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f938c0a2c10 0x7f938c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.656+0000 7f93a8e00640 1 --2- 172.21.15.31:0/2128060311 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f938c0a2c10 0x7f938c0a50d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f93a4102600 tx=0x7f9398005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:06.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.663+0000 7f93a1600640 1 -- 172.21.15.31:0/2128060311 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f939009bbc0 con 0x7f93a41027a0 2024-04-25T01:13:06.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.913+0000 7f93aa200640 1 -- 172.21.15.31:0/2128060311 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93a402d170 con 0x7f93a41027a0 2024-04-25T01:13:06.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.914+0000 7f93a1600640 1 -- 172.21.15.31:0/2128060311 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5518 (secure 0 0 0) 0x7f939009fec0 con 0x7f93a41027a0 2024-04-25T01:13:06.901 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:13:06.902 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/498218714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":498218714},{"type":"v1","addr":"172.21.15.31:6835","nonce":498218714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/420362823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":420362823},{"type":"v1","addr":"172.21.15.169:6833","nonce":420362823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1943710438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1943710438},{"type":"v1","addr":"172.21.15.40:6833","nonce":1943710438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/4191344115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":4191344115},{"type":"v1","addr":"172.21.15.31:6837","nonce":4191344115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2916305801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2916305801},{"type":"v1","addr":"172.21.15.40:6835","nonce":2916305801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2253264784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2253264784},{"type":"v1","addr":"172.21.15.169:6835","nonce":2253264784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-25T01:13:06.902 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 7 2024-04-25T01:13:06.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.916+0000 7f93aa200640 1 -- 172.21.15.31:0/2128060311 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f938c0a2c10 msgr2=0x7f938c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:06.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.916+0000 7f93aa200640 1 --2- 172.21.15.31:0/2128060311 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f938c0a2c10 0x7f938c0a50d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f93a4102600 tx=0x7f9398005f90 comp rx=0 tx=0).stop 2024-04-25T01:13:06.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.916+0000 7f93aa200640 1 -- 172.21.15.31:0/2128060311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93a41027a0 msgr2=0x7f93a4197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:06.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.916+0000 7f93aa200640 1 --2- 172.21.15.31:0/2128060311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93a41027a0 0x7f93a4197de0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f939000e9f0 tx=0x7f939000eeb0 comp rx=0 tx=0).stop 2024-04-25T01:13:06.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.917+0000 7f93aa200640 1 -- 172.21.15.31:0/2128060311 shutdown_connections 2024-04-25T01:13:06.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.917+0000 7f93aa200640 1 --2- 172.21.15.31:0/2128060311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a4103160 0x7f93a419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:06.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.917+0000 7f93aa200640 1 --2- 172.21.15.31:0/2128060311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93a41027a0 0x7f93a4197de0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:06.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.917+0000 7f93aa200640 1 --2- 172.21.15.31:0/2128060311 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f938c0a2c10 0x7f938c0a50d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:06.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.917+0000 7f93aa200640 1 --2- 172.21.15.31:0/2128060311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93a41015b0 0x7f93a41978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:06.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.917+0000 7f93aa200640 1 -- 172.21.15.31:0/2128060311 >> 172.21.15.31:0/2128060311 conn(0x7f93a40fcb20 msgr2=0x7f93a4105aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:06.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.917+0000 7f93aa200640 1 -- 172.21.15.31:0/2128060311 shutdown_connections 2024-04-25T01:13:06.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:06.917+0000 7f93aa200640 1 -- 172.21.15.31:0/2128060311 wait complete. 2024-04-25T01:13:06.946 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 01:13:06 smithi169 systemd[1]: Starting Ceph mds.c for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:13:07.390 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 01:13:07 smithi169 podman[63225]: 2024-04-25 01:13:07.085289744 +0000 UTC m=+0.099178586 container create a80419ae6cb96211dd12912a0d598e0fbca776e0c0a443a289fb9b222c0b9723 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T01:13:07.391 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 01:13:07 smithi169 podman[63225]: 2024-04-25 01:13:07.004106141 +0000 UTC m=+0.017994942 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:13:07.391 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 01:13:07 smithi169 podman[63225]: 2024-04-25 01:13:07.304640722 +0000 UTC m=+0.318529523 container init a80419ae6cb96211dd12912a0d598e0fbca776e0c0a443a289fb9b222c0b9723 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:13:07.391 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 01:13:07 smithi169 podman[63225]: 2024-04-25 01:13:07.307073462 +0000 UTC m=+0.320962271 container start a80419ae6cb96211dd12912a0d598e0fbca776e0c0a443a289fb9b222c0b9723 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-c, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:13:07.391 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 01:13:07 smithi169 ceph-mds[63245]: -- 172.21.15.169:0/1354076286 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561eaf8f1a40 con 0x561eaf8c5000 2024-04-25T01:13:07.391 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 01:13:07 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-c[63241]: starting mds.c at 2024-04-25T01:13:07.391 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 01:13:07 smithi169 bash[63225]: a80419ae6cb96211dd12912a0d598e0fbca776e0c0a443a289fb9b222c0b9723 2024-04-25T01:13:07.695 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 01:13:07 smithi169 systemd[1]: Started Ceph mds.c for 95849b94-029f-11ef-bc93-c7b262605968. 2024-04-25T01:13:08.174 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:07 smithi040 ceph-mon[32518]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:08.174 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:07 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2128060311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:08.174 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:07 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:08.174 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:07 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:08.174 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:07 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:08.174 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:07 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:13:08.174 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:07 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:13:08.174 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:07 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:08.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:07 smithi169 ceph-mon[32759]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:08.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:07 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2128060311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:08.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:07 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:08.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:07 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:08.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:07 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:08.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:07 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:13:08.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:07 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:13:08.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:07 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:08.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:07 smithi031 ceph-mon[28053]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:08.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:07 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2128060311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:08.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:07 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:08.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:07 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:08.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:07 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:08.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:07 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:13:08.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:07 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:13:08.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:07 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:08.925 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:13:09.094 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:08 smithi040 ceph-mon[32518]: Deploying daemon mds.h on smithi040 2024-04-25T01:13:09.094 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:08 smithi040 ceph-mon[32518]: mds.? [v2:172.21.15.169:6836/2883308641,v1:172.21.15.169:6837/2883308641] up:boot 2024-04-25T01:13:09.094 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:08 smithi040 ceph-mon[32518]: fsmap 7 up:standby 2024-04-25T01:13:09.094 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:08 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T01:13:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.161+0000 7f050cc00640 1 -- 172.21.15.31:0/165691741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05081049b0 msgr2=0x7f0508104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.161+0000 7f050cc00640 1 --2- 172.21.15.31:0/165691741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05081049b0 0x7f0508104e30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009860 tx=0x7f04f002a340 comp rx=0 tx=0).stop 2024-04-25T01:13:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.161+0000 7f050cc00640 1 -- 172.21.15.31:0/165691741 shutdown_connections 2024-04-25T01:13:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.161+0000 7f050cc00640 1 --2- 172.21.15.31:0/165691741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0508105370 0x7f050810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.161+0000 7f050cc00640 1 --2- 172.21.15.31:0/165691741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05081049b0 0x7f0508104e30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.161+0000 7f050cc00640 1 --2- 172.21.15.31:0/165691741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05081037c0 0x7f0508103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.161+0000 7f050cc00640 1 -- 172.21.15.31:0/165691741 >> 172.21.15.31:0/165691741 conn(0x7f05080fed50 msgr2=0x7f0508101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.161+0000 7f050cc00640 1 -- 172.21.15.31:0/165691741 shutdown_connections 2024-04-25T01:13:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.161+0000 7f050cc00640 1 -- 172.21.15.31:0/165691741 wait complete. 2024-04-25T01:13:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.162+0000 7f050cc00640 1 Processor -- start 2024-04-25T01:13:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.163+0000 7f050cc00640 1 -- start start 2024-04-25T01:13:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.163+0000 7f050cc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05081037c0 0x7f0508199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.163+0000 7f050cc00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05081049b0 0x7f0508199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.163+0000 7f050cc00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0508105370 0x7f05081a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.163+0000 7f050cc00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f050810f440 con 0x7f05081049b0 2024-04-25T01:13:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.163+0000 7f050cc00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f050810f2c0 con 0x7f0508105370 2024-04-25T01:13:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.163+0000 7f050cc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f050810f5c0 con 0x7f05081037c0 2024-04-25T01:13:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.163+0000 7f0506a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05081049b0 0x7f0508199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.163+0000 7f0506a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05081049b0 0x7f0508199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48172/0 (socket says 172.21.15.31:48172) 2024-04-25T01:13:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.163+0000 7f0507400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05081037c0 0x7f0508199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.163+0000 7f0506a00640 1 -- 172.21.15.31:0/2729807786 learned_addr learned my addr 172.21.15.31:0/2729807786 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.163+0000 7f0507e00640 1 --2- 172.21.15.31:0/2729807786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0508105370 0x7f05081a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.164+0000 7f0506a00640 1 -- 172.21.15.31:0/2729807786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05081037c0 msgr2=0x7f0508199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.164+0000 7f0506a00640 1 --2- 172.21.15.31:0/2729807786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05081037c0 0x7f0508199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.164+0000 7f0506a00640 1 -- 172.21.15.31:0/2729807786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0508105370 msgr2=0x7f05081a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.164+0000 7f0506a00640 1 --2- 172.21.15.31:0/2729807786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0508105370 0x7f05081a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.164+0000 7f0506a00640 1 -- 172.21.15.31:0/2729807786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05081a16d0 con 0x7f05081049b0 2024-04-25T01:13:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.164+0000 7f0507400640 1 --2- 172.21.15.31:0/2729807786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05081037c0 0x7f0508199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:13:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.164+0000 7f0506a00640 1 --2- 172.21.15.31:0/2729807786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05081049b0 0x7f0508199ff0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f04f0035040 tx=0x7f04f0033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.165+0000 7f04ffe00640 1 -- 172.21.15.31:0/2729807786 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0037040 con 0x7f05081049b0 2024-04-25T01:13:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.165+0000 7f04ffe00640 1 -- 172.21.15.31:0/2729807786 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04f0030850 con 0x7f05081049b0 2024-04-25T01:13:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.165+0000 7f04ffe00640 1 -- 172.21.15.31:0/2729807786 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0004920 con 0x7f05081049b0 2024-04-25T01:13:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.165+0000 7f050cc00640 1 -- 172.21.15.31:0/2729807786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05081a1960 con 0x7f05081049b0 2024-04-25T01:13:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.165+0000 7f050cc00640 1 -- 172.21.15.31:0/2729807786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05081a1e50 con 0x7f05081049b0 2024-04-25T01:13:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.166+0000 7f050cc00640 1 -- 172.21.15.31:0/2729807786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f050810aae0 con 0x7f05081049b0 2024-04-25T01:13:09.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.168+0000 7f04ffe00640 1 -- 172.21.15.31:0/2729807786 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f04f0004050 con 0x7f05081049b0 2024-04-25T01:13:09.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.169+0000 7f04ffe00640 1 --2- 172.21.15.31:0/2729807786 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f04ec0a2b90 0x7f04ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:09.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.170+0000 7f04ffe00640 1 -- 172.21.15.31:0/2729807786 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f04f00eebb0 con 0x7f05081049b0 2024-04-25T01:13:09.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.170+0000 7f0507400640 1 --2- 172.21.15.31:0/2729807786 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f04ec0a2b90 0x7f04ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:09.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.170+0000 7f0507400640 1 --2- 172.21.15.31:0/2729807786 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f04ec0a2b90 0x7f04ec0a5050 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0508104810 tx=0x7f04f8010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:09.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.174+0000 7f04ffe00640 1 -- 172.21.15.31:0/2729807786 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f04f00bbf10 con 0x7f05081049b0 2024-04-25T01:13:09.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:08 smithi169 ceph-mon[32759]: Deploying daemon mds.h on smithi040 2024-04-25T01:13:09.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:08 smithi169 ceph-mon[32759]: mds.? [v2:172.21.15.169:6836/2883308641,v1:172.21.15.169:6837/2883308641] up:boot 2024-04-25T01:13:09.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:08 smithi169 ceph-mon[32759]: fsmap 7 up:standby 2024-04-25T01:13:09.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:08 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T01:13:09.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:08 smithi031 ceph-mon[28053]: Deploying daemon mds.h on smithi040 2024-04-25T01:13:09.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:08 smithi031 ceph-mon[28053]: mds.? [v2:172.21.15.169:6836/2883308641,v1:172.21.15.169:6837/2883308641] up:boot 2024-04-25T01:13:09.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:08 smithi031 ceph-mon[28053]: fsmap 7 up:standby 2024-04-25T01:13:09.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:08 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T01:13:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.422+0000 7f050cc00640 1 -- 172.21.15.31:0/2729807786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0508062340 con 0x7f05081049b0 2024-04-25T01:13:09.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.424+0000 7f04ffe00640 1 -- 172.21.15.31:0/2729807786 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6346 (secure 0 0 0) 0x7f04f00bb810 con 0x7f05081049b0 2024-04-25T01:13:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:13:09.411 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/498218714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":498218714},{"type":"v1","addr":"172.21.15.31:6835","nonce":498218714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/420362823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":420362823},{"type":"v1","addr":"172.21.15.169:6833","nonce":420362823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1943710438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1943710438},{"type":"v1","addr":"172.21.15.40:6833","nonce":1943710438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/4191344115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":4191344115},{"type":"v1","addr":"172.21.15.31:6837","nonce":4191344115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2916305801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2916305801},{"type":"v1","addr":"172.21.15.40:6835","nonce":2916305801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2253264784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2253264784},{"type":"v1","addr":"172.21.15.169:6835","nonce":2253264784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2883308641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2883308641},{"type":"v1","addr":"172.21.15.169:6837","nonce":2883308641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T01:13:09.412 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2024-04-25T01:13:09.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.426+0000 7f050cc00640 1 -- 172.21.15.31:0/2729807786 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f04ec0a2b90 msgr2=0x7f04ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:09.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.426+0000 7f050cc00640 1 --2- 172.21.15.31:0/2729807786 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f04ec0a2b90 0x7f04ec0a5050 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0508104810 tx=0x7f04f8010040 comp rx=0 tx=0).stop 2024-04-25T01:13:09.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.426+0000 7f050cc00640 1 -- 172.21.15.31:0/2729807786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05081049b0 msgr2=0x7f0508199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:09.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.426+0000 7f050cc00640 1 --2- 172.21.15.31:0/2729807786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05081049b0 0x7f0508199ff0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f04f0035040 tx=0x7f04f0033d90 comp rx=0 tx=0).stop 2024-04-25T01:13:09.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.427+0000 7f050cc00640 1 -- 172.21.15.31:0/2729807786 shutdown_connections 2024-04-25T01:13:09.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.427+0000 7f050cc00640 1 --2- 172.21.15.31:0/2729807786 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f04ec0a2b90 0x7f04ec0a5050 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:09.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.427+0000 7f050cc00640 1 --2- 172.21.15.31:0/2729807786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0508105370 0x7f05081a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:09.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.427+0000 7f050cc00640 1 --2- 172.21.15.31:0/2729807786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05081049b0 0x7f0508199ff0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:09.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.427+0000 7f050cc00640 1 --2- 172.21.15.31:0/2729807786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05081037c0 0x7f0508199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:09.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.427+0000 7f050cc00640 1 -- 172.21.15.31:0/2729807786 >> 172.21.15.31:0/2729807786 conn(0x7f05080fed50 msgr2=0x7f0508107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:09.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.427+0000 7f050cc00640 1 -- 172.21.15.31:0/2729807786 shutdown_connections 2024-04-25T01:13:09.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:09.427+0000 7f050cc00640 1 -- 172.21.15.31:0/2729807786 wait complete. 2024-04-25T01:13:10.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:09 smithi169 ceph-mon[32759]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:10.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:09 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2729807786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:10.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:09 smithi040 ceph-mon[32518]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:10.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:09 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2729807786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:10.199 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 01:13:10 smithi040 systemd[1]: Started Ceph mds.h for 95849b94-029f-11ef-bc93-c7b262605968. 2024-04-25T01:13:10.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:09 smithi031 ceph-mon[28053]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:10.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:09 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2729807786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:11.436 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:13:11.445 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:11 smithi169 ceph-mon[32759]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:11.445 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:11 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:11.445 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:11 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:11.446 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:11 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:11.446 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:11 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:13:11.446 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:11 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:13:11.446 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:11 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:11.446 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:11 smithi169 ceph-mon[32759]: Deploying daemon mds.i on smithi169 2024-04-25T01:13:11.446 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:11 smithi169 ceph-mon[32759]: mds.? [v2:172.21.15.40:6836/590109562,v1:172.21.15.40:6837/590109562] up:boot 2024-04-25T01:13:11.446 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:11 smithi169 ceph-mon[32759]: fsmap 8 up:standby 2024-04-25T01:13:11.446 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:11 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T01:13:11.448 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:11 smithi040 ceph-mon[32518]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:11.448 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:11 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:11.448 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:11 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:11.448 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:11 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:11.449 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:11 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:13:11.449 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:11 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:13:11.449 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:11 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:11.449 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:11 smithi040 ceph-mon[32518]: Deploying daemon mds.i on smithi169 2024-04-25T01:13:11.449 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:11 smithi040 ceph-mon[32518]: mds.? [v2:172.21.15.40:6836/590109562,v1:172.21.15.40:6837/590109562] up:boot 2024-04-25T01:13:11.449 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:11 smithi040 ceph-mon[32518]: fsmap 8 up:standby 2024-04-25T01:13:11.449 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:11 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T01:13:11.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:11 smithi031 ceph-mon[28053]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:11.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:11 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:11.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:11 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:11.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:11 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:11.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:11 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:13:11.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:11 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:13:11.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:11 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:11.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:11 smithi031 ceph-mon[28053]: Deploying daemon mds.i on smithi169 2024-04-25T01:13:11.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:11 smithi031 ceph-mon[28053]: mds.? [v2:172.21.15.40:6836/590109562,v1:172.21.15.40:6837/590109562] up:boot 2024-04-25T01:13:11.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:11 smithi031 ceph-mon[28053]: fsmap 8 up:standby 2024-04-25T01:13:11.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:11 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T01:13:11.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.674+0000 7f083de00640 1 -- 172.21.15.31:0/1027427559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08381037c0 msgr2=0x7f0838103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:11.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.674+0000 7f083de00640 1 --2- 172.21.15.31:0/1027427559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08381037c0 0x7f0838103be0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0828009900 tx=0x7f082802a410 comp rx=0 tx=0).stop 2024-04-25T01:13:11.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.674+0000 7f083de00640 1 -- 172.21.15.31:0/1027427559 shutdown_connections 2024-04-25T01:13:11.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.674+0000 7f083de00640 1 --2- 172.21.15.31:0/1027427559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0838105370 0x7f083810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:11.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.674+0000 7f083de00640 1 --2- 172.21.15.31:0/1027427559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08381049b0 0x7f0838104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:11.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.674+0000 7f083de00640 1 --2- 172.21.15.31:0/1027427559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08381037c0 0x7f0838103be0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:11.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.674+0000 7f083de00640 1 -- 172.21.15.31:0/1027427559 >> 172.21.15.31:0/1027427559 conn(0x7f08380fed50 msgr2=0x7f0838101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:11.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.674+0000 7f083de00640 1 -- 172.21.15.31:0/1027427559 shutdown_connections 2024-04-25T01:13:11.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.674+0000 7f083de00640 1 -- 172.21.15.31:0/1027427559 wait complete. 2024-04-25T01:13:11.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.675+0000 7f083de00640 1 Processor -- start 2024-04-25T01:13:11.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.675+0000 7f083de00640 1 -- start start 2024-04-25T01:13:11.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.676+0000 7f083de00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08381037c0 0x7f0838199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:11.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.676+0000 7f083de00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08381049b0 0x7f083819a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:11.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.676+0000 7f083de00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0838105370 0x7f08381a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:11.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.676+0000 7f083de00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f083810f460 con 0x7f0838105370 2024-04-25T01:13:11.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.676+0000 7f083de00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f083810f2e0 con 0x7f08381049b0 2024-04-25T01:13:11.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.676+0000 7f083de00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f083810f5e0 con 0x7f08381037c0 2024-04-25T01:13:11.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.676+0000 7f083d400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0838105370 0x7f08381a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:11.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.676+0000 7f083ca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08381037c0 0x7f0838199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:11.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.676+0000 7f083ca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08381037c0 0x7f0838199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:33652/0 (socket says 172.21.15.31:33652) 2024-04-25T01:13:11.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.676+0000 7f083d400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0838105370 0x7f08381a10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48194/0 (socket says 172.21.15.31:48194) 2024-04-25T01:13:11.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.676+0000 7f083ca00640 1 -- 172.21.15.31:0/4235320735 learned_addr learned my addr 172.21.15.31:0/4235320735 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:11.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.676+0000 7f082fe00640 1 --2- 172.21.15.31:0/4235320735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08381049b0 0x7f083819a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:11.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.676+0000 7f083d400640 1 -- 172.21.15.31:0/4235320735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08381037c0 msgr2=0x7f0838199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:11.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.676+0000 7f083d400640 1 --2- 172.21.15.31:0/4235320735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08381037c0 0x7f0838199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:11.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.676+0000 7f083d400640 1 -- 172.21.15.31:0/4235320735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08381049b0 msgr2=0x7f083819a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:11.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.676+0000 7f083d400640 1 --2- 172.21.15.31:0/4235320735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08381049b0 0x7f083819a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:11.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.676+0000 7f083d400640 1 -- 172.21.15.31:0/4235320735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08381a1730 con 0x7f0838105370 2024-04-25T01:13:11.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.677+0000 7f083d400640 1 --2- 172.21.15.31:0/4235320735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0838105370 0x7f08381a10c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f083000da10 tx=0x7f083000ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:11.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.677+0000 7f0836000640 1 -- 172.21.15.31:0/4235320735 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083000bd70 con 0x7f0838105370 2024-04-25T01:13:11.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.677+0000 7f0836000640 1 -- 172.21.15.31:0/4235320735 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0830003e50 con 0x7f0838105370 2024-04-25T01:13:11.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.677+0000 7f083de00640 1 -- 172.21.15.31:0/4235320735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08381a1a20 con 0x7f0838105370 2024-04-25T01:13:11.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.677+0000 7f0836000640 1 -- 172.21.15.31:0/4235320735 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08300106a0 con 0x7f0838105370 2024-04-25T01:13:11.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.677+0000 7f083de00640 1 -- 172.21.15.31:0/4235320735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08381a1f20 con 0x7f0838105370 2024-04-25T01:13:11.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.678+0000 7f083de00640 1 -- 172.21.15.31:0/4235320735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f083810aae0 con 0x7f0838105370 2024-04-25T01:13:11.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.680+0000 7f0836000640 1 -- 172.21.15.31:0/4235320735 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0830003970 con 0x7f0838105370 2024-04-25T01:13:11.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.682+0000 7f0836000640 1 --2- 172.21.15.31:0/4235320735 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f081c0a2b90 0x7f081c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:11.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.682+0000 7f0836000640 1 -- 172.21.15.31:0/4235320735 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f08300cefe0 con 0x7f0838105370 2024-04-25T01:13:11.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.682+0000 7f083ca00640 1 --2- 172.21.15.31:0/4235320735 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f081c0a2b90 0x7f081c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:11.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.683+0000 7f083ca00640 1 --2- 172.21.15.31:0/4235320735 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f081c0a2b90 0x7f081c0a5050 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0828000c00 tx=0x7f08280315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:11.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.687+0000 7f0836000640 1 -- 172.21.15.31:0/4235320735 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f083009c380 con 0x7f0838105370 2024-04-25T01:13:11.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.934+0000 7f083de00640 1 -- 172.21.15.31:0/4235320735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f083802d170 con 0x7f0838105370 2024-04-25T01:13:11.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.935+0000 7f0836000640 1 -- 172.21.15.31:0/4235320735 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7168 (secure 0 0 0) 0x7f083009bc80 con 0x7f0838105370 2024-04-25T01:13:11.923 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:13:11.923 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/498218714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":498218714},{"type":"v1","addr":"172.21.15.31:6835","nonce":498218714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/420362823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":420362823},{"type":"v1","addr":"172.21.15.169:6833","nonce":420362823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1943710438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1943710438},{"type":"v1","addr":"172.21.15.40:6833","nonce":1943710438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/4191344115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":4191344115},{"type":"v1","addr":"172.21.15.31:6837","nonce":4191344115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2916305801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2916305801},{"type":"v1","addr":"172.21.15.40:6835","nonce":2916305801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2253264784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2253264784},{"type":"v1","addr":"172.21.15.169:6835","nonce":2253264784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2883308641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2883308641},{"type":"v1","addr":"172.21.15.169:6837","nonce":2883308641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24548,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/590109562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":590109562},{"type":"v1","addr":"172.21.15.40:6837","nonce":590109562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T01:13:11.923 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 9 2024-04-25T01:13:11.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.938+0000 7f083de00640 1 -- 172.21.15.31:0/4235320735 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f081c0a2b90 msgr2=0x7f081c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:11.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.938+0000 7f083de00640 1 --2- 172.21.15.31:0/4235320735 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f081c0a2b90 0x7f081c0a5050 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0828000c00 tx=0x7f08280315b0 comp rx=0 tx=0).stop 2024-04-25T01:13:11.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.938+0000 7f083de00640 1 -- 172.21.15.31:0/4235320735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0838105370 msgr2=0x7f08381a10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:11.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.938+0000 7f083de00640 1 --2- 172.21.15.31:0/4235320735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0838105370 0x7f08381a10c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f083000da10 tx=0x7f083000ded0 comp rx=0 tx=0).stop 2024-04-25T01:13:11.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.938+0000 7f083de00640 1 -- 172.21.15.31:0/4235320735 shutdown_connections 2024-04-25T01:13:11.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.938+0000 7f083de00640 1 --2- 172.21.15.31:0/4235320735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0838105370 0x7f08381a10c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:11.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.938+0000 7f083de00640 1 --2- 172.21.15.31:0/4235320735 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f081c0a2b90 0x7f081c0a5050 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:11.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.938+0000 7f083de00640 1 --2- 172.21.15.31:0/4235320735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08381049b0 0x7f083819a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:11.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.938+0000 7f083de00640 1 --2- 172.21.15.31:0/4235320735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08381037c0 0x7f0838199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:11.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.938+0000 7f083de00640 1 -- 172.21.15.31:0/4235320735 >> 172.21.15.31:0/4235320735 conn(0x7f08380fed50 msgr2=0x7f0838107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:11.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.938+0000 7f083de00640 1 -- 172.21.15.31:0/4235320735 shutdown_connections 2024-04-25T01:13:11.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:11.939+0000 7f083de00640 1 -- 172.21.15.31:0/4235320735 wait complete. 2024-04-25T01:13:12.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:12 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/4235320735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:12.431 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:12 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/4235320735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:12.431 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 01:13:12 smithi169 systemd[1]: Starting Ceph mds.i for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:13:12.448 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:12 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/4235320735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:12.699 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 01:13:12 smithi169 podman[63537]: 2024-04-25 01:13:12.346033733 +0000 UTC m=+0.018479877 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:13:12.699 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 01:13:12 smithi169 podman[63537]: 2024-04-25 01:13:12.446968086 +0000 UTC m=+0.119414217 container create d04b8c94532444375d32c168575c9add64ab3f756e19337ff806455de59f3c55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:13:12.699 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 01:13:12 smithi169 podman[63537]: 2024-04-25 01:13:12.657567989 +0000 UTC m=+0.330014120 container init d04b8c94532444375d32c168575c9add64ab3f756e19337ff806455de59f3c55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T01:13:12.699 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 01:13:12 smithi169 podman[63537]: 2024-04-25 01:13:12.660031193 +0000 UTC m=+0.332477327 container start d04b8c94532444375d32c168575c9add64ab3f756e19337ff806455de59f3c55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T01:13:12.700 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 01:13:12 smithi169 ceph-mds[63556]: -- 172.21.15.169:0/1467346072 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cdefd63a40 con 0x55cdefd36800 2024-04-25T01:13:12.700 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 01:13:12 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-i[63552]: starting mds.i at 2024-04-25T01:13:13.196 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 01:13:12 smithi169 bash[63537]: d04b8c94532444375d32c168575c9add64ab3f756e19337ff806455de59f3c55 2024-04-25T01:13:13.196 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 01:13:12 smithi169 systemd[1]: Started Ceph mds.i for 95849b94-029f-11ef-bc93-c7b262605968. 2024-04-25T01:13:13.946 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:13:14.154 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:13 smithi031 ceph-mon[28053]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:14.155 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:14.155 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:14.155 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:14.155 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:13:14.155 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:13:14.155 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:14.155 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:13 smithi031 ceph-mon[28053]: Deploying daemon mds.g on smithi031 2024-04-25T01:13:14.155 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:13 smithi031 ceph-mon[28053]: mds.? [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] up:boot 2024-04-25T01:13:14.155 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:13 smithi031 ceph-mon[28053]: fsmap 9 up:standby 2024-04-25T01:13:14.155 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T01:13:14.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.195+0000 7effd3000640 1 -- 172.21.15.31:0/872045360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effcc105150 msgr2=0x7effcc1055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:14.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.195+0000 7effd3000640 1 --2- 172.21.15.31:0/872045360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effcc105150 0x7effcc1055d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7effc4009ca0 tx=0x7effc402a8c0 comp rx=0 tx=0).stop 2024-04-25T01:13:14.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.195+0000 7effd3000640 1 -- 172.21.15.31:0/872045360 shutdown_connections 2024-04-25T01:13:14.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.195+0000 7effd3000640 1 --2- 172.21.15.31:0/872045360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effcc105150 0x7effcc1055d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:14.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.195+0000 7effd3000640 1 --2- 172.21.15.31:0/872045360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effcc1027a0 0x7effcc104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:14.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.195+0000 7effd3000640 1 --2- 172.21.15.31:0/872045360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effcc0ffe30 0x7effcc102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:14.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.195+0000 7effd3000640 1 -- 172.21.15.31:0/872045360 >> 172.21.15.31:0/872045360 conn(0x7effcc0fba50 msgr2=0x7effcc0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:14.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.195+0000 7effd3000640 1 -- 172.21.15.31:0/872045360 shutdown_connections 2024-04-25T01:13:14.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.195+0000 7effd3000640 1 -- 172.21.15.31:0/872045360 wait complete. 2024-04-25T01:13:14.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.196+0000 7effd3000640 1 Processor -- start 2024-04-25T01:13:14.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.196+0000 7effd3000640 1 -- start start 2024-04-25T01:13:14.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.197+0000 7effd3000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effcc0ffe30 0x7effcc19dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:14.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.197+0000 7effd3000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effcc1027a0 0x7effcc19e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:14.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.197+0000 7effd3000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effcc105150 0x7effcc1a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:14.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.197+0000 7effd3000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7effcc113880 con 0x7effcc1027a0 2024-04-25T01:13:14.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.197+0000 7effd3000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7effcc113700 con 0x7effcc105150 2024-04-25T01:13:14.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.197+0000 7effd3000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7effcc113a00 con 0x7effcc0ffe30 2024-04-25T01:13:14.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.197+0000 7effd1200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effcc1027a0 0x7effcc19e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:14.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.197+0000 7effd1c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effcc0ffe30 0x7effcc19dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:14.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.197+0000 7effd1200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effcc1027a0 0x7effcc19e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48220/0 (socket says 172.21.15.31:48220) 2024-04-25T01:13:14.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.197+0000 7effd1200640 1 -- 172.21.15.31:0/3555009979 learned_addr learned my addr 172.21.15.31:0/3555009979 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:14.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.197+0000 7effd2600640 1 --2- 172.21.15.31:0/3555009979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effcc105150 0x7effcc1a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:14.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.198+0000 7effd1c00640 1 -- 172.21.15.31:0/3555009979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effcc105150 msgr2=0x7effcc1a5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:14.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.198+0000 7effd1c00640 1 --2- 172.21.15.31:0/3555009979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effcc105150 0x7effcc1a5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:14.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.198+0000 7effd1c00640 1 -- 172.21.15.31:0/3555009979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effcc1027a0 msgr2=0x7effcc19e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:14.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.198+0000 7effd1c00640 1 --2- 172.21.15.31:0/3555009979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effcc1027a0 0x7effcc19e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:14.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.198+0000 7effd1c00640 1 -- 172.21.15.31:0/3555009979 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effcc1a5a90 con 0x7effcc0ffe30 2024-04-25T01:13:14.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.198+0000 7effd2600640 1 --2- 172.21.15.31:0/3555009979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effcc105150 0x7effcc1a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:13:14.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.198+0000 7effd1200640 1 --2- 172.21.15.31:0/3555009979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effcc1027a0 0x7effcc19e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:13:14.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.198+0000 7effd1c00640 1 --2- 172.21.15.31:0/3555009979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effcc0ffe30 0x7effcc19dec0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7effc000e9f0 tx=0x7effc000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:14.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.198+0000 7effca000640 1 -- 172.21.15.31:0/3555009979 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc000ccc0 con 0x7effcc0ffe30 2024-04-25T01:13:14.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.198+0000 7effd3000640 1 -- 172.21.15.31:0/3555009979 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effcc1a5d80 con 0x7effcc0ffe30 2024-04-25T01:13:14.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.198+0000 7effca000640 1 -- 172.21.15.31:0/3555009979 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc0005020 con 0x7effcc0ffe30 2024-04-25T01:13:14.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.198+0000 7effca000640 1 -- 172.21.15.31:0/3555009979 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc00049e0 con 0x7effcc0ffe30 2024-04-25T01:13:14.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.198+0000 7effd3000640 1 -- 172.21.15.31:0/3555009979 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effcc1a61c0 con 0x7effcc0ffe30 2024-04-25T01:13:14.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.200+0000 7effd3000640 1 -- 172.21.15.31:0/3555009979 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effcc0fd540 con 0x7effcc0ffe30 2024-04-25T01:13:14.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.202+0000 7effca000640 1 -- 172.21.15.31:0/3555009979 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7effc0005560 con 0x7effcc0ffe30 2024-04-25T01:13:14.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.203+0000 7effca000640 1 --2- 172.21.15.31:0/3555009979 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7effb40a2b90 0x7effb40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:14.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.203+0000 7effca000640 1 -- 172.21.15.31:0/3555009979 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7effc00cec40 con 0x7effcc0ffe30 2024-04-25T01:13:14.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.203+0000 7effd1200640 1 --2- 172.21.15.31:0/3555009979 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7effb40a2b90 0x7effb40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:14.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.204+0000 7effd1200640 1 --2- 172.21.15.31:0/3555009979 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7effb40a2b90 0x7effb40a5050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7effcc19f380 tx=0x7effb8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:14.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.207+0000 7effca000640 1 -- 172.21.15.31:0/3555009979 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7effc00051d0 con 0x7effcc0ffe30 2024-04-25T01:13:14.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:13 smithi169 ceph-mon[32759]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:14.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:14.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:14.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:14.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:13:14.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:13:14.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:14.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:13 smithi169 ceph-mon[32759]: Deploying daemon mds.g on smithi031 2024-04-25T01:13:14.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:13 smithi169 ceph-mon[32759]: mds.? [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] up:boot 2024-04-25T01:13:14.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:13 smithi169 ceph-mon[32759]: fsmap 9 up:standby 2024-04-25T01:13:14.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T01:13:14.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:13 smithi040 ceph-mon[32518]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:14.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:14.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:14.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:14.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:13:14.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:13:14.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:14.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:13 smithi040 ceph-mon[32518]: Deploying daemon mds.g on smithi031 2024-04-25T01:13:14.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:13 smithi040 ceph-mon[32518]: mds.? [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] up:boot 2024-04-25T01:13:14.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:13 smithi040 ceph-mon[32518]: fsmap 9 up:standby 2024-04-25T01:13:14.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T01:13:14.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.461+0000 7effd3000640 1 -- 172.21.15.31:0/3555009979 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effcc02d530 con 0x7effcc0ffe30 2024-04-25T01:13:14.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.463+0000 7effca000640 1 -- 172.21.15.31:0/3555009979 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7998 (secure 0 0 0) 0x7effc009bfa0 con 0x7effcc0ffe30 2024-04-25T01:13:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:13:14.450 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/498218714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":498218714},{"type":"v1","addr":"172.21.15.31:6835","nonce":498218714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/420362823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":420362823},{"type":"v1","addr":"172.21.15.169:6833","nonce":420362823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1943710438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1943710438},{"type":"v1","addr":"172.21.15.40:6833","nonce":1943710438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/4191344115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":4191344115},{"type":"v1","addr":"172.21.15.31:6837","nonce":4191344115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2916305801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2916305801},{"type":"v1","addr":"172.21.15.40:6835","nonce":2916305801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2253264784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2253264784},{"type":"v1","addr":"172.21.15.169:6835","nonce":2253264784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2883308641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2883308641},{"type":"v1","addr":"172.21.15.169:6837","nonce":2883308641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24548,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/590109562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":590109562},{"type":"v1","addr":"172.21.15.40:6837","nonce":590109562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6839/4185710218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":4185710218},{"type":"v1","addr":"172.21.15.169:6839","nonce":4185710218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T01:13:14.451 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 10 2024-04-25T01:13:14.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.466+0000 7effd3000640 1 -- 172.21.15.31:0/3555009979 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7effb40a2b90 msgr2=0x7effb40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:14.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.466+0000 7effd3000640 1 --2- 172.21.15.31:0/3555009979 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7effb40a2b90 0x7effb40a5050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7effcc19f380 tx=0x7effb8005f90 comp rx=0 tx=0).stop 2024-04-25T01:13:14.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.466+0000 7effd3000640 1 -- 172.21.15.31:0/3555009979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effcc0ffe30 msgr2=0x7effcc19dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:14.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.466+0000 7effd3000640 1 --2- 172.21.15.31:0/3555009979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effcc0ffe30 0x7effcc19dec0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7effc000e9f0 tx=0x7effc000eeb0 comp rx=0 tx=0).stop 2024-04-25T01:13:14.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.466+0000 7effd3000640 1 -- 172.21.15.31:0/3555009979 shutdown_connections 2024-04-25T01:13:14.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.466+0000 7effd3000640 1 --2- 172.21.15.31:0/3555009979 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7effb40a2b90 0x7effb40a5050 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:14.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.466+0000 7effd3000640 1 --2- 172.21.15.31:0/3555009979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effcc105150 0x7effcc1a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:14.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.466+0000 7effd3000640 1 --2- 172.21.15.31:0/3555009979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effcc1027a0 0x7effcc19e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:14.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.466+0000 7effd3000640 1 --2- 172.21.15.31:0/3555009979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effcc0ffe30 0x7effcc19dec0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:14.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.466+0000 7effd3000640 1 -- 172.21.15.31:0/3555009979 >> 172.21.15.31:0/3555009979 conn(0x7effcc0fba50 msgr2=0x7effcc100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:14.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.466+0000 7effd3000640 1 -- 172.21.15.31:0/3555009979 shutdown_connections 2024-04-25T01:13:14.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:14.466+0000 7effd3000640 1 -- 172.21.15.31:0/3555009979 wait complete. 2024-04-25T01:13:15.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:14 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3555009979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:15.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:14 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3555009979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:15.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:14 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3555009979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:16.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:15 smithi169 ceph-mon[32759]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:16.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:15 smithi040 ceph-mon[32518]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:16.212 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:15 smithi031 ceph-mon[28053]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:16.474 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:13:16.506 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 01:13:16 smithi031 systemd[1]: Starting Ceph mds.g for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:13:16.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.680+0000 7f16a6c00640 1 -- 172.21.15.31:0/2603316171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16a01072a0 msgr2=0x7f16a01096f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:16.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.680+0000 7f16a6c00640 1 --2- 172.21.15.31:0/2603316171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16a01072a0 0x7f16a01096f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1698009900 tx=0x7f169802a3f0 comp rx=0 tx=0).stop 2024-04-25T01:13:16.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.680+0000 7f16a6c00640 1 -- 172.21.15.31:0/2603316171 shutdown_connections 2024-04-25T01:13:16.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.680+0000 7f16a6c00640 1 --2- 172.21.15.31:0/2603316171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16a01072a0 0x7f16a01096f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:16.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.680+0000 7f16a6c00640 1 --2- 172.21.15.31:0/2603316171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16a0104970 0x7f16a0106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:16.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.680+0000 7f16a6c00640 1 --2- 172.21.15.31:0/2603316171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a0102040 0x7f16a0104430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:16.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.680+0000 7f16a6c00640 1 -- 172.21.15.31:0/2603316171 >> 172.21.15.31:0/2603316171 conn(0x7f16a00fba10 msgr2=0x7f16a00fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:16.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.680+0000 7f16a6c00640 1 -- 172.21.15.31:0/2603316171 shutdown_connections 2024-04-25T01:13:16.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.680+0000 7f16a6c00640 1 -- 172.21.15.31:0/2603316171 wait complete. 2024-04-25T01:13:16.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.681+0000 7f16a6c00640 1 Processor -- start 2024-04-25T01:13:16.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.681+0000 7f16a6c00640 1 -- start start 2024-04-25T01:13:16.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.682+0000 7f16a6c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16a0102040 0x7f16a0199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:16.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.682+0000 7f16a6c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16a0104970 0x7f16a0199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:16.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.682+0000 7f16a6c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a01072a0 0x7f16a01a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:16.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.682+0000 7f16a6c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f16a010d2a0 con 0x7f16a0102040 2024-04-25T01:13:16.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.682+0000 7f16a6c00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f16a010d120 con 0x7f16a0104970 2024-04-25T01:13:16.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.682+0000 7f16a6c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f16a010d420 con 0x7f16a01072a0 2024-04-25T01:13:16.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.682+0000 7f16a5800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16a0102040 0x7f16a0199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:16.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.682+0000 7f16a5800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16a0102040 0x7f16a0199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48230/0 (socket says 172.21.15.31:48230) 2024-04-25T01:13:16.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.682+0000 7f16a5800640 1 -- 172.21.15.31:0/1401844199 learned_addr learned my addr 172.21.15.31:0/1401844199 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:16.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.682+0000 7f16a4e00640 1 --2- 172.21.15.31:0/1401844199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16a0104970 0x7f16a0199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:16.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.682+0000 7f16a6200640 1 --2- 172.21.15.31:0/1401844199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a01072a0 0x7f16a01a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:16.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.682+0000 7f16a5800640 1 -- 172.21.15.31:0/1401844199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a01072a0 msgr2=0x7f16a01a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:16.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.682+0000 7f16a5800640 1 --2- 172.21.15.31:0/1401844199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a01072a0 0x7f16a01a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:16.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.682+0000 7f16a5800640 1 -- 172.21.15.31:0/1401844199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16a0104970 msgr2=0x7f16a0199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:16.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.682+0000 7f16a5800640 1 --2- 172.21.15.31:0/1401844199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16a0104970 0x7f16a0199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:16.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.682+0000 7f16a5800640 1 -- 172.21.15.31:0/1401844199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a01a16d0 con 0x7f16a0102040 2024-04-25T01:13:16.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.683+0000 7f16a5800640 1 --2- 172.21.15.31:0/1401844199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16a0102040 0x7f16a0199ab0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f169400eb90 tx=0x7f169400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:16.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.683+0000 7f169e000640 1 -- 172.21.15.31:0/1401844199 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1694004d20 con 0x7f16a0102040 2024-04-25T01:13:16.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.683+0000 7f169e000640 1 -- 172.21.15.31:0/1401844199 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f169400ee90 con 0x7f16a0102040 2024-04-25T01:13:16.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.683+0000 7f169e000640 1 -- 172.21.15.31:0/1401844199 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1694010830 con 0x7f16a0102040 2024-04-25T01:13:16.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.683+0000 7f16a6c00640 1 -- 172.21.15.31:0/1401844199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a01a19c0 con 0x7f16a0102040 2024-04-25T01:13:16.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.683+0000 7f16a6c00640 1 -- 172.21.15.31:0/1401844199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a01a1ec0 con 0x7f16a0102040 2024-04-25T01:13:16.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.684+0000 7f16a6c00640 1 -- 172.21.15.31:0/1401844199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16a0108d90 con 0x7f16a0102040 2024-04-25T01:13:16.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.688+0000 7f169e000640 1 -- 172.21.15.31:0/1401844199 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1694005000 con 0x7f16a0102040 2024-04-25T01:13:16.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.689+0000 7f169e000640 1 --2- 172.21.15.31:0/1401844199 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f16880a2c10 0x7f16880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:16.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.690+0000 7f169e000640 1 -- 172.21.15.31:0/1401844199 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f16940cf3f0 con 0x7f16a0102040 2024-04-25T01:13:16.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.690+0000 7f16a4e00640 1 --2- 172.21.15.31:0/1401844199 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f16880a2c10 0x7f16880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:16.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.690+0000 7f16a4e00640 1 --2- 172.21.15.31:0/1401844199 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f16880a2c10 0x7f16880a50d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f168c0095f0 tx=0x7f168c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:16.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.692+0000 7f169e000640 1 -- 172.21.15.31:0/1401844199 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1694053740 con 0x7f16a0102040 2024-04-25T01:13:16.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.945+0000 7f16a6c00640 1 -- 172.21.15.31:0/1401844199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16a002d170 con 0x7f16a0102040 2024-04-25T01:13:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.946+0000 7f169e000640 1 -- 172.21.15.31:0/1401844199 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7998 (secure 0 0 0) 0x7f169409c7f0 con 0x7f16a0102040 2024-04-25T01:13:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:13:16.933 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/498218714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":498218714},{"type":"v1","addr":"172.21.15.31:6835","nonce":498218714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/420362823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":420362823},{"type":"v1","addr":"172.21.15.169:6833","nonce":420362823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1943710438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1943710438},{"type":"v1","addr":"172.21.15.40:6833","nonce":1943710438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/4191344115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":4191344115},{"type":"v1","addr":"172.21.15.31:6837","nonce":4191344115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2916305801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2916305801},{"type":"v1","addr":"172.21.15.40:6835","nonce":2916305801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2253264784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2253264784},{"type":"v1","addr":"172.21.15.169:6835","nonce":2253264784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2883308641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2883308641},{"type":"v1","addr":"172.21.15.169:6837","nonce":2883308641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24548,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/590109562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":590109562},{"type":"v1","addr":"172.21.15.40:6837","nonce":590109562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6839/4185710218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":4185710218},{"type":"v1","addr":"172.21.15.169:6839","nonce":4185710218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T01:13:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 10 2024-04-25T01:13:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.948+0000 7f16a6c00640 1 -- 172.21.15.31:0/1401844199 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f16880a2c10 msgr2=0x7f16880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.948+0000 7f16a6c00640 1 --2- 172.21.15.31:0/1401844199 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f16880a2c10 0x7f16880a50d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f168c0095f0 tx=0x7f168c005f90 comp rx=0 tx=0).stop 2024-04-25T01:13:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.948+0000 7f16a6c00640 1 -- 172.21.15.31:0/1401844199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16a0102040 msgr2=0x7f16a0199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.948+0000 7f16a6c00640 1 --2- 172.21.15.31:0/1401844199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16a0102040 0x7f16a0199ab0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f169400eb90 tx=0x7f169400c6a0 comp rx=0 tx=0).stop 2024-04-25T01:13:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.948+0000 7f16a6c00640 1 -- 172.21.15.31:0/1401844199 shutdown_connections 2024-04-25T01:13:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.948+0000 7f16a6c00640 1 --2- 172.21.15.31:0/1401844199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a01072a0 0x7f16a01a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.948+0000 7f16a6c00640 1 --2- 172.21.15.31:0/1401844199 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f16880a2c10 0x7f16880a50d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.948+0000 7f16a6c00640 1 --2- 172.21.15.31:0/1401844199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16a0104970 0x7f16a0199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.948+0000 7f16a6c00640 1 --2- 172.21.15.31:0/1401844199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16a0102040 0x7f16a0199ab0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.948+0000 7f16a6c00640 1 -- 172.21.15.31:0/1401844199 >> 172.21.15.31:0/1401844199 conn(0x7f16a00fba10 msgr2=0x7f16a0101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:16.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.949+0000 7f16a6c00640 1 -- 172.21.15.31:0/1401844199 shutdown_connections 2024-04-25T01:13:16.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:16.949+0000 7f16a6c00640 1 -- 172.21.15.31:0/1401844199 wait complete. 2024-04-25T01:13:16.953 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 01:13:16 smithi031 podman[69279]: 2024-04-25 01:13:16.506596262 +0000 UTC m=+0.041524552 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:13:16.953 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 01:13:16 smithi031 podman[69279]: 2024-04-25 01:13:16.723749913 +0000 UTC m=+0.258678247 container create 1ac2c36f4ed6af11793c288c16c85788ebaafc4a4ed9979db019e75147f2ef8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-25T01:13:17.256 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 01:13:17 smithi031 podman[69279]: 2024-04-25 01:13:17.059834956 +0000 UTC m=+0.594763271 container init 1ac2c36f4ed6af11793c288c16c85788ebaafc4a4ed9979db019e75147f2ef8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T01:13:17.256 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 01:13:17 smithi031 podman[69279]: 2024-04-25 01:13:17.070044691 +0000 UTC m=+0.604973033 container start 1ac2c36f4ed6af11793c288c16c85788ebaafc4a4ed9979db019e75147f2ef8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5) 2024-04-25T01:13:17.256 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 01:13:17 smithi031 ceph-mds[69338]: -- 172.21.15.31:0/3814672667 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5599e9565a40 con 0x5599e9538c00 2024-04-25T01:13:17.256 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 01:13:17 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-g[69332]: starting mds.g at 2024-04-25T01:13:17.256 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 01:13:17 smithi031 bash[69279]: 1ac2c36f4ed6af11793c288c16c85788ebaafc4a4ed9979db019e75147f2ef8f 2024-04-25T01:13:17.257 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 01:13:17 smithi031 systemd[1]: Started Ceph mds.g for 95849b94-029f-11ef-bc93-c7b262605968. 2024-04-25T01:13:18.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:17 smithi169 ceph-mon[32759]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:18.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:17 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1401844199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:18.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:18.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:18.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:18.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:13:18.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:13:18.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:17 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:18.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:17 smithi040 ceph-mon[32518]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:18.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:17 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1401844199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:18.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:18.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:18.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:18.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:13:18.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:13:18.199 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:17 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:18.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:17 smithi031 ceph-mon[28053]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:18.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:17 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1401844199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:18.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:18.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:18.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:18.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:13:18.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:13:18.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:17 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:18.957 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:13:19.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.195+0000 7f1feb200640 1 -- 172.21.15.31:0/3791744999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fec105370 msgr2=0x7f1fec10bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:19.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.195+0000 7f1feb200640 1 --2- 172.21.15.31:0/3791744999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fec105370 0x7f1fec10bc00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009ca0 tx=0x7f1fe402a8c0 comp rx=0 tx=0).stop 2024-04-25T01:13:19.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.195+0000 7f1feb200640 1 -- 172.21.15.31:0/3791744999 shutdown_connections 2024-04-25T01:13:19.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.195+0000 7f1feb200640 1 --2- 172.21.15.31:0/3791744999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fec105370 0x7f1fec10bc00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:19.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.195+0000 7f1feb200640 1 --2- 172.21.15.31:0/3791744999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fec1049b0 0x7f1fec104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:19.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.195+0000 7f1feb200640 1 --2- 172.21.15.31:0/3791744999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fec1037c0 0x7f1fec103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:19.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.195+0000 7f1feb200640 1 -- 172.21.15.31:0/3791744999 >> 172.21.15.31:0/3791744999 conn(0x7f1fec0fed50 msgr2=0x7f1fec101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:19.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.196+0000 7f1feb200640 1 -- 172.21.15.31:0/3791744999 shutdown_connections 2024-04-25T01:13:19.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.196+0000 7f1feb200640 1 -- 172.21.15.31:0/3791744999 wait complete. 2024-04-25T01:13:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.197+0000 7f1feb200640 1 Processor -- start 2024-04-25T01:13:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.197+0000 7f1feb200640 1 -- start start 2024-04-25T01:13:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.197+0000 7f1feb200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fec1037c0 0x7f1fec199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.197+0000 7f1feb200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fec1049b0 0x7f1fec199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.197+0000 7f1feb200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fec105370 0x7f1fec1a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.197+0000 7f1feb200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec10f490 con 0x7f1fec105370 2024-04-25T01:13:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.197+0000 7f1feb200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec10f310 con 0x7f1fec1049b0 2024-04-25T01:13:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.197+0000 7f1feb200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec10f610 con 0x7f1fec1037c0 2024-04-25T01:13:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.197+0000 7f1fea800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fec105370 0x7f1fec1a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.198+0000 7f1fe3e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fec1049b0 0x7f1fec199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.197+0000 7f1fe9e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fec1037c0 0x7f1fec199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.198+0000 7f1fea800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fec105370 0x7f1fec1a1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56540/0 (socket says 172.21.15.31:56540) 2024-04-25T01:13:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.198+0000 7f1fea800640 1 -- 172.21.15.31:0/3496918686 learned_addr learned my addr 172.21.15.31:0/3496918686 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.198+0000 7f1fe3e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fec1049b0 0x7f1fec199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:37384/0 (socket says 172.21.15.31:37384) 2024-04-25T01:13:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.198+0000 7f1fe9e00640 1 -- 172.21.15.31:0/3496918686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fec1049b0 msgr2=0x7f1fec199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.198+0000 7f1fe9e00640 1 --2- 172.21.15.31:0/3496918686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fec1049b0 0x7f1fec199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.198+0000 7f1fe9e00640 1 -- 172.21.15.31:0/3496918686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fec105370 msgr2=0x7f1fec1a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.198+0000 7f1fe9e00640 1 --2- 172.21.15.31:0/3496918686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fec105370 0x7f1fec1a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.198+0000 7f1fe9e00640 1 -- 172.21.15.31:0/3496918686 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fec1a16c0 con 0x7f1fec1037c0 2024-04-25T01:13:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.198+0000 7f1fe9e00640 1 --2- 172.21.15.31:0/3496918686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fec1037c0 0x7f1fec199aa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00e9c0 tx=0x7f1fdc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.198+0000 7f1fe2a00640 1 -- 172.21.15.31:0/3496918686 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc00ccc0 con 0x7f1fec1037c0 2024-04-25T01:13:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.199+0000 7f1fe2a00640 1 -- 172.21.15.31:0/3496918686 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fdc0035f0 con 0x7f1fec1037c0 2024-04-25T01:13:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.199+0000 7f1fe2a00640 1 -- 172.21.15.31:0/3496918686 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc00ce60 con 0x7f1fec1037c0 2024-04-25T01:13:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.199+0000 7f1feb200640 1 -- 172.21.15.31:0/3496918686 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fec1a19b0 con 0x7f1fec1037c0 2024-04-25T01:13:19.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.199+0000 7f1feb200640 1 -- 172.21.15.31:0/3496918686 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fec1a1eb0 con 0x7f1fec1037c0 2024-04-25T01:13:19.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.200+0000 7f1feb200640 1 -- 172.21.15.31:0/3496918686 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fec10aa80 con 0x7f1fec1037c0 2024-04-25T01:13:19.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.202+0000 7f1fe2a00640 1 -- 172.21.15.31:0/3496918686 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1fdc0037a0 con 0x7f1fec1037c0 2024-04-25T01:13:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.203+0000 7f1fe2a00640 1 --2- 172.21.15.31:0/3496918686 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1fd00a2c10 0x7f1fd00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.203+0000 7f1fe2a00640 1 -- 172.21.15.31:0/3496918686 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f1fdc0cef50 con 0x7f1fec1037c0 2024-04-25T01:13:19.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.204+0000 7f1fe3e00640 1 --2- 172.21.15.31:0/3496918686 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1fd00a2c10 0x7f1fd00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.204+0000 7f1fe3e00640 1 --2- 172.21.15.31:0/3496918686 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1fd00a2c10 0x7f1fd00a50d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1fec19afb0 tx=0x7f1fd4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:19.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:18 smithi169 ceph-mon[32759]: Deploying daemon mds.a on smithi031 2024-04-25T01:13:19.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:18 smithi169 ceph-mon[32759]: mds.? [v2:172.21.15.31:6838/2647199475,v1:172.21.15.31:6839/2647199475] up:boot 2024-04-25T01:13:19.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:18 smithi169 ceph-mon[32759]: fsmap 10 up:standby 2024-04-25T01:13:19.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:18 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T01:13:19.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:18 smithi040 ceph-mon[32518]: Deploying daemon mds.a on smithi031 2024-04-25T01:13:19.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:18 smithi040 ceph-mon[32518]: mds.? [v2:172.21.15.31:6838/2647199475,v1:172.21.15.31:6839/2647199475] up:boot 2024-04-25T01:13:19.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:18 smithi040 ceph-mon[32518]: fsmap 10 up:standby 2024-04-25T01:13:19.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:18 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T01:13:19.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.211+0000 7f1fe2a00640 1 -- 172.21.15.31:0/3496918686 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1fdc09c2b0 con 0x7f1fec1037c0 2024-04-25T01:13:19.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:18 smithi031 ceph-mon[28053]: Deploying daemon mds.a on smithi031 2024-04-25T01:13:19.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:18 smithi031 ceph-mon[28053]: mds.? [v2:172.21.15.31:6838/2647199475,v1:172.21.15.31:6839/2647199475] up:boot 2024-04-25T01:13:19.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:18 smithi031 ceph-mon[28053]: fsmap 10 up:standby 2024-04-25T01:13:19.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T01:13:19.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.479+0000 7f1feb200640 1 -- 172.21.15.31:0/3496918686 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1fec02d170 con 0x7f1fec1037c0 2024-04-25T01:13:19.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.480+0000 7f1fe2a00640 1 -- 172.21.15.31:0/3496918686 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8824 (secure 0 0 0) 0x7f1fdc09bbb0 con 0x7f1fec1037c0 2024-04-25T01:13:19.467 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:13:19.467 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/498218714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":498218714},{"type":"v1","addr":"172.21.15.31:6835","nonce":498218714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2647199475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2647199475},{"type":"v1","addr":"172.21.15.31:6839","nonce":2647199475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/420362823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":420362823},{"type":"v1","addr":"172.21.15.169:6833","nonce":420362823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1943710438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1943710438},{"type":"v1","addr":"172.21.15.40:6833","nonce":1943710438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/4191344115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":4191344115},{"type":"v1","addr":"172.21.15.31:6837","nonce":4191344115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2916305801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2916305801},{"type":"v1","addr":"172.21.15.40:6835","nonce":2916305801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2253264784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2253264784},{"type":"v1","addr":"172.21.15.169:6835","nonce":2253264784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2883308641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2883308641},{"type":"v1","addr":"172.21.15.169:6837","nonce":2883308641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24548,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/590109562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":590109562},{"type":"v1","addr":"172.21.15.40:6837","nonce":590109562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6839/4185710218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":4185710218},{"type":"v1","addr":"172.21.15.169:6839","nonce":4185710218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T01:13:19.468 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 11 2024-04-25T01:13:19.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.483+0000 7f1feb200640 1 -- 172.21.15.31:0/3496918686 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1fd00a2c10 msgr2=0x7f1fd00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:19.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.483+0000 7f1feb200640 1 --2- 172.21.15.31:0/3496918686 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1fd00a2c10 0x7f1fd00a50d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1fec19afb0 tx=0x7f1fd4005f90 comp rx=0 tx=0).stop 2024-04-25T01:13:19.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.483+0000 7f1feb200640 1 -- 172.21.15.31:0/3496918686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fec1037c0 msgr2=0x7f1fec199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:19.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.483+0000 7f1feb200640 1 --2- 172.21.15.31:0/3496918686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fec1037c0 0x7f1fec199aa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00e9c0 tx=0x7f1fdc00ee80 comp rx=0 tx=0).stop 2024-04-25T01:13:19.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.483+0000 7f1feb200640 1 -- 172.21.15.31:0/3496918686 shutdown_connections 2024-04-25T01:13:19.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.483+0000 7f1feb200640 1 --2- 172.21.15.31:0/3496918686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fec105370 0x7f1fec1a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:19.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.483+0000 7f1feb200640 1 --2- 172.21.15.31:0/3496918686 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1fd00a2c10 0x7f1fd00a50d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:19.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.483+0000 7f1feb200640 1 --2- 172.21.15.31:0/3496918686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fec1049b0 0x7f1fec199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:19.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.483+0000 7f1feb200640 1 --2- 172.21.15.31:0/3496918686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fec1037c0 0x7f1fec199aa0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:19.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.483+0000 7f1feb200640 1 -- 172.21.15.31:0/3496918686 >> 172.21.15.31:0/3496918686 conn(0x7f1fec0fed50 msgr2=0x7f1fec107c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:19.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.483+0000 7f1feb200640 1 -- 172.21.15.31:0/3496918686 shutdown_connections 2024-04-25T01:13:19.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:19.483+0000 7f1feb200640 1 -- 172.21.15.31:0/3496918686 wait complete. 2024-04-25T01:13:20.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:19 smithi169 ceph-mon[32759]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:20.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:19 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3496918686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:20.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:19 smithi040 ceph-mon[32518]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:20.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:19 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3496918686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:20.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:19 smithi031 ceph-mon[28053]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:20.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:19 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3496918686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:21.006 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 01:13:20 smithi031 systemd[1]: Starting Ceph mds.a for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:13:21.319 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 01:13:21 smithi031 podman[69663]: 2024-04-25 01:13:20.931366021 +0000 UTC m=+0.041479775 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:13:21.319 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 01:13:21 smithi031 podman[69663]: 2024-04-25 01:13:21.033871823 +0000 UTC m=+0.143985620 container create 296222b64e937a87c4c6e55e737ea91d17e65b8ac07b45a9088d37cd85092f33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T01:13:21.319 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 01:13:21 smithi031 podman[69663]: 2024-04-25 01:13:21.331620003 +0000 UTC m=+0.441733777 container init 296222b64e937a87c4c6e55e737ea91d17e65b8ac07b45a9088d37cd85092f33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-25T01:13:21.491 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:13:21.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.717+0000 7fa323200640 1 -- 172.21.15.31:0/2029323910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa324103190 msgr2=0x7fa324109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.717+0000 7fa323200640 1 --2- 172.21.15.31:0/2029323910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa324103190 0x7fa324109a20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009900 tx=0x7fa31c02a3f0 comp rx=0 tx=0).stop 2024-04-25T01:13:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.717+0000 7fa323200640 1 -- 172.21.15.31:0/2029323910 shutdown_connections 2024-04-25T01:13:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.717+0000 7fa323200640 1 --2- 172.21.15.31:0/2029323910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa324103190 0x7fa324109a20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.717+0000 7fa323200640 1 --2- 172.21.15.31:0/2029323910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3241027d0 0x7fa324102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.717+0000 7fa323200640 1 --2- 172.21.15.31:0/2029323910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3241015e0 0x7fa324101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.717+0000 7fa323200640 1 -- 172.21.15.31:0/2029323910 >> 172.21.15.31:0/2029323910 conn(0x7fa3240fcb50 msgr2=0x7fa3240fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.717+0000 7fa323200640 1 -- 172.21.15.31:0/2029323910 shutdown_connections 2024-04-25T01:13:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.717+0000 7fa323200640 1 -- 172.21.15.31:0/2029323910 wait complete. 2024-04-25T01:13:21.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.719+0000 7fa323200640 1 Processor -- start 2024-04-25T01:13:21.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.720+0000 7fa323200640 1 -- start start 2024-04-25T01:13:21.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.720+0000 7fa323200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3241015e0 0x7fa3240754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:21.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.720+0000 7fa323200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3241027d0 0x7fa324073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:21.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.720+0000 7fa323200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa324103190 0x7fa324074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:21.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.720+0000 7fa323200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa32410d2b0 con 0x7fa3241015e0 2024-04-25T01:13:21.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.720+0000 7fa323200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa32410d130 con 0x7fa324103190 2024-04-25T01:13:21.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.720+0000 7fa323200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa32410d430 con 0x7fa3241027d0 2024-04-25T01:13:21.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.720+0000 7fa321e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3241015e0 0x7fa3240754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:21.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.721+0000 7fa322800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa324103190 0x7fa324074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:21.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.721+0000 7fa322800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa324103190 0x7fa324074070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:37426/0 (socket says 172.21.15.31:37426) 2024-04-25T01:13:21.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.721+0000 7fa322800640 1 -- 172.21.15.31:0/3407667093 learned_addr learned my addr 172.21.15.31:0/3407667093 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:21.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.721+0000 7fa321e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3241015e0 0x7fa3240754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56566/0 (socket says 172.21.15.31:56566) 2024-04-25T01:13:21.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.721+0000 7fa321400640 1 --2- 172.21.15.31:0/3407667093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3241027d0 0x7fa324073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:21.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.721+0000 7fa322800640 1 -- 172.21.15.31:0/3407667093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3241027d0 msgr2=0x7fa324073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:21.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.721+0000 7fa322800640 1 --2- 172.21.15.31:0/3407667093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3241027d0 0x7fa324073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:21.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.721+0000 7fa322800640 1 -- 172.21.15.31:0/3407667093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3241015e0 msgr2=0x7fa3240754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:21.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.721+0000 7fa322800640 1 --2- 172.21.15.31:0/3407667093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3241015e0 0x7fa3240754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:21.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.721+0000 7fa322800640 1 -- 172.21.15.31:0/3407667093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa324074900 con 0x7fa324103190 2024-04-25T01:13:21.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.721+0000 7fa322800640 1 --2- 172.21.15.31:0/3407667093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa324103190 0x7fa324074070 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa31c02a910 tx=0x7fa31c032c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:21.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.722+0000 7fa31aa00640 1 -- 172.21.15.31:0/3407667093 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c03f030 con 0x7fa324103190 2024-04-25T01:13:21.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.722+0000 7fa323200640 1 -- 172.21.15.31:0/3407667093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3241a1950 con 0x7fa324103190 2024-04-25T01:13:21.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.722+0000 7fa31aa00640 1 -- 172.21.15.31:0/3407667093 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa31c032e50 con 0x7fa324103190 2024-04-25T01:13:21.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.722+0000 7fa31aa00640 1 -- 172.21.15.31:0/3407667093 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c035070 con 0x7fa324103190 2024-04-25T01:13:21.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.722+0000 7fa323200640 1 -- 172.21.15.31:0/3407667093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3241a1ea0 con 0x7fa324103190 2024-04-25T01:13:21.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.723+0000 7fa323200640 1 -- 172.21.15.31:0/3407667093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa324068480 con 0x7fa324103190 2024-04-25T01:13:21.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.725+0000 7fa31aa00640 1 -- 172.21.15.31:0/3407667093 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa31c042030 con 0x7fa324103190 2024-04-25T01:13:21.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.726+0000 7fa31aa00640 1 --2- 172.21.15.31:0/3407667093 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa3080a2b90 0x7fa3080a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:21.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.727+0000 7fa31aa00640 1 -- 172.21.15.31:0/3407667093 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fa31c0eefd0 con 0x7fa324103190 2024-04-25T01:13:21.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.727+0000 7fa321e00640 1 --2- 172.21.15.31:0/3407667093 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa3080a2b90 0x7fa3080a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:21.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.727+0000 7fa321e00640 1 --2- 172.21.15.31:0/3407667093 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa3080a2b90 0x7fa3080a5050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa324102630 tx=0x7fa314009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:21.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.731+0000 7fa31aa00640 1 -- 172.21.15.31:0/3407667093 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa31c0bbce0 con 0x7fa324103190 2024-04-25T01:13:21.756 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 01:13:21 smithi031 podman[69663]: 2024-04-25 01:13:21.343329348 +0000 UTC m=+0.453443084 container start 296222b64e937a87c4c6e55e737ea91d17e65b8ac07b45a9088d37cd85092f33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T01:13:21.757 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 01:13:21 smithi031 ceph-mds[69681]: -- 172.21.15.31:0/1259817809 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557feee2fa40 con 0x557feee03000 2024-04-25T01:13:21.757 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 01:13:21 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-a[69677]: starting mds.a at 2024-04-25T01:13:21.757 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 01:13:21 smithi031 bash[69663]: 296222b64e937a87c4c6e55e737ea91d17e65b8ac07b45a9088d37cd85092f33 2024-04-25T01:13:21.757 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 01:13:21 smithi031 systemd[1]: Started Ceph mds.a for 95849b94-029f-11ef-bc93-c7b262605968. 2024-04-25T01:13:21.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.979+0000 7fa323200640 1 -- 172.21.15.31:0/3407667093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa324062340 con 0x7fa324103190 2024-04-25T01:13:21.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.980+0000 7fa31aa00640 1 -- 172.21.15.31:0/3407667093 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9644 (secure 0 0 0) 0x7fa31c0bffe0 con 0x7fa324103190 2024-04-25T01:13:21.967 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:13:21.967 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/498218714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":498218714},{"type":"v1","addr":"172.21.15.31:6835","nonce":498218714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2647199475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2647199475},{"type":"v1","addr":"172.21.15.31:6839","nonce":2647199475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/30518311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":30518311},{"type":"v1","addr":"172.21.15.31:6841","nonce":30518311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/420362823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":420362823},{"type":"v1","addr":"172.21.15.169:6833","nonce":420362823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1943710438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1943710438},{"type":"v1","addr":"172.21.15.40:6833","nonce":1943710438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/4191344115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":4191344115},{"type":"v1","addr":"172.21.15.31:6837","nonce":4191344115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2916305801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2916305801},{"type":"v1","addr":"172.21.15.40:6835","nonce":2916305801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2253264784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2253264784},{"type":"v1","addr":"172.21.15.169:6835","nonce":2253264784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2883308641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2883308641},{"type":"v1","addr":"172.21.15.169:6837","nonce":2883308641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24548,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/590109562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":590109562},{"type":"v1","addr":"172.21.15.40:6837","nonce":590109562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6839/4185710218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":4185710218},{"type":"v1","addr":"172.21.15.169:6839","nonce":4185710218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T01:13:21.968 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 12 2024-04-25T01:13:21.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.982+0000 7fa323200640 1 -- 172.21.15.31:0/3407667093 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa3080a2b90 msgr2=0x7fa3080a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:21.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.982+0000 7fa323200640 1 --2- 172.21.15.31:0/3407667093 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa3080a2b90 0x7fa3080a5050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa324102630 tx=0x7fa314009500 comp rx=0 tx=0).stop 2024-04-25T01:13:21.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.983+0000 7fa323200640 1 -- 172.21.15.31:0/3407667093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa324103190 msgr2=0x7fa324074070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:21.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.983+0000 7fa323200640 1 --2- 172.21.15.31:0/3407667093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa324103190 0x7fa324074070 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa31c02a910 tx=0x7fa31c032c60 comp rx=0 tx=0).stop 2024-04-25T01:13:21.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.983+0000 7fa323200640 1 -- 172.21.15.31:0/3407667093 shutdown_connections 2024-04-25T01:13:21.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.983+0000 7fa323200640 1 --2- 172.21.15.31:0/3407667093 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa3080a2b90 0x7fa3080a5050 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:21.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.983+0000 7fa323200640 1 --2- 172.21.15.31:0/3407667093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa324103190 0x7fa324074070 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:21.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.983+0000 7fa323200640 1 --2- 172.21.15.31:0/3407667093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3241027d0 0x7fa324073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:21.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.983+0000 7fa323200640 1 --2- 172.21.15.31:0/3407667093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3241015e0 0x7fa3240754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:21.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.983+0000 7fa323200640 1 -- 172.21.15.31:0/3407667093 >> 172.21.15.31:0/3407667093 conn(0x7fa3240fcb50 msgr2=0x7fa324105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:21.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.983+0000 7fa323200640 1 -- 172.21.15.31:0/3407667093 shutdown_connections 2024-04-25T01:13:21.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:21.983+0000 7fa323200640 1 -- 172.21.15.31:0/3407667093 wait complete. 2024-04-25T01:13:22.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:21 smithi169 ceph-mon[32759]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:22.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:21 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:22.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:21 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:22.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:21 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:22.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:21 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:13:22.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:21 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:13:22.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:21 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:22.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:21 smithi040 ceph-mon[32518]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:22.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:21 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:22.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:21 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:22.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:21 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:22.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:21 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:13:22.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:21 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:13:22.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:21 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:22.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:21 smithi031 ceph-mon[28053]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:22.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:21 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:22.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:21 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:22.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:21 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:22.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:21 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:13:22.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:21 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:13:22.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:21 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:23.195 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:22 smithi040 ceph-mon[32518]: Deploying daemon mds.b on smithi040 2024-04-25T01:13:23.195 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:22 smithi040 ceph-mon[32518]: mds.? [v2:172.21.15.31:6840/30518311,v1:172.21.15.31:6841/30518311] up:boot 2024-04-25T01:13:23.195 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:22 smithi040 ceph-mon[32518]: fsmap 11 up:standby 2024-04-25T01:13:23.195 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:22 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T01:13:23.195 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:22 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3407667093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:23.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:22 smithi169 ceph-mon[32759]: Deploying daemon mds.b on smithi040 2024-04-25T01:13:23.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:22 smithi169 ceph-mon[32759]: mds.? [v2:172.21.15.31:6840/30518311,v1:172.21.15.31:6841/30518311] up:boot 2024-04-25T01:13:23.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:22 smithi169 ceph-mon[32759]: fsmap 11 up:standby 2024-04-25T01:13:23.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:22 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T01:13:23.196 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:22 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3407667093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:23.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:22 smithi031 ceph-mon[28053]: Deploying daemon mds.b on smithi040 2024-04-25T01:13:23.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:22 smithi031 ceph-mon[28053]: mds.? [v2:172.21.15.31:6840/30518311,v1:172.21.15.31:6841/30518311] up:boot 2024-04-25T01:13:23.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:22 smithi031 ceph-mon[28053]: fsmap 11 up:standby 2024-04-25T01:13:23.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:22 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T01:13:23.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:22 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3407667093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:23.868 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 01:13:23 smithi040 systemd[1]: Starting Ceph mds.b for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:13:23.991 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:13:24.133 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:23 smithi040 ceph-mon[32518]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:24.134 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 01:13:23 smithi040 podman[63211]: 2024-04-25 01:13:23.827404942 +0000 UTC m=+0.018599113 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:13:24.134 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 01:13:23 smithi040 podman[63211]: 2024-04-25 01:13:23.94859796 +0000 UTC m=+0.139792130 container create 2d7279fa8895ce38cb77ccf3f7c158f1c010d6ef6520c87d86911d02468fda6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-b, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-04-25T01:13:24.134 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 01:13:24 smithi040 podman[63211]: 2024-04-25 01:13:24.147885091 +0000 UTC m=+0.339079262 container init 2d7279fa8895ce38cb77ccf3f7c158f1c010d6ef6520c87d86911d02468fda6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T01:13:24.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:23 smithi169 ceph-mon[32759]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:24.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.229+0000 7f51a3200640 1 -- 172.21.15.31:0/3836262509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51a4105150 msgr2=0x7f51a41055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:24.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.229+0000 7f51a3200640 1 --2- 172.21.15.31:0/3836262509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51a4105150 0x7f51a41055d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f519c009ca0 tx=0x7f519c02a8c0 comp rx=0 tx=0).stop 2024-04-25T01:13:24.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.229+0000 7f51a3200640 1 -- 172.21.15.31:0/3836262509 shutdown_connections 2024-04-25T01:13:24.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.229+0000 7f51a3200640 1 --2- 172.21.15.31:0/3836262509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51a4105150 0x7f51a41055d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:24.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.229+0000 7f51a3200640 1 --2- 172.21.15.31:0/3836262509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a41027a0 0x7f51a4104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:24.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.229+0000 7f51a3200640 1 --2- 172.21.15.31:0/3836262509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51a40ffe30 0x7f51a4102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:24.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.229+0000 7f51a3200640 1 -- 172.21.15.31:0/3836262509 >> 172.21.15.31:0/3836262509 conn(0x7f51a40fba10 msgr2=0x7f51a40fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:24.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.229+0000 7f51a3200640 1 -- 172.21.15.31:0/3836262509 shutdown_connections 2024-04-25T01:13:24.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.229+0000 7f51a3200640 1 -- 172.21.15.31:0/3836262509 wait complete. 2024-04-25T01:13:24.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.230+0000 7f51a3200640 1 Processor -- start 2024-04-25T01:13:24.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.230+0000 7f51a3200640 1 -- start start 2024-04-25T01:13:24.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.231+0000 7f51a3200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51a40ffe30 0x7f51a419de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:24.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.231+0000 7f51a3200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a41027a0 0x7f51a419e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:24.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.231+0000 7f51a3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51a4105150 0x7f51a41a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:24.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.231+0000 7f51a3200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4113860 con 0x7f51a41027a0 2024-04-25T01:13:24.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.231+0000 7f51a3200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f51a41136e0 con 0x7f51a40ffe30 2024-04-25T01:13:24.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.231+0000 7f51a3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f51a41139e0 con 0x7f51a4105150 2024-04-25T01:13:24.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.231+0000 7f51a1e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51a40ffe30 0x7f51a419de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:24.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.231+0000 7f5199400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a41027a0 0x7f51a419e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:24.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.231+0000 7f51a1e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51a40ffe30 0x7f51a419de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:37442/0 (socket says 172.21.15.31:37442) 2024-04-25T01:13:24.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.231+0000 7f51a1e00640 1 -- 172.21.15.31:0/1773849437 learned_addr learned my addr 172.21.15.31:0/1773849437 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:24.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.231+0000 7f51a2800640 1 --2- 172.21.15.31:0/1773849437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51a4105150 0x7f51a41a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:24.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.231+0000 7f51a2800640 1 -- 172.21.15.31:0/1773849437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51a40ffe30 msgr2=0x7f51a419de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:24.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.231+0000 7f51a2800640 1 --2- 172.21.15.31:0/1773849437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51a40ffe30 0x7f51a419de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:24.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.231+0000 7f51a2800640 1 -- 172.21.15.31:0/1773849437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a41027a0 msgr2=0x7f51a419e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:24.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.231+0000 7f51a2800640 1 --2- 172.21.15.31:0/1773849437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a41027a0 0x7f51a419e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:24.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.231+0000 7f51a2800640 1 -- 172.21.15.31:0/1773849437 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a41a5a60 con 0x7f51a4105150 2024-04-25T01:13:24.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.231+0000 7f51a1e00640 1 --2- 172.21.15.31:0/1773849437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51a40ffe30 0x7f51a419de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:13:24.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.231+0000 7f5199400640 1 --2- 172.21.15.31:0/1773849437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a41027a0 0x7f51a419e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:13:24.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.232+0000 7f51a2800640 1 --2- 172.21.15.31:0/1773849437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51a4105150 0x7f51a41a53f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f519c031ae0 tx=0x7f519c031750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:24.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.232+0000 7f519b400640 1 -- 172.21.15.31:0/1773849437 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c003b70 con 0x7f51a4105150 2024-04-25T01:13:24.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.232+0000 7f51a3200640 1 -- 172.21.15.31:0/1773849437 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a41a5cf0 con 0x7f51a4105150 2024-04-25T01:13:24.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.232+0000 7f519b400640 1 -- 172.21.15.31:0/1773849437 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f519c003d10 con 0x7f51a4105150 2024-04-25T01:13:24.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.232+0000 7f519b400640 1 -- 172.21.15.31:0/1773849437 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c02fb10 con 0x7f51a4105150 2024-04-25T01:13:24.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.232+0000 7f51a3200640 1 -- 172.21.15.31:0/1773849437 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a41a6230 con 0x7f51a4105150 2024-04-25T01:13:24.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.233+0000 7f51a3200640 1 -- 172.21.15.31:0/1773849437 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a40fd500 con 0x7f51a4105150 2024-04-25T01:13:24.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.235+0000 7f519b400640 1 -- 172.21.15.31:0/1773849437 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f519c004640 con 0x7f51a4105150 2024-04-25T01:13:24.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.237+0000 7f519b400640 1 --2- 172.21.15.31:0/1773849437 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f51880a2b90 0x7f51880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:24.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.237+0000 7f519b400640 1 -- 172.21.15.31:0/1773849437 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f519c0eff30 con 0x7f51a4105150 2024-04-25T01:13:24.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.237+0000 7f51a1e00640 1 --2- 172.21.15.31:0/1773849437 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f51880a2b90 0x7f51880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:24.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.238+0000 7f51a1e00640 1 --2- 172.21.15.31:0/1773849437 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f51880a2b90 0x7f51880a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5194011550 tx=0x7f5194005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:24.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.241+0000 7f519b400640 1 -- 172.21.15.31:0/1773849437 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f519c0bd290 con 0x7f51a4105150 2024-04-25T01:13:24.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:23 smithi031 ceph-mon[28053]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:24.451 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 01:13:24 smithi040 podman[63211]: 2024-04-25 01:13:24.150789918 +0000 UTC m=+0.341984088 container start 2d7279fa8895ce38cb77ccf3f7c158f1c010d6ef6520c87d86911d02468fda6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9) 2024-04-25T01:13:24.452 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 01:13:24 smithi040 ceph-mds[63230]: -- 172.21.15.40:0/2936595234 <== mon.2 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556e8c233a40 con 0x556e8c206800 2024-04-25T01:13:24.452 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 01:13:24 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-b[63226]: starting mds.b at 2024-04-25T01:13:24.452 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 01:13:24 smithi040 bash[63211]: 2d7279fa8895ce38cb77ccf3f7c158f1c010d6ef6520c87d86911d02468fda6e 2024-04-25T01:13:24.452 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 01:13:24 smithi040 systemd[1]: Started Ceph mds.b for 95849b94-029f-11ef-bc93-c7b262605968. 2024-04-25T01:13:24.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.490+0000 7f51a3200640 1 -- 172.21.15.31:0/1773849437 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51a402d170 con 0x7f51a4105150 2024-04-25T01:13:24.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.491+0000 7f519b400640 1 -- 172.21.15.31:0/1773849437 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9644 (secure 0 0 0) 0x7f519c0bcb90 con 0x7f51a4105150 2024-04-25T01:13:24.478 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:13:24.478 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/498218714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":498218714},{"type":"v1","addr":"172.21.15.31:6835","nonce":498218714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2647199475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2647199475},{"type":"v1","addr":"172.21.15.31:6839","nonce":2647199475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/30518311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":30518311},{"type":"v1","addr":"172.21.15.31:6841","nonce":30518311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/420362823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":420362823},{"type":"v1","addr":"172.21.15.169:6833","nonce":420362823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1943710438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1943710438},{"type":"v1","addr":"172.21.15.40:6833","nonce":1943710438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/4191344115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":4191344115},{"type":"v1","addr":"172.21.15.31:6837","nonce":4191344115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2916305801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2916305801},{"type":"v1","addr":"172.21.15.40:6835","nonce":2916305801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2253264784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2253264784},{"type":"v1","addr":"172.21.15.169:6835","nonce":2253264784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2883308641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2883308641},{"type":"v1","addr":"172.21.15.169:6837","nonce":2883308641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24548,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/590109562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":590109562},{"type":"v1","addr":"172.21.15.40:6837","nonce":590109562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6839/4185710218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":4185710218},{"type":"v1","addr":"172.21.15.169:6839","nonce":4185710218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T01:13:24.478 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 12 2024-04-25T01:13:24.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.493+0000 7f51a3200640 1 -- 172.21.15.31:0/1773849437 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f51880a2b90 msgr2=0x7f51880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:24.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.493+0000 7f51a3200640 1 --2- 172.21.15.31:0/1773849437 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f51880a2b90 0x7f51880a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5194011550 tx=0x7f5194005f90 comp rx=0 tx=0).stop 2024-04-25T01:13:24.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.493+0000 7f51a3200640 1 -- 172.21.15.31:0/1773849437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51a4105150 msgr2=0x7f51a41a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:24.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.493+0000 7f51a3200640 1 --2- 172.21.15.31:0/1773849437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51a4105150 0x7f51a41a53f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f519c031ae0 tx=0x7f519c031750 comp rx=0 tx=0).stop 2024-04-25T01:13:24.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.494+0000 7f51a3200640 1 -- 172.21.15.31:0/1773849437 shutdown_connections 2024-04-25T01:13:24.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.494+0000 7f51a3200640 1 --2- 172.21.15.31:0/1773849437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51a4105150 0x7f51a41a53f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:24.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.494+0000 7f51a3200640 1 --2- 172.21.15.31:0/1773849437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a41027a0 0x7f51a419e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:24.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.494+0000 7f51a3200640 1 --2- 172.21.15.31:0/1773849437 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f51880a2b90 0x7f51880a5050 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:24.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.494+0000 7f51a3200640 1 --2- 172.21.15.31:0/1773849437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51a40ffe30 0x7f51a419de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:24.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.494+0000 7f51a3200640 1 -- 172.21.15.31:0/1773849437 >> 172.21.15.31:0/1773849437 conn(0x7f51a40fba10 msgr2=0x7f51a41007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:24.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.494+0000 7f51a3200640 1 -- 172.21.15.31:0/1773849437 shutdown_connections 2024-04-25T01:13:24.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:24.494+0000 7f51a3200640 1 -- 172.21.15.31:0/1773849437 wait complete. 2024-04-25T01:13:25.681 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:25 smithi040 ceph-mon[32518]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:25.681 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:25.681 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:25.681 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:25.681 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:25.681 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:25.681 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:25 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1773849437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:25.681 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:13:25.681 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:25 smithi040 ceph-mon[32518]: mds.? [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] up:boot 2024-04-25T01:13:25.681 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:25 smithi040 ceph-mon[32518]: fsmap 12 up:standby 2024-04-25T01:13:25.681 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T01:13:25.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:25 smithi169 ceph-mon[32759]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:25.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:25.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:25.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:25.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:25.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:25.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:25 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1773849437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:25.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:13:25.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:25 smithi169 ceph-mon[32759]: mds.? [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] up:boot 2024-04-25T01:13:25.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:25 smithi169 ceph-mon[32759]: fsmap 12 up:standby 2024-04-25T01:13:25.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T01:13:25.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:25 smithi031 ceph-mon[28053]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:25.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:25.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:25.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:25.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:25.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:25.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:25 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1773849437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:25.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:13:25.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:25 smithi031 ceph-mon[28053]: mds.? [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] up:boot 2024-04-25T01:13:25.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:25 smithi031 ceph-mon[28053]: fsmap 12 up:standby 2024-04-25T01:13:25.757 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T01:13:26.501 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:13:26.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.707+0000 7f0c68400640 1 -- 172.21.15.31:0/3547778478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c60104b90 msgr2=0x7f0c60105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:26.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.707+0000 7f0c68400640 1 --2- 172.21.15.31:0/3547778478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c60104b90 0x7f0c60105010 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009900 tx=0x7f0c4c02a3f0 comp rx=0 tx=0).stop 2024-04-25T01:13:26.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.707+0000 7f0c68400640 1 -- 172.21.15.31:0/3547778478 shutdown_connections 2024-04-25T01:13:26.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.707+0000 7f0c68400640 1 --2- 172.21.15.31:0/3547778478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c60105550 0x7f0c6010bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:26.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.707+0000 7f0c68400640 1 --2- 172.21.15.31:0/3547778478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c60104b90 0x7f0c60105010 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009900 tx=0x7f0c4c02a3f0 comp rx=0 tx=0).stop 2024-04-25T01:13:26.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.707+0000 7f0c68400640 1 --2- 172.21.15.31:0/3547778478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c601039a0 0x7f0c60103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:26.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.707+0000 7f0c68400640 1 -- 172.21.15.31:0/3547778478 >> 172.21.15.31:0/3547778478 conn(0x7f0c600fef30 msgr2=0x7f0c60101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:26.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.708+0000 7f0c68400640 1 -- 172.21.15.31:0/3547778478 shutdown_connections 2024-04-25T01:13:26.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.708+0000 7f0c68400640 1 -- 172.21.15.31:0/3547778478 wait complete. 2024-04-25T01:13:26.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.709+0000 7f0c68400640 1 Processor -- start 2024-04-25T01:13:26.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.709+0000 7f0c68400640 1 -- start start 2024-04-25T01:13:26.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.709+0000 7f0c68400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c601039a0 0x7f0c60199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:26.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.709+0000 7f0c68400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c60105550 0x7f0c6019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:26.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.709+0000 7f0c68400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c6019f430 0x7f0c601a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:26.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.709+0000 7f0c68400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6010f620 con 0x7f0c601039a0 2024-04-25T01:13:26.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.709+0000 7f0c68400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6010f4a0 con 0x7f0c60105550 2024-04-25T01:13:26.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.709+0000 7f0c68400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6010f7a0 con 0x7f0c6019f430 2024-04-25T01:13:26.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.709+0000 7f0c65000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c60105550 0x7f0c6019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:26.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.710+0000 7f0c65000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c60105550 0x7f0c6019a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:37454/0 (socket says 172.21.15.31:37454) 2024-04-25T01:13:26.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.710+0000 7f0c65000640 1 -- 172.21.15.31:0/1838481043 learned_addr learned my addr 172.21.15.31:0/1838481043 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:26.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.710+0000 7f0c65a00640 1 --2- 172.21.15.31:0/1838481043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c601039a0 0x7f0c60199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:26.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.710+0000 7f0c66400640 1 --2- 172.21.15.31:0/1838481043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c6019f430 0x7f0c601a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:26.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.710+0000 7f0c65000640 1 -- 172.21.15.31:0/1838481043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c6019f430 msgr2=0x7f0c601a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:26.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.710+0000 7f0c65000640 1 --2- 172.21.15.31:0/1838481043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c6019f430 0x7f0c601a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:26.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.710+0000 7f0c65000640 1 -- 172.21.15.31:0/1838481043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c601039a0 msgr2=0x7f0c60199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:26.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.710+0000 7f0c65000640 1 --2- 172.21.15.31:0/1838481043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c601039a0 0x7f0c60199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:26.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.710+0000 7f0c65000640 1 -- 172.21.15.31:0/1838481043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c601a1e90 con 0x7f0c60105550 2024-04-25T01:13:26.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.710+0000 7f0c66400640 1 --2- 172.21.15.31:0/1838481043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c6019f430 0x7f0c601a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:13:26.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.710+0000 7f0c65000640 1 --2- 172.21.15.31:0/1838481043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c60105550 0x7f0c6019a3d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009230 tx=0x7f0c4c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:26.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.711+0000 7f0c5e000640 1 -- 172.21.15.31:0/1838481043 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c03f030 con 0x7f0c60105550 2024-04-25T01:13:26.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.711+0000 7f0c5e000640 1 -- 172.21.15.31:0/1838481043 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c4c031a20 con 0x7f0c60105550 2024-04-25T01:13:26.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.711+0000 7f0c5e000640 1 -- 172.21.15.31:0/1838481043 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c035070 con 0x7f0c60105550 2024-04-25T01:13:26.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.711+0000 7f0c68400640 1 -- 172.21.15.31:0/1838481043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c601a2120 con 0x7f0c60105550 2024-04-25T01:13:26.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.711+0000 7f0c68400640 1 -- 172.21.15.31:0/1838481043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c601a25c0 con 0x7f0c60105550 2024-04-25T01:13:26.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.712+0000 7f0c68400640 1 -- 172.21.15.31:0/1838481043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c6010ace0 con 0x7f0c60105550 2024-04-25T01:13:26.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.714+0000 7f0c5e000640 1 -- 172.21.15.31:0/1838481043 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0c4c042030 con 0x7f0c60105550 2024-04-25T01:13:26.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.716+0000 7f0c5e000640 1 --2- 172.21.15.31:0/1838481043 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0c480a2c10 0x7f0c480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:26.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.716+0000 7f0c5e000640 1 -- 172.21.15.31:0/1838481043 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f0c4c0eef00 con 0x7f0c60105550 2024-04-25T01:13:26.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.716+0000 7f0c65a00640 1 --2- 172.21.15.31:0/1838481043 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0c480a2c10 0x7f0c480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:26.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.717+0000 7f0c65a00640 1 --2- 172.21.15.31:0/1838481043 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0c480a2c10 0x7f0c480a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0c540095f0 tx=0x7f0c54005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:26.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.724+0000 7f0c5e000640 1 -- 172.21.15.31:0/1838481043 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0c4c049030 con 0x7f0c60105550 2024-04-25T01:13:26.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.978+0000 7f0c68400640 1 -- 172.21.15.31:0/1838481043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c6002d170 con 0x7f0c60105550 2024-04-25T01:13:26.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.978+0000 7f0c5e000640 1 -- 172.21.15.31:0/1838481043 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10467 (secure 0 0 0) 0x7f0c4c0bc260 con 0x7f0c60105550 2024-04-25T01:13:26.966 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:13:26.966 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/498218714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":498218714},{"type":"v1","addr":"172.21.15.31:6835","nonce":498218714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2647199475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2647199475},{"type":"v1","addr":"172.21.15.31:6839","nonce":2647199475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/30518311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":30518311},{"type":"v1","addr":"172.21.15.31:6841","nonce":30518311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/420362823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":420362823},{"type":"v1","addr":"172.21.15.169:6833","nonce":420362823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1943710438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1943710438},{"type":"v1","addr":"172.21.15.40:6833","nonce":1943710438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/4191344115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":4191344115},{"type":"v1","addr":"172.21.15.31:6837","nonce":4191344115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2916305801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2916305801},{"type":"v1","addr":"172.21.15.40:6835","nonce":2916305801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24532,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2253264784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2253264784},{"type":"v1","addr":"172.21.15.169:6835","nonce":2253264784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2883308641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2883308641},{"type":"v1","addr":"172.21.15.169:6837","nonce":2883308641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24548,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/590109562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":590109562},{"type":"v1","addr":"172.21.15.40:6837","nonce":590109562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6839/4185710218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":4185710218},{"type":"v1","addr":"172.21.15.169:6839","nonce":4185710218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6839/927194004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":927194004},{"type":"v1","addr":"172.21.15.40:6839","nonce":927194004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T01:13:26.966 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 13 2024-04-25T01:13:26.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.981+0000 7f0c68400640 1 -- 172.21.15.31:0/1838481043 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0c480a2c10 msgr2=0x7f0c480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:26.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.981+0000 7f0c68400640 1 --2- 172.21.15.31:0/1838481043 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0c480a2c10 0x7f0c480a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0c540095f0 tx=0x7f0c54005f90 comp rx=0 tx=0).stop 2024-04-25T01:13:26.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.981+0000 7f0c68400640 1 -- 172.21.15.31:0/1838481043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c60105550 msgr2=0x7f0c6019a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:26.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.981+0000 7f0c68400640 1 --2- 172.21.15.31:0/1838481043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c60105550 0x7f0c6019a3d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009230 tx=0x7f0c4c0042f0 comp rx=0 tx=0).stop 2024-04-25T01:13:26.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.981+0000 7f0c68400640 1 -- 172.21.15.31:0/1838481043 shutdown_connections 2024-04-25T01:13:26.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.981+0000 7f0c68400640 1 --2- 172.21.15.31:0/1838481043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c6019f430 0x7f0c601a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:26.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.981+0000 7f0c68400640 1 --2- 172.21.15.31:0/1838481043 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0c480a2c10 0x7f0c480a50d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:26.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.981+0000 7f0c68400640 1 --2- 172.21.15.31:0/1838481043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c60105550 0x7f0c6019a3d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:26.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.981+0000 7f0c68400640 1 --2- 172.21.15.31:0/1838481043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c601039a0 0x7f0c60199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:26.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.981+0000 7f0c68400640 1 -- 172.21.15.31:0/1838481043 >> 172.21.15.31:0/1838481043 conn(0x7f0c600fef30 msgr2=0x7f0c60107eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:26.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.982+0000 7f0c68400640 1 -- 172.21.15.31:0/1838481043 shutdown_connections 2024-04-25T01:13:26.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:26.982+0000 7f0c68400640 1 -- 172.21.15.31:0/1838481043 wait complete. 2024-04-25T01:13:26.992 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:13:27.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.195+0000 7fe89ea00640 1 -- 172.21.15.31:0/332236569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8981039a0 msgr2=0x7fe898103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:27.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.195+0000 7fe89ea00640 1 --2- 172.21.15.31:0/332236569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8981039a0 0x7fe898103dc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe888009900 tx=0x7fe88802a410 comp rx=0 tx=0).stop 2024-04-25T01:13:27.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.195+0000 7fe89ea00640 1 -- 172.21.15.31:0/332236569 shutdown_connections 2024-04-25T01:13:27.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.195+0000 7fe89ea00640 1 --2- 172.21.15.31:0/332236569 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe898105550 0x7fe89810bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:27.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.195+0000 7fe89ea00640 1 --2- 172.21.15.31:0/332236569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe898104b90 0x7fe898105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:27.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.195+0000 7fe89ea00640 1 --2- 172.21.15.31:0/332236569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8981039a0 0x7fe898103dc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:27.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.195+0000 7fe89ea00640 1 -- 172.21.15.31:0/332236569 >> 172.21.15.31:0/332236569 conn(0x7fe8980fef30 msgr2=0x7fe898101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:27.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.195+0000 7fe89ea00640 1 -- 172.21.15.31:0/332236569 shutdown_connections 2024-04-25T01:13:27.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.195+0000 7fe89ea00640 1 -- 172.21.15.31:0/332236569 wait complete. 2024-04-25T01:13:27.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.196+0000 7fe89ea00640 1 Processor -- start 2024-04-25T01:13:27.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.196+0000 7fe89ea00640 1 -- start start 2024-04-25T01:13:27.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.197+0000 7fe89ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8981039a0 0x7fe898199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:27.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.197+0000 7fe89ea00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe898104b90 0x7fe89819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:27.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.197+0000 7fe89ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe898105550 0x7fe8981a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:27.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.197+0000 7fe89ea00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe89810f660 con 0x7fe898104b90 2024-04-25T01:13:27.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.197+0000 7fe89ea00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe89810f4e0 con 0x7fe8981039a0 2024-04-25T01:13:27.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.197+0000 7fe89ea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe89810f7e0 con 0x7fe898105550 2024-04-25T01:13:27.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.197+0000 7fe897400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe898104b90 0x7fe89819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:27.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.197+0000 7fe89ca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe898105550 0x7fe8981a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:27.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.197+0000 7fe897e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8981039a0 0x7fe898199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:27.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.197+0000 7fe897400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe898104b90 0x7fe89819a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43352/0 (socket says 172.21.15.31:43352) 2024-04-25T01:13:27.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.197+0000 7fe897400640 1 -- 172.21.15.31:0/398879797 learned_addr learned my addr 172.21.15.31:0/398879797 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:27.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.197+0000 7fe89ca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe898105550 0x7fe8981a1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:60516/0 (socket says 172.21.15.31:60516) 2024-04-25T01:13:27.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.198+0000 7fe89ca00640 1 -- 172.21.15.31:0/398879797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8981039a0 msgr2=0x7fe898199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:27.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.198+0000 7fe89ca00640 1 --2- 172.21.15.31:0/398879797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8981039a0 0x7fe898199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:27.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.198+0000 7fe89ca00640 1 -- 172.21.15.31:0/398879797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe898104b90 msgr2=0x7fe89819a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:27.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.198+0000 7fe89ca00640 1 --2- 172.21.15.31:0/398879797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe898104b90 0x7fe89819a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:27.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.198+0000 7fe89ca00640 1 -- 172.21.15.31:0/398879797 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8981a1900 con 0x7fe898105550 2024-04-25T01:13:27.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.198+0000 7fe89ca00640 1 --2- 172.21.15.31:0/398879797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe898105550 0x7fe8981a1260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe890009fd0 tx=0x7fe89000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:27.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.198+0000 7fe894c00640 1 -- 172.21.15.31:0/398879797 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89000ce20 con 0x7fe898105550 2024-04-25T01:13:27.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.198+0000 7fe89ea00640 1 -- 172.21.15.31:0/398879797 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8981a1bf0 con 0x7fe898105550 2024-04-25T01:13:27.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.198+0000 7fe894c00640 1 -- 172.21.15.31:0/398879797 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe890005020 con 0x7fe898105550 2024-04-25T01:13:27.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.198+0000 7fe894c00640 1 -- 172.21.15.31:0/398879797 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890010720 con 0x7fe898105550 2024-04-25T01:13:27.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.199+0000 7fe89ea00640 1 -- 172.21.15.31:0/398879797 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8981a20c0 con 0x7fe898105550 2024-04-25T01:13:27.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.200+0000 7fe89ea00640 1 -- 172.21.15.31:0/398879797 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8780052c0 con 0x7fe898105550 2024-04-25T01:13:27.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.202+0000 7fe894c00640 1 -- 172.21.15.31:0/398879797 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe890004a40 con 0x7fe898105550 2024-04-25T01:13:27.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.203+0000 7fe894c00640 1 --2- 172.21.15.31:0/398879797 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe87c0a2b90 0x7fe87c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:27.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.203+0000 7fe894c00640 1 -- 172.21.15.31:0/398879797 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fe8900cf260 con 0x7fe898105550 2024-04-25T01:13:27.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.204+0000 7fe897e00640 1 --2- 172.21.15.31:0/398879797 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe87c0a2b90 0x7fe87c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:27.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.204+0000 7fe897e00640 1 --2- 172.21.15.31:0/398879797 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe87c0a2b90 0x7fe87c0a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe88802aa00 tx=0x7fe8880315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:27.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.208+0000 7fe894c00640 1 -- 172.21.15.31:0/398879797 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe89009c5c0 con 0x7fe898105550 2024-04-25T01:13:27.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:27 smithi031 ceph-mon[28053]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:27.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:27 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:27.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:27 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:27.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:27 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:27.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:27 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:27.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:27 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1838481043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:27.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:27 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:27.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.428+0000 7fe89ea00640 1 -- 172.21.15.31:0/398879797 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8780055b0 con 0x7fe898105550 2024-04-25T01:13:27.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.430+0000 7fe894c00640 1 -- 172.21.15.31:0/398879797 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+18409 (secure 0 0 0) 0x7fe89009bec0 con 0x7fe898105550 2024-04-25T01:13:27.417 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:13:27.417 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":83,"fsid":"95849b94-029f-11ef-bc93-c7b262605968","created":"2024-04-25T01:04:23.310856+0000","modified":"2024-04-25T01:12:30.554122+0000","last_up_change":"2024-04-25T01:12:29.393907+0000","last_in_change":"2024-04-25T01:12:08.978148+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:18.662391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa08e4c-fe6b-4116-bdf5-1adf5b465d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6803","nonce":788174091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6805","nonce":788174091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6809","nonce":788174091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6807","nonce":788174091}]},"public_addr":"172.21.15.31:6803/788174091","cluster_addr":"172.21.15.31:6805/788174091","heartbeat_back_addr":"172.21.15.31:6809/788174091","heartbeat_front_addr":"172.21.15.31:6807/788174091","state":["exists","up"]},{"osd":1,"uuid":"85339fe7-0dbc-4485-9334-2b039993c1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6801","nonce":3196394509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6803","nonce":3196394509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6807","nonce":3196394509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6805","nonce":3196394509}]},"public_addr":"172.21.15.40:6801/3196394509","cluster_addr":"172.21.15.40:6803/3196394509","heartbeat_back_addr":"172.21.15.40:6807/3196394509","heartbeat_front_addr":"172.21.15.40:6805/3196394509","state":["exists","up"]},{"osd":2,"uuid":"ab62ecb2-8aae-4987-9e2e-589e0f2fee5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6801","nonce":1955211197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6803","nonce":1955211197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6807","nonce":1955211197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6805","nonce":1955211197}]},"public_addr":"172.21.15.169:6801/1955211197","cluster_addr":"172.21.15.169:6803/1955211197","heartbeat_back_addr":"172.21.15.169:6807/1955211197","heartbeat_front_addr":"172.21.15.169:6805/1955211197","state":["exists","up"]},{"osd":3,"uuid":"8b5a7881-3401-495c-8ca4-c90685919d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6811","nonce":1228688643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6813","nonce":1228688643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6817","nonce":1228688643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6815","nonce":1228688643}]},"public_addr":"172.21.15.31:6811/1228688643","cluster_addr":"172.21.15.31:6813/1228688643","heartbeat_back_addr":"172.21.15.31:6817/1228688643","heartbeat_front_addr":"172.21.15.31:6815/1228688643","state":["exists","up"]},{"osd":4,"uuid":"56deb61d-a334-4cf2-8ac7-fbb6e6947092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6809","nonce":1913779091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6811","nonce":1913779091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6815","nonce":1913779091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6813","nonce":1913779091}]},"public_addr":"172.21.15.40:6809/1913779091","cluster_addr":"172.21.15.40:6811/1913779091","heartbeat_back_addr":"172.21.15.40:6815/1913779091","heartbeat_front_addr":"172.21.15.40:6813/1913779091","state":["exists","up"]},{"osd":5,"uuid":"8da55693-f9c7-49a7-99e0-165078f8001b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6809","nonce":2549650781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6811","nonce":2549650781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6815","nonce":2549650781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6813","nonce":2549650781}]},"public_addr":"172.21.15.169:6809/2549650781","cluster_addr":"172.21.15.169:6811/2549650781","heartbeat_back_addr":"172.21.15.169:6815/2549650781","heartbeat_front_addr":"172.21.15.169:6813/2549650781","state":["exists","up"]},{"osd":6,"uuid":"fc8350d2-1ad6-43cb-89f4-d336afc99ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6819","nonce":611507624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6821","nonce":611507624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6825","nonce":611507624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6823","nonce":611507624}]},"public_addr":"172.21.15.31:6819/611507624","cluster_addr":"172.21.15.31:6821/611507624","heartbeat_back_addr":"172.21.15.31:6825/611507624","heartbeat_front_addr":"172.21.15.31:6823/611507624","state":["exists","up"]},{"osd":7,"uuid":"3121b58b-8c94-40b5-9d48-f56552d7c39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6817","nonce":3776085849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6819","nonce":3776085849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6823","nonce":3776085849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6821","nonce":3776085849}]},"public_addr":"172.21.15.40:6817/3776085849","cluster_addr":"172.21.15.40:6819/3776085849","heartbeat_back_addr":"172.21.15.40:6823/3776085849","heartbeat_front_addr":"172.21.15.40:6821/3776085849","state":["exists","up"]},{"osd":8,"uuid":"f60407df-91f5-477a-9702-ddaef24faf82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6817","nonce":2736546122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6819","nonce":2736546122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6823","nonce":2736546122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6821","nonce":2736546122}]},"public_addr":"172.21.15.169:6817/2736546122","cluster_addr":"172.21.15.169:6819/2736546122","heartbeat_back_addr":"172.21.15.169:6823/2736546122","heartbeat_front_addr":"172.21.15.169:6821/2736546122","state":["exists","up"]},{"osd":9,"uuid":"7506a407-1960-49bc-9e38-7f2a8bf996ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6827","nonce":1187207880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6829","nonce":1187207880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6833","nonce":1187207880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6831","nonce":1187207880}]},"public_addr":"172.21.15.31:6827/1187207880","cluster_addr":"172.21.15.31:6829/1187207880","heartbeat_back_addr":"172.21.15.31:6833/1187207880","heartbeat_front_addr":"172.21.15.31:6831/1187207880","state":["exists","up"]},{"osd":10,"uuid":"831c48e2-b0d0-451f-959b-228e927542cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6825","nonce":3050148001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6827","nonce":3050148001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6831","nonce":3050148001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6829","nonce":3050148001}]},"public_addr":"172.21.15.40:6825/3050148001","cluster_addr":"172.21.15.40:6827/3050148001","heartbeat_back_addr":"172.21.15.40:6831/3050148001","heartbeat_front_addr":"172.21.15.40:6829/3050148001","state":["exists","up"]},{"osd":11,"uuid":"144fe0aa-f757-4b3c-90ad-99775e17d57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6825","nonce":262401944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6827","nonce":262401944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6831","nonce":262401944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6829","nonce":262401944}]},"public_addr":"172.21.15.169:6825/262401944","cluster_addr":"172.21.15.169:6827/262401944","heartbeat_back_addr":"172.21.15.169:6831/262401944","heartbeat_front_addr":"172.21.15.169:6829/262401944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:25.640995+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:51.036909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.635633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:48.467912+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:14.454971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:37.609315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:12.441807+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:38.239215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:02.230758+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:35.762553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:02.051405+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:26.569593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6801/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4004014066":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/1930101646":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/2638309002":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2238638505":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2902841144":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4190696323":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/480558521":"2024-04-26T01:04:57.149272+0000","172.21.15.31:6801/2638309002":"2024-04-26T01:04:57.149272+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:13:27.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.433+0000 7fe89ea00640 1 -- 172.21.15.31:0/398879797 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe87c0a2b90 msgr2=0x7fe87c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:27.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.433+0000 7fe89ea00640 1 --2- 172.21.15.31:0/398879797 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe87c0a2b90 0x7fe87c0a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe88802aa00 tx=0x7fe8880315b0 comp rx=0 tx=0).stop 2024-04-25T01:13:27.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.433+0000 7fe89ea00640 1 -- 172.21.15.31:0/398879797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe898105550 msgr2=0x7fe8981a1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:27.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.433+0000 7fe89ea00640 1 --2- 172.21.15.31:0/398879797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe898105550 0x7fe8981a1260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe890009fd0 tx=0x7fe89000ef10 comp rx=0 tx=0).stop 2024-04-25T01:13:27.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.433+0000 7fe89ea00640 1 -- 172.21.15.31:0/398879797 shutdown_connections 2024-04-25T01:13:27.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.433+0000 7fe89ea00640 1 --2- 172.21.15.31:0/398879797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe898105550 0x7fe8981a1260 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:27.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.433+0000 7fe89ea00640 1 --2- 172.21.15.31:0/398879797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe898104b90 0x7fe89819a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:27.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.433+0000 7fe89ea00640 1 --2- 172.21.15.31:0/398879797 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe87c0a2b90 0x7fe87c0a5050 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:27.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.433+0000 7fe89ea00640 1 --2- 172.21.15.31:0/398879797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8981039a0 0x7fe898199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:27.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.433+0000 7fe89ea00640 1 -- 172.21.15.31:0/398879797 >> 172.21.15.31:0/398879797 conn(0x7fe8980fef30 msgr2=0x7fe898107e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:27.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.433+0000 7fe89ea00640 1 -- 172.21.15.31:0/398879797 shutdown_connections 2024-04-25T01:13:27.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.433+0000 7fe89ea00640 1 -- 172.21.15.31:0/398879797 wait complete. 2024-04-25T01:13:27.438 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:18.662391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:13:27.439 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:13:27.448 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:27 smithi040 ceph-mon[32518]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:27.448 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:27 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:27.448 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:27 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:27.449 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:27 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:27.449 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:27 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:27.449 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:27 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1838481043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:27.449 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:27 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:27.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.631+0000 7fe0ef200640 1 -- 172.21.15.31:0/2602607769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0f0101670 msgr2=0x7fe0f0101a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:27.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.631+0000 7fe0ef200640 1 --2- 172.21.15.31:0/2602607769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0f0101670 0x7fe0f0101a90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009900 tx=0x7fe0e002a3f0 comp rx=0 tx=0).stop 2024-04-25T01:13:27.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.631+0000 7fe0ef200640 1 -- 172.21.15.31:0/2602607769 shutdown_connections 2024-04-25T01:13:27.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.631+0000 7fe0ef200640 1 --2- 172.21.15.31:0/2602607769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0f0103220 0x7fe0f0109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:27.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.631+0000 7fe0ef200640 1 --2- 172.21.15.31:0/2602607769 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0f0102860 0x7fe0f0102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:27.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.631+0000 7fe0ef200640 1 --2- 172.21.15.31:0/2602607769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0f0101670 0x7fe0f0101a90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:27.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.631+0000 7fe0ef200640 1 -- 172.21.15.31:0/2602607769 >> 172.21.15.31:0/2602607769 conn(0x7fe0f00fcbe0 msgr2=0x7fe0f00ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:27.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.631+0000 7fe0ef200640 1 -- 172.21.15.31:0/2602607769 shutdown_connections 2024-04-25T01:13:27.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.631+0000 7fe0ef200640 1 -- 172.21.15.31:0/2602607769 wait complete. 2024-04-25T01:13:27.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.632+0000 7fe0ef200640 1 Processor -- start 2024-04-25T01:13:27.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.632+0000 7fe0ef200640 1 -- start start 2024-04-25T01:13:27.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.633+0000 7fe0ef200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0f0101670 0x7fe0f0199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:27.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.633+0000 7fe0ef200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0f0102860 0x7fe0f019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.633+0000 7fe0ef200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0f0103220 0x7fe0f01a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.633+0000 7fe0ef200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f010d310 con 0x7fe0f0103220 2024-04-25T01:13:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.633+0000 7fe0ef200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f010d190 con 0x7fe0f0102860 2024-04-25T01:13:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.633+0000 7fe0ef200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f010d490 con 0x7fe0f0101670 2024-04-25T01:13:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.633+0000 7fe0ede00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0f0101670 0x7fe0f0199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.633+0000 7fe0ede00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0f0101670 0x7fe0f0199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:60538/0 (socket says 172.21.15.31:60538) 2024-04-25T01:13:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.633+0000 7fe0ede00640 1 -- 172.21.15.31:0/2810862143 learned_addr learned my addr 172.21.15.31:0/2810862143 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.633+0000 7fe0ee800640 1 --2- 172.21.15.31:0/2810862143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0f0103220 0x7fe0f01a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.633+0000 7fe0e7e00640 1 --2- 172.21.15.31:0/2810862143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0f0102860 0x7fe0f019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.633+0000 7fe0ede00640 1 -- 172.21.15.31:0/2810862143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0f0102860 msgr2=0x7fe0f019a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.634+0000 7fe0ede00640 1 --2- 172.21.15.31:0/2810862143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0f0102860 0x7fe0f019a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.634+0000 7fe0ede00640 1 -- 172.21.15.31:0/2810862143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0f0103220 msgr2=0x7fe0f01a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.634+0000 7fe0ede00640 1 --2- 172.21.15.31:0/2810862143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0f0103220 0x7fe0f01a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:27.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.634+0000 7fe0ede00640 1 -- 172.21.15.31:0/2810862143 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f01a17e0 con 0x7fe0f0101670 2024-04-25T01:13:27.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.634+0000 7fe0ede00640 1 --2- 172.21.15.31:0/2810862143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0f0101670 0x7fe0f0199bc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0031440 tx=0x7fe0e0032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:27.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.634+0000 7fe0e6a00640 1 -- 172.21.15.31:0/2810862143 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e003f030 con 0x7fe0f0101670 2024-04-25T01:13:27.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.634+0000 7fe0e6a00640 1 -- 172.21.15.31:0/2810862143 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0e0003ce0 con 0x7fe0f0101670 2024-04-25T01:13:27.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.634+0000 7fe0ef200640 1 -- 172.21.15.31:0/2810862143 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f01a1a70 con 0x7fe0f0101670 2024-04-25T01:13:27.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.634+0000 7fe0e6a00640 1 -- 172.21.15.31:0/2810862143 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0035070 con 0x7fe0f0101670 2024-04-25T01:13:27.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.634+0000 7fe0ef200640 1 -- 172.21.15.31:0/2810862143 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f01a1f60 con 0x7fe0f0101670 2024-04-25T01:13:27.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.635+0000 7fe0ef200640 1 -- 172.21.15.31:0/2810862143 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0f01a2190 con 0x7fe0f0101670 2024-04-25T01:13:27.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.638+0000 7fe0e6a00640 1 -- 172.21.15.31:0/2810862143 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe0e0042030 con 0x7fe0f0101670 2024-04-25T01:13:27.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.639+0000 7fe0e6a00640 1 --2- 172.21.15.31:0/2810862143 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe0d40a2b90 0x7fe0d40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:27.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.639+0000 7fe0e6a00640 1 -- 172.21.15.31:0/2810862143 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fe0e00ef580 con 0x7fe0f0101670 2024-04-25T01:13:27.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.639+0000 7fe0e7e00640 1 --2- 172.21.15.31:0/2810862143 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe0d40a2b90 0x7fe0d40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:27.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.640+0000 7fe0e7e00640 1 --2- 172.21.15.31:0/2810862143 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe0d40a2b90 0x7fe0d40a5050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80095f0 tx=0x7fe0d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:27.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.641+0000 7fe0e6a00640 1 -- 172.21.15.31:0/2810862143 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe0e00bc290 con 0x7fe0f0101670 2024-04-25T01:13:27.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:27 smithi169 ceph-mon[32759]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:27.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:27.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:27.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:27.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:27.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:27 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1838481043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:27.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:27 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:27.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.837+0000 7fe0ef200640 1 -- 172.21.15.31:0/2810862143 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe0f01a2360 con 0x7fe0f0101670 2024-04-25T01:13:27.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.837+0000 7fe0e6a00640 1 -- 172.21.15.31:0/2810862143 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7fe0e00dae20 con 0x7fe0f0101670 2024-04-25T01:13:27.824 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T01:13:27.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.839+0000 7fe0ef200640 1 -- 172.21.15.31:0/2810862143 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe0d40a2b90 msgr2=0x7fe0d40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:27.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.839+0000 7fe0ef200640 1 --2- 172.21.15.31:0/2810862143 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe0d40a2b90 0x7fe0d40a5050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80095f0 tx=0x7fe0d8005f90 comp rx=0 tx=0).stop 2024-04-25T01:13:27.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.839+0000 7fe0ef200640 1 -- 172.21.15.31:0/2810862143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0f0101670 msgr2=0x7fe0f0199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:27.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.840+0000 7fe0ef200640 1 --2- 172.21.15.31:0/2810862143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0f0101670 0x7fe0f0199bc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0031440 tx=0x7fe0e0032ce0 comp rx=0 tx=0).stop 2024-04-25T01:13:27.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.840+0000 7fe0ef200640 1 -- 172.21.15.31:0/2810862143 shutdown_connections 2024-04-25T01:13:27.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.840+0000 7fe0ef200640 1 --2- 172.21.15.31:0/2810862143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0f0103220 0x7fe0f01a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:27.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.840+0000 7fe0ef200640 1 --2- 172.21.15.31:0/2810862143 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe0d40a2b90 0x7fe0d40a5050 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:27.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.840+0000 7fe0ef200640 1 --2- 172.21.15.31:0/2810862143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0f0102860 0x7fe0f019a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:27.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.840+0000 7fe0ef200640 1 --2- 172.21.15.31:0/2810862143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0f0101670 0x7fe0f0199bc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:27.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.840+0000 7fe0ef200640 1 -- 172.21.15.31:0/2810862143 >> 172.21.15.31:0/2810862143 conn(0x7fe0f00fcbe0 msgr2=0x7fe0f0073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:27.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.840+0000 7fe0ef200640 1 -- 172.21.15.31:0/2810862143 shutdown_connections 2024-04-25T01:13:27.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:27.840+0000 7fe0ef200640 1 -- 172.21.15.31:0/2810862143 wait complete. 2024-04-25T01:13:27.845 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T01:13:28.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.031+0000 7f19e5400640 1 -- 172.21.15.31:0/2880969676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19e0100000 msgr2=0x7f19e0102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:28.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.031+0000 7f19e5400640 1 --2- 172.21.15.31:0/2880969676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19e0100000 0x7f19e0102430 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009860 tx=0x7f19d002a340 comp rx=0 tx=0).stop 2024-04-25T01:13:28.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.031+0000 7f19e5400640 1 -- 172.21.15.31:0/2880969676 shutdown_connections 2024-04-25T01:13:28.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.031+0000 7f19e5400640 1 --2- 172.21.15.31:0/2880969676 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19e0105320 0x7f19e01057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:28.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.031+0000 7f19e5400640 1 --2- 172.21.15.31:0/2880969676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19e0102970 0x7f19e0104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:28.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.031+0000 7f19e5400640 1 --2- 172.21.15.31:0/2880969676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19e0100000 0x7f19e0102430 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:28.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.031+0000 7f19e5400640 1 -- 172.21.15.31:0/2880969676 >> 172.21.15.31:0/2880969676 conn(0x7f19e00fbc20 msgr2=0x7f19e00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:28.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.031+0000 7f19e5400640 1 -- 172.21.15.31:0/2880969676 shutdown_connections 2024-04-25T01:13:28.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.031+0000 7f19e5400640 1 -- 172.21.15.31:0/2880969676 wait complete. 2024-04-25T01:13:28.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.032+0000 7f19e5400640 1 Processor -- start 2024-04-25T01:13:28.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.032+0000 7f19e5400640 1 -- start start 2024-04-25T01:13:28.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.032+0000 7f19e5400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19e0100000 0x7f19e019e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:28.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.032+0000 7f19e5400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19e0102970 0x7f19e019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:28.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.032+0000 7f19e5400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19e0105320 0x7f19e01a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:28.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.032+0000 7f19e5400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f19e0113a00 con 0x7f19e0102970 2024-04-25T01:13:28.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.032+0000 7f19e5400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f19e0113880 con 0x7f19e0105320 2024-04-25T01:13:28.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.032+0000 7f19e5400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f19e0113b80 con 0x7f19e0100000 2024-04-25T01:13:28.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.032+0000 7f19de800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19e0100000 0x7f19e019e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:28.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.033+0000 7f19df200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19e0105320 0x7f19e01a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:28.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.033+0000 7f19de800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19e0100000 0x7f19e019e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:60548/0 (socket says 172.21.15.31:60548) 2024-04-25T01:13:28.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.033+0000 7f19de800640 1 -- 172.21.15.31:0/2005794079 learned_addr learned my addr 172.21.15.31:0/2005794079 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:28.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.033+0000 7f19d7e00640 1 --2- 172.21.15.31:0/2005794079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19e0102970 0x7f19e019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:28.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.033+0000 7f19de800640 1 -- 172.21.15.31:0/2005794079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19e0105320 msgr2=0x7f19e01a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:28.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.033+0000 7f19de800640 1 --2- 172.21.15.31:0/2005794079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19e0105320 0x7f19e01a5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:28.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.033+0000 7f19de800640 1 -- 172.21.15.31:0/2005794079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19e0102970 msgr2=0x7f19e019e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:28.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.033+0000 7f19de800640 1 --2- 172.21.15.31:0/2005794079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19e0102970 0x7f19e019e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:28.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.033+0000 7f19de800640 1 -- 172.21.15.31:0/2005794079 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e01a5ca0 con 0x7f19e0100000 2024-04-25T01:13:28.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.033+0000 7f19de800640 1 --2- 172.21.15.31:0/2005794079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19e0100000 0x7f19e019e0d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f19d0035040 tx=0x7f19d0033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:28.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.033+0000 7f19d7400640 1 -- 172.21.15.31:0/2005794079 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0037040 con 0x7f19e0100000 2024-04-25T01:13:28.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.033+0000 7f19d7400640 1 -- 172.21.15.31:0/2005794079 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19d0030850 con 0x7f19e0100000 2024-04-25T01:13:28.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.034+0000 7f19e5400640 1 -- 172.21.15.31:0/2005794079 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e01a5f30 con 0x7f19e0100000 2024-04-25T01:13:28.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.034+0000 7f19d7400640 1 -- 172.21.15.31:0/2005794079 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0004920 con 0x7f19e0100000 2024-04-25T01:13:28.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.034+0000 7f19e5400640 1 -- 172.21.15.31:0/2005794079 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e01a6390 con 0x7f19e0100000 2024-04-25T01:13:28.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.035+0000 7f19e5400640 1 -- 172.21.15.31:0/2005794079 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e00fd710 con 0x7f19e0100000 2024-04-25T01:13:28.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.037+0000 7f19d7400640 1 -- 172.21.15.31:0/2005794079 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f19d0004050 con 0x7f19e0100000 2024-04-25T01:13:28.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.038+0000 7f19d7400640 1 --2- 172.21.15.31:0/2005794079 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f19c40a2c10 0x7f19c40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:28.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.038+0000 7f19d7400640 1 -- 172.21.15.31:0/2005794079 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f19d00eeb70 con 0x7f19e0100000 2024-04-25T01:13:28.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.038+0000 7f19d7e00640 1 --2- 172.21.15.31:0/2005794079 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f19c40a2c10 0x7f19c40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:28.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.039+0000 7f19d7e00640 1 --2- 172.21.15.31:0/2005794079 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f19c40a2c10 0x7f19c40a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f19e019f590 tx=0x7f19c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:28.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.042+0000 7f19d7400640 1 -- 172.21.15.31:0/2005794079 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f19d00bbed0 con 0x7f19e0100000 2024-04-25T01:13:28.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.248+0000 7f19e5400640 1 -- 172.21.15.31:0/2005794079 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f19e002d390 con 0x7f19e0100000 2024-04-25T01:13:28.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.248+0000 7f19d7400640 1 -- 172.21.15.31:0/2005794079 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) ==== 84+0+65 (secure 0 0 0) 0x7f19d00bb7d0 con 0x7f19e0100000 2024-04-25T01:13:28.236 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:13:28.236 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2024-04-25T01:13:28.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:13:28.237 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2024-04-25T01:13:28.237 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2024-04-25T01:13:28.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:13:28.237 INFO:teuthology.orchestra.run.smithi031.stdout:] 2024-04-25T01:13:28.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.252+0000 7f19e5400640 1 -- 172.21.15.31:0/2005794079 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f19c40a2c10 msgr2=0x7f19c40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:28.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.252+0000 7f19e5400640 1 --2- 172.21.15.31:0/2005794079 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f19c40a2c10 0x7f19c40a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f19e019f590 tx=0x7f19c8009260 comp rx=0 tx=0).stop 2024-04-25T01:13:28.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.252+0000 7f19e5400640 1 -- 172.21.15.31:0/2005794079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19e0100000 msgr2=0x7f19e019e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:28.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.252+0000 7f19e5400640 1 --2- 172.21.15.31:0/2005794079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19e0100000 0x7f19e019e0d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f19d0035040 tx=0x7f19d0033d90 comp rx=0 tx=0).stop 2024-04-25T01:13:28.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.252+0000 7f19e5400640 1 -- 172.21.15.31:0/2005794079 shutdown_connections 2024-04-25T01:13:28.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.252+0000 7f19e5400640 1 --2- 172.21.15.31:0/2005794079 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f19c40a2c10 0x7f19c40a50d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:28.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.252+0000 7f19e5400640 1 --2- 172.21.15.31:0/2005794079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19e0105320 0x7f19e01a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:28.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.252+0000 7f19e5400640 1 --2- 172.21.15.31:0/2005794079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19e0102970 0x7f19e019e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:28.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.252+0000 7f19e5400640 1 --2- 172.21.15.31:0/2005794079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19e0100000 0x7f19e019e0d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:28.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.252+0000 7f19e5400640 1 -- 172.21.15.31:0/2005794079 >> 172.21.15.31:0/2005794079 conn(0x7f19e00fbc20 msgr2=0x7f19e0103320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:28.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.252+0000 7f19e5400640 1 -- 172.21.15.31:0/2005794079 shutdown_connections 2024-04-25T01:13:28.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.252+0000 7f19e5400640 1 -- 172.21.15.31:0/2005794079 wait complete. 2024-04-25T01:13:28.258 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T01:13:28.258 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T01:13:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.473+0000 7f483ac00640 1 -- 172.21.15.31:0/3447388987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4834103880 msgr2=0x7f4834103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.473+0000 7f483ac00640 1 --2- 172.21.15.31:0/3447388987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4834103880 0x7f4834103ca0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4824009ca0 tx=0x7f482402a8b0 comp rx=0 tx=0).stop 2024-04-25T01:13:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.473+0000 7f483ac00640 1 -- 172.21.15.31:0/3447388987 shutdown_connections 2024-04-25T01:13:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.473+0000 7f483ac00640 1 --2- 172.21.15.31:0/3447388987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4834105430 0x7f483410bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.473+0000 7f483ac00640 1 --2- 172.21.15.31:0/3447388987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834104a70 0x7f4834104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.473+0000 7f483ac00640 1 --2- 172.21.15.31:0/3447388987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4834103880 0x7f4834103ca0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.473+0000 7f483ac00640 1 -- 172.21.15.31:0/3447388987 >> 172.21.15.31:0/3447388987 conn(0x7f48340fedf0 msgr2=0x7f4834101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.473+0000 7f483ac00640 1 -- 172.21.15.31:0/3447388987 shutdown_connections 2024-04-25T01:13:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.473+0000 7f483ac00640 1 -- 172.21.15.31:0/3447388987 wait complete. 2024-04-25T01:13:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.474+0000 7f483ac00640 1 Processor -- start 2024-04-25T01:13:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.474+0000 7f483ac00640 1 -- start start 2024-04-25T01:13:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.475+0000 7f483ac00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4834103880 0x7f4834199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.475+0000 7f483ac00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834104a70 0x7f483419a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.475+0000 7f483ac00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4834105430 0x7f48341a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.475+0000 7f483ac00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f483410f520 con 0x7f4834104a70 2024-04-25T01:13:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.475+0000 7f483ac00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f483410f3a0 con 0x7f4834105430 2024-04-25T01:13:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.475+0000 7f483ac00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f483410f6a0 con 0x7f4834103880 2024-04-25T01:13:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.475+0000 7f4839800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4834103880 0x7f4834199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.475+0000 7f4839800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4834103880 0x7f4834199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:60566/0 (socket says 172.21.15.31:60566) 2024-04-25T01:13:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.475+0000 7f4839800640 1 -- 172.21.15.31:0/2071262930 learned_addr learned my addr 172.21.15.31:0/2071262930 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.475+0000 7f4838e00640 1 --2- 172.21.15.31:0/2071262930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834104a70 0x7f483419a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.475+0000 7f483a200640 1 --2- 172.21.15.31:0/2071262930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4834105430 0x7f48341a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.475+0000 7f4839800640 1 -- 172.21.15.31:0/2071262930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4834105430 msgr2=0x7f48341a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.475+0000 7f4839800640 1 --2- 172.21.15.31:0/2071262930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4834105430 0x7f48341a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.475+0000 7f4839800640 1 -- 172.21.15.31:0/2071262930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834104a70 msgr2=0x7f483419a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.475+0000 7f4839800640 1 --2- 172.21.15.31:0/2071262930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834104a70 0x7f483419a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.475+0000 7f4839800640 1 -- 172.21.15.31:0/2071262930 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48341a17d0 con 0x7f4834103880 2024-04-25T01:13:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.476+0000 7f4839800640 1 --2- 172.21.15.31:0/2071262930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4834103880 0x7f4834199bb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4824000f80 tx=0x7f4824003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.476+0000 7f4832000640 1 -- 172.21.15.31:0/2071262930 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824004040 con 0x7f4834103880 2024-04-25T01:13:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.476+0000 7f4832000640 1 -- 172.21.15.31:0/2071262930 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4824040070 con 0x7f4834103880 2024-04-25T01:13:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.476+0000 7f4832000640 1 -- 172.21.15.31:0/2071262930 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824031dd0 con 0x7f4834103880 2024-04-25T01:13:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.476+0000 7f483ac00640 1 -- 172.21.15.31:0/2071262930 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48341a1a60 con 0x7f4834103880 2024-04-25T01:13:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.476+0000 7f483ac00640 1 -- 172.21.15.31:0/2071262930 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48341a1f60 con 0x7f4834103880 2024-04-25T01:13:28.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.477+0000 7f483ac00640 1 -- 172.21.15.31:0/2071262930 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f483410ac40 con 0x7f4834103880 2024-04-25T01:13:28.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.479+0000 7f4832000640 1 -- 172.21.15.31:0/2071262930 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f48240314d0 con 0x7f4834103880 2024-04-25T01:13:28.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.480+0000 7f4832000640 1 --2- 172.21.15.31:0/2071262930 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f48180a2b90 0x7f48180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:28.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.481+0000 7f4832000640 1 -- 172.21.15.31:0/2071262930 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f48240eff50 con 0x7f4834103880 2024-04-25T01:13:28.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.481+0000 7f4838e00640 1 --2- 172.21.15.31:0/2071262930 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f48180a2b90 0x7f48180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:28.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.481+0000 7f4838e00640 1 --2- 172.21.15.31:0/2071262930 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f48180a2b90 0x7f48180a5050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f48200095f0 tx=0x7f4820005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:28.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.484+0000 7f4832000640 1 -- 172.21.15.31:0/2071262930 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f48240bd2b0 con 0x7f4834103880 2024-04-25T01:13:28.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:28 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:28.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:28 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:28.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:28 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:28.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:28 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/398879797' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:13:28.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:28 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:13:28.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:28 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:28.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:28 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:13:28.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:28 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2810862143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:13:28.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:28 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2005794079' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T01:13:28.674 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:28 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:28.675 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:28 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:28.675 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:28 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:28.675 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:28 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/398879797' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:13:28.675 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:28 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:13:28.675 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:28 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:28.675 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:28 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:13:28.675 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:28 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2810862143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:13:28.675 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:28 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2005794079' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T01:13:28.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:28.706+0000 7f483ac00640 1 -- 172.21.15.31:0/2071262930 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4834062340 con 0x7f4834103880 2024-04-25T01:13:28.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:28 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:28.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:28 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:28.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:28 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:28.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:28 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/398879797' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:13:28.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:28 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:13:28.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:28 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:28.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:28 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:13:28.699 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:28 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2810862143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:13:28.699 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:28 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2005794079' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T01:13:29.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.454+0000 7f4832000640 1 -- 172.21.15.31:0/2071262930 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) ==== 138+0+0 (secure 0 0 0) 0x7f48240bcbb0 con 0x7f4834103880 2024-04-25T01:13:29.443 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2024-04-25T01:13:29.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.465+0000 7f483ac00640 1 -- 172.21.15.31:0/2071262930 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f48180a2b90 msgr2=0x7f48180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:29.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.465+0000 7f483ac00640 1 --2- 172.21.15.31:0/2071262930 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f48180a2b90 0x7f48180a5050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f48200095f0 tx=0x7f4820005f90 comp rx=0 tx=0).stop 2024-04-25T01:13:29.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.466+0000 7f483ac00640 1 -- 172.21.15.31:0/2071262930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4834103880 msgr2=0x7f4834199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:29.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.466+0000 7f483ac00640 1 --2- 172.21.15.31:0/2071262930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4834103880 0x7f4834199bb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4824000f80 tx=0x7f4824003ba0 comp rx=0 tx=0).stop 2024-04-25T01:13:29.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.466+0000 7f483ac00640 1 -- 172.21.15.31:0/2071262930 shutdown_connections 2024-04-25T01:13:29.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.466+0000 7f483ac00640 1 --2- 172.21.15.31:0/2071262930 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f48180a2b90 0x7f48180a5050 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:29.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.466+0000 7f483ac00640 1 --2- 172.21.15.31:0/2071262930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4834105430 0x7f48341a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:29.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.466+0000 7f483ac00640 1 --2- 172.21.15.31:0/2071262930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834104a70 0x7f483419a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:29.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.466+0000 7f483ac00640 1 --2- 172.21.15.31:0/2071262930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4834103880 0x7f4834199bb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:29.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.466+0000 7f483ac00640 1 -- 172.21.15.31:0/2071262930 >> 172.21.15.31:0/2071262930 conn(0x7f48340fedf0 msgr2=0x7f4834107e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:29.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.467+0000 7f483ac00640 1 -- 172.21.15.31:0/2071262930 shutdown_connections 2024-04-25T01:13:29.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.467+0000 7f483ac00640 1 -- 172.21.15.31:0/2071262930 wait complete. 2024-04-25T01:13:29.477 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T01:13:29.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.676+0000 7f3936800640 1 -- 172.21.15.31:0/1547956153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39301039c0 msgr2=0x7f3930103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:29.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.676+0000 7f3936800640 1 --2- 172.21.15.31:0/1547956153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39301039c0 0x7f3930103de0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3920009860 tx=0x7f392002a340 comp rx=0 tx=0).stop 2024-04-25T01:13:29.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.676+0000 7f3936800640 1 -- 172.21.15.31:0/1547956153 shutdown_connections 2024-04-25T01:13:29.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.676+0000 7f3936800640 1 --2- 172.21.15.31:0/1547956153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3930105570 0x7f393010be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:29.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.676+0000 7f3936800640 1 --2- 172.21.15.31:0/1547956153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3930104bb0 0x7f3930105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:29.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.676+0000 7f3936800640 1 --2- 172.21.15.31:0/1547956153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39301039c0 0x7f3930103de0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:29.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.676+0000 7f3936800640 1 -- 172.21.15.31:0/1547956153 >> 172.21.15.31:0/1547956153 conn(0x7f39300fef30 msgr2=0x7f3930101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:29.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.676+0000 7f3936800640 1 -- 172.21.15.31:0/1547956153 shutdown_connections 2024-04-25T01:13:29.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.676+0000 7f3936800640 1 -- 172.21.15.31:0/1547956153 wait complete. 2024-04-25T01:13:29.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.677+0000 7f3936800640 1 Processor -- start 2024-04-25T01:13:29.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.677+0000 7f3936800640 1 -- start start 2024-04-25T01:13:29.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.678+0000 7f3936800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39301039c0 0x7f3930199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:29.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.678+0000 7f3936800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3930104bb0 0x7f393019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:29.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.678+0000 7f3936800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3930105570 0x7f39301a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:29.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.678+0000 7f3936800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f393010f640 con 0x7f39301039c0 2024-04-25T01:13:29.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.678+0000 7f3936800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f393010f4c0 con 0x7f3930104bb0 2024-04-25T01:13:29.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.678+0000 7f3936800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f393010f7c0 con 0x7f3930105570 2024-04-25T01:13:29.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.678+0000 7f392f400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39301039c0 0x7f3930199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:29.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.678+0000 7f392f400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39301039c0 0x7f3930199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43426/0 (socket says 172.21.15.31:43426) 2024-04-25T01:13:29.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.678+0000 7f392f400640 1 -- 172.21.15.31:0/2572078141 learned_addr learned my addr 172.21.15.31:0/2572078141 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:29.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.678+0000 7f392fe00640 1 --2- 172.21.15.31:0/2572078141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3930105570 0x7f39301a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:29.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.678+0000 7f392ea00640 1 --2- 172.21.15.31:0/2572078141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3930104bb0 0x7f393019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:29.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.678+0000 7f392f400640 1 -- 172.21.15.31:0/2572078141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3930105570 msgr2=0x7f39301a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:29.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.678+0000 7f392f400640 1 --2- 172.21.15.31:0/2572078141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3930105570 0x7f39301a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:29.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.678+0000 7f392f400640 1 -- 172.21.15.31:0/2572078141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3930104bb0 msgr2=0x7f393019a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:29.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.678+0000 7f392f400640 1 --2- 172.21.15.31:0/2572078141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3930104bb0 0x7f393019a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:29.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.678+0000 7f392f400640 1 -- 172.21.15.31:0/2572078141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39301a1940 con 0x7f39301039c0 2024-04-25T01:13:29.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.679+0000 7f392f400640 1 --2- 172.21.15.31:0/2572078141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39301039c0 0x7f3930199d30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f39200305b0 tx=0x7f3920033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:29.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.679+0000 7f3927e00640 1 -- 172.21.15.31:0/2572078141 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392003f020 con 0x7f39301039c0 2024-04-25T01:13:29.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.679+0000 7f3936800640 1 -- 172.21.15.31:0/2572078141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39301a1bd0 con 0x7f39301039c0 2024-04-25T01:13:29.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.679+0000 7f3927e00640 1 -- 172.21.15.31:0/2572078141 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3920030850 con 0x7f39301039c0 2024-04-25T01:13:29.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.679+0000 7f3936800640 1 -- 172.21.15.31:0/2572078141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39301a2110 con 0x7f39301039c0 2024-04-25T01:13:29.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.680+0000 7f3927e00640 1 -- 172.21.15.31:0/2572078141 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920037040 con 0x7f39301039c0 2024-04-25T01:13:29.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.681+0000 7f3936800640 1 -- 172.21.15.31:0/2572078141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f393010ad00 con 0x7f39301039c0 2024-04-25T01:13:29.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.683+0000 7f3927e00640 1 -- 172.21.15.31:0/2572078141 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3920042020 con 0x7f39301039c0 2024-04-25T01:13:29.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.684+0000 7f3927e00640 1 --2- 172.21.15.31:0/2572078141 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f39140a2b90 0x7f39140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:29.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.685+0000 7f3927e00640 1 -- 172.21.15.31:0/2572078141 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7497+0+0 (secure 0 0 0) 0x7f39200ef0c0 con 0x7f39301039c0 2024-04-25T01:13:29.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.685+0000 7f392ea00640 1 --2- 172.21.15.31:0/2572078141 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f39140a2b90 0x7f39140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:29.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.685+0000 7f392ea00640 1 --2- 172.21.15.31:0/2572078141 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f39140a2b90 0x7f39140a5050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f391c009530 tx=0x7f391c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:29.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.692+0000 7f3927e00640 1 -- 172.21.15.31:0/2572078141 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3920049020 con 0x7f39301039c0 2024-04-25T01:13:29.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:29 smithi169 ceph-mon[32759]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:29.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:29 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2071262930' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T01:13:29.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:29 smithi169 ceph-mon[32759]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T01:13:29.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:29 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:29.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:29 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:29.696 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:29 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:29.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:29 smithi040 ceph-mon[32518]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:29.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:29 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2071262930' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T01:13:29.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:29 smithi040 ceph-mon[32518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T01:13:29.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:29 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:29.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:29 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:29.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:29 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:29.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:29 smithi031 ceph-mon[28053]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:29.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:29 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2071262930' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T01:13:29.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:29 smithi031 ceph-mon[28053]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T01:13:29.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:29 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:29.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:29 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:29.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:29 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:29.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:29.910+0000 7f3936800640 1 -- 172.21.15.31:0/2572078141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f393002d170 con 0x7f39301039c0 2024-04-25T01:13:30.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.493+0000 7f3927e00640 1 -- 172.21.15.31:0/2572078141 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) ==== 170+0+0 (secure 0 0 0) 0x7f39200bc2b0 con 0x7f39301039c0 2024-04-25T01:13:30.481 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2024-04-25T01:13:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.497+0000 7f3936800640 1 -- 172.21.15.31:0/2572078141 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f39140a2b90 msgr2=0x7f39140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.497+0000 7f3936800640 1 --2- 172.21.15.31:0/2572078141 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f39140a2b90 0x7f39140a5050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f391c009530 tx=0x7f391c009260 comp rx=0 tx=0).stop 2024-04-25T01:13:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.497+0000 7f3936800640 1 -- 172.21.15.31:0/2572078141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39301039c0 msgr2=0x7f3930199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.497+0000 7f3936800640 1 --2- 172.21.15.31:0/2572078141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39301039c0 0x7f3930199d30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f39200305b0 tx=0x7f3920033d90 comp rx=0 tx=0).stop 2024-04-25T01:13:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.498+0000 7f3936800640 1 -- 172.21.15.31:0/2572078141 shutdown_connections 2024-04-25T01:13:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.498+0000 7f3936800640 1 --2- 172.21.15.31:0/2572078141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3930105570 0x7f39301a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.498+0000 7f3936800640 1 --2- 172.21.15.31:0/2572078141 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f39140a2b90 0x7f39140a5050 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.498+0000 7f3936800640 1 --2- 172.21.15.31:0/2572078141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3930104bb0 0x7f393019a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.498+0000 7f3936800640 1 --2- 172.21.15.31:0/2572078141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39301039c0 0x7f3930199d30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.498+0000 7f3936800640 1 -- 172.21.15.31:0/2572078141 >> 172.21.15.31:0/2572078141 conn(0x7f39300fef30 msgr2=0x7f3930107ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:30.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.499+0000 7f3936800640 1 -- 172.21.15.31:0/2572078141 shutdown_connections 2024-04-25T01:13:30.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.499+0000 7f3936800640 1 -- 172.21.15.31:0/2572078141 wait complete. 2024-04-25T01:13:30.519 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T01:13:30.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.714+0000 7f0491a00640 1 -- 172.21.15.31:0/2230952029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f048c075780 msgr2=0x7f048c073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:30.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.714+0000 7f0491a00640 1 --2- 172.21.15.31:0/2230952029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f048c075780 0x7f048c073bd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f047c009900 tx=0x7f047c02a3f0 comp rx=0 tx=0).stop 2024-04-25T01:13:30.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.714+0000 7f0491a00640 1 -- 172.21.15.31:0/2230952029 shutdown_connections 2024-04-25T01:13:30.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.714+0000 7f0491a00640 1 --2- 172.21.15.31:0/2230952029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c10dca0 0x7f048c110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:30.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.714+0000 7f0491a00640 1 --2- 172.21.15.31:0/2230952029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f048c0741a0 0x7f048c074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:30.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.714+0000 7f0491a00640 1 --2- 172.21.15.31:0/2230952029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f048c075780 0x7f048c073bd0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:30.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.714+0000 7f0491a00640 1 -- 172.21.15.31:0/2230952029 >> 172.21.15.31:0/2230952029 conn(0x7f048c0fcc40 msgr2=0x7f048c0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:30.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.714+0000 7f0491a00640 1 -- 172.21.15.31:0/2230952029 shutdown_connections 2024-04-25T01:13:30.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.714+0000 7f0491a00640 1 -- 172.21.15.31:0/2230952029 wait complete. 2024-04-25T01:13:30.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.716+0000 7f0491a00640 1 Processor -- start 2024-04-25T01:13:30.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.716+0000 7f0491a00640 1 -- start start 2024-04-25T01:13:30.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.716+0000 7f0491a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f048c0741a0 0x7f048c19df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:30.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.716+0000 7f0491a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c075780 0x7f048c19e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:30.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.716+0000 7f0491a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f048c10dca0 0x7f048c1a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:30.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.716+0000 7f0491a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f048c113900 con 0x7f048c075780 2024-04-25T01:13:30.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.716+0000 7f0491a00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f048c113780 con 0x7f048c0741a0 2024-04-25T01:13:30.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.716+0000 7f0491a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f048c113a80 con 0x7f048c10dca0 2024-04-25T01:13:30.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.717+0000 7f048be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f048c0741a0 0x7f048c19df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:30.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.717+0000 7f048be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f048c0741a0 0x7f048c19df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:44778/0 (socket says 172.21.15.31:44778) 2024-04-25T01:13:30.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.717+0000 7f048be00640 1 -- 172.21.15.31:0/3631613721 learned_addr learned my addr 172.21.15.31:0/3631613721 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:30.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.717+0000 7f048b400640 1 --2- 172.21.15.31:0/3631613721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c075780 0x7f048c19e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:30.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.717+0000 7f0491000640 1 --2- 172.21.15.31:0/3631613721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f048c10dca0 0x7f048c1a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:30.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.717+0000 7f048be00640 1 -- 172.21.15.31:0/3631613721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f048c10dca0 msgr2=0x7f048c1a5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:30.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.717+0000 7f048be00640 1 --2- 172.21.15.31:0/3631613721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f048c10dca0 0x7f048c1a5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:30.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.717+0000 7f048be00640 1 -- 172.21.15.31:0/3631613721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c075780 msgr2=0x7f048c19e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:30.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.717+0000 7f048be00640 1 --2- 172.21.15.31:0/3631613721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c075780 0x7f048c19e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:30.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.717+0000 7f048be00640 1 -- 172.21.15.31:0/3631613721 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f048c1a5b90 con 0x7f048c0741a0 2024-04-25T01:13:30.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.717+0000 7f048be00640 1 --2- 172.21.15.31:0/3631613721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f048c0741a0 0x7f048c19df70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f047c0315a0 tx=0x7f047c032c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:30.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.717+0000 7f0488c00640 1 -- 172.21.15.31:0/3631613721 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c03f030 con 0x7f048c0741a0 2024-04-25T01:13:30.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.717+0000 7f0488c00640 1 -- 172.21.15.31:0/3631613721 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f047c003ce0 con 0x7f048c0741a0 2024-04-25T01:13:30.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.718+0000 7f0491a00640 1 -- 172.21.15.31:0/3631613721 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f048c1a5e20 con 0x7f048c0741a0 2024-04-25T01:13:30.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.718+0000 7f0488c00640 1 -- 172.21.15.31:0/3631613721 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c035070 con 0x7f048c0741a0 2024-04-25T01:13:30.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.718+0000 7f0491a00640 1 -- 172.21.15.31:0/3631613721 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f048c1a6280 con 0x7f048c0741a0 2024-04-25T01:13:30.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.719+0000 7f0491a00640 1 -- 172.21.15.31:0/3631613721 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f048c103e70 con 0x7f048c0741a0 2024-04-25T01:13:30.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.721+0000 7f0488c00640 1 -- 172.21.15.31:0/3631613721 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f047c042030 con 0x7f048c0741a0 2024-04-25T01:13:30.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.722+0000 7f0488c00640 1 --2- 172.21.15.31:0/3631613721 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f04700a2c10 0x7f04700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:30.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.722+0000 7f0488c00640 1 -- 172.21.15.31:0/3631613721 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7879+0+0 (secure 0 0 0) 0x7f047c0ef5b0 con 0x7f048c0741a0 2024-04-25T01:13:30.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.722+0000 7f048b400640 1 --2- 172.21.15.31:0/3631613721 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f04700a2c10 0x7f04700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:30.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.723+0000 7f048b400640 1 --2- 172.21.15.31:0/3631613721 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f04700a2c10 0x7f04700a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f04740095f0 tx=0x7f0474005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:30.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.726+0000 7f0488c00640 1 -- 172.21.15.31:0/3631613721 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f047c0bc620 con 0x7f048c0741a0 2024-04-25T01:13:30.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:30 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:30.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:30 smithi031 ceph-mon[28053]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T01:13:30.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:30 smithi031 ceph-mon[28053]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T01:13:30.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:30 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:30.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:30 smithi169 ceph-mon[32759]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T01:13:30.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:30 smithi169 ceph-mon[32759]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T01:13:30.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:30 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:30.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:30 smithi040 ceph-mon[32518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T01:13:30.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:30 smithi040 ceph-mon[32518]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T01:13:30.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:30.972+0000 7f0491a00640 1 -- 172.21.15.31:0/3631613721 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f048c062450 con 0x7f048c0741a0 2024-04-25T01:13:31.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.552+0000 7f0488c00640 1 -- 172.21.15.31:0/3631613721 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f047c0bbf20 con 0x7f048c0741a0 2024-04-25T01:13:31.539 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T01:13:31.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.556+0000 7f0491a00640 1 -- 172.21.15.31:0/3631613721 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f04700a2c10 msgr2=0x7f04700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:31.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.556+0000 7f0491a00640 1 --2- 172.21.15.31:0/3631613721 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f04700a2c10 0x7f04700a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f04740095f0 tx=0x7f0474005f90 comp rx=0 tx=0).stop 2024-04-25T01:13:31.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.556+0000 7f0491a00640 1 -- 172.21.15.31:0/3631613721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f048c0741a0 msgr2=0x7f048c19df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:31.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.556+0000 7f0491a00640 1 --2- 172.21.15.31:0/3631613721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f048c0741a0 0x7f048c19df70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f047c0315a0 tx=0x7f047c032c70 comp rx=0 tx=0).stop 2024-04-25T01:13:31.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.556+0000 7f0491a00640 1 -- 172.21.15.31:0/3631613721 shutdown_connections 2024-04-25T01:13:31.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.556+0000 7f0491a00640 1 --2- 172.21.15.31:0/3631613721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f048c10dca0 0x7f048c1a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:31.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.557+0000 7f0491a00640 1 --2- 172.21.15.31:0/3631613721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f048c075780 0x7f048c19e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:31.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.557+0000 7f0491a00640 1 --2- 172.21.15.31:0/3631613721 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f04700a2c10 0x7f04700a50d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:31.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.557+0000 7f0491a00640 1 --2- 172.21.15.31:0/3631613721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f048c0741a0 0x7f048c19df70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:31.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.557+0000 7f0491a00640 1 -- 172.21.15.31:0/3631613721 >> 172.21.15.31:0/3631613721 conn(0x7f048c0fcc40 msgr2=0x7f048c101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:31.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.557+0000 7f0491a00640 1 -- 172.21.15.31:0/3631613721 shutdown_connections 2024-04-25T01:13:31.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.557+0000 7f0491a00640 1 -- 172.21.15.31:0/3631613721 wait complete. 2024-04-25T01:13:31.565 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:13:31.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.759+0000 7f4ebbe00640 1 -- 172.21.15.31:0/579280832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ebc101130 msgr2=0x7f4ebc1015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:31.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.759+0000 7f4ebbe00640 1 --2- 172.21.15.31:0/579280832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ebc101130 0x7f4ebc1015b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009900 tx=0x7f4eb402a3f0 comp rx=0 tx=0).stop 2024-04-25T01:13:31.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.759+0000 7f4ebbe00640 1 -- 172.21.15.31:0/579280832 shutdown_connections 2024-04-25T01:13:31.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.759+0000 7f4ebbe00640 1 --2- 172.21.15.31:0/579280832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ebc101130 0x7f4ebc1015b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:31.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.759+0000 7f4ebbe00640 1 --2- 172.21.15.31:0/579280832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ebc100790 0x7f4ebc100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:31.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.759+0000 7f4ebbe00640 1 --2- 172.21.15.31:0/579280832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ebc0ffe30 0x7f4ebc100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:31.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.759+0000 7f4ebbe00640 1 -- 172.21.15.31:0/579280832 >> 172.21.15.31:0/579280832 conn(0x7f4ebc0fba10 msgr2=0x7f4ebc0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:31.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.759+0000 7f4ebbe00640 1 -- 172.21.15.31:0/579280832 shutdown_connections 2024-04-25T01:13:31.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.759+0000 7f4ebbe00640 1 -- 172.21.15.31:0/579280832 wait complete. 2024-04-25T01:13:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.760+0000 7f4ebbe00640 1 Processor -- start 2024-04-25T01:13:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.760+0000 7f4ebbe00640 1 -- start start 2024-04-25T01:13:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.760+0000 7f4ebbe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ebc0ffe30 0x7f4ebc199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.760+0000 7f4ebbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ebc100790 0x7f4ebc19a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.760+0000 7f4ebbe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ebc101130 0x7f4ebc1a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.760+0000 7f4ebbe00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4ebc10f480 con 0x7f4ebc101130 2024-04-25T01:13:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.760+0000 7f4ebbe00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4ebc10f300 con 0x7f4ebc0ffe30 2024-04-25T01:13:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.760+0000 7f4ebbe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4ebc10f600 con 0x7f4ebc100790 2024-04-25T01:13:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.761+0000 7f4ebb400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ebc101130 0x7f4ebc1a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.761+0000 7f4ebb400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ebc101130 0x7f4ebc1a1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43472/0 (socket says 172.21.15.31:43472) 2024-04-25T01:13:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.761+0000 7f4ebb400640 1 -- 172.21.15.31:0/2514993231 learned_addr learned my addr 172.21.15.31:0/2514993231 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:31.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.761+0000 7f4ebaa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ebc0ffe30 0x7f4ebc199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:31.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.761+0000 7f4eb2000640 1 --2- 172.21.15.31:0/2514993231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ebc100790 0x7f4ebc19a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:31.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.761+0000 7f4ebaa00640 1 -- 172.21.15.31:0/2514993231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ebc100790 msgr2=0x7f4ebc19a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:31.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.761+0000 7f4ebaa00640 1 --2- 172.21.15.31:0/2514993231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ebc100790 0x7f4ebc19a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:31.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.761+0000 7f4ebaa00640 1 -- 172.21.15.31:0/2514993231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ebc101130 msgr2=0x7f4ebc1a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:31.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.761+0000 7f4ebaa00640 1 --2- 172.21.15.31:0/2514993231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ebc101130 0x7f4ebc1a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:31.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.761+0000 7f4ebaa00640 1 -- 172.21.15.31:0/2514993231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ebc1a1690 con 0x7f4ebc0ffe30 2024-04-25T01:13:31.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.761+0000 7f4ebb400640 1 --2- 172.21.15.31:0/2514993231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ebc101130 0x7f4ebc1a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:13:31.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.761+0000 7f4ebaa00640 1 --2- 172.21.15.31:0/2514993231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ebc0ffe30 0x7f4ebc199ac0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00e9f0 tx=0x7f4eac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:31.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.762+0000 7f4eb3e00640 1 -- 172.21.15.31:0/2514993231 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac00ccc0 con 0x7f4ebc0ffe30 2024-04-25T01:13:31.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.762+0000 7f4eb3e00640 1 -- 172.21.15.31:0/2514993231 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eac005020 con 0x7f4ebc0ffe30 2024-04-25T01:13:31.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.762+0000 7f4ebbe00640 1 -- 172.21.15.31:0/2514993231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ebc1a1980 con 0x7f4ebc0ffe30 2024-04-25T01:13:31.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.762+0000 7f4eb3e00640 1 -- 172.21.15.31:0/2514993231 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac0049e0 con 0x7f4ebc0ffe30 2024-04-25T01:13:31.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.762+0000 7f4ebbe00640 1 -- 172.21.15.31:0/2514993231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ebc1a1e80 con 0x7f4ebc0ffe30 2024-04-25T01:13:31.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.763+0000 7f4ebbe00640 1 -- 172.21.15.31:0/2514993231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ebc10ab80 con 0x7f4ebc0ffe30 2024-04-25T01:13:31.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.765+0000 7f4eb3e00640 1 -- 172.21.15.31:0/2514993231 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4eac005560 con 0x7f4ebc0ffe30 2024-04-25T01:13:31.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.767+0000 7f4eb3e00640 1 --2- 172.21.15.31:0/2514993231 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f4ea00a2b90 0x7f4ea00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:31.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.767+0000 7f4eb3e00640 1 -- 172.21.15.31:0/2514993231 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f4eac0ceef0 con 0x7f4ebc0ffe30 2024-04-25T01:13:31.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.767+0000 7f4eb2000640 1 --2- 172.21.15.31:0/2514993231 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f4ea00a2b90 0x7f4ea00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:31.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.768+0000 7f4eb2000640 1 --2- 172.21.15.31:0/2514993231 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f4ea00a2b90 0x7f4ea00a5050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4ea40099a0 tx=0x7f4ea4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:31.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:31 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2572078141' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T01:13:31.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:31 smithi031 ceph-mon[28053]: pgmap v313: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:31.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:31 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2572078141' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T01:13:31.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:31 smithi031 ceph-mon[28053]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T01:13:31.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:31 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:31.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:31 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:31.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:31 smithi031 ceph-mon[28053]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T01:13:31.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:31 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:31.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:31 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:13:31.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:31 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:31.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:31 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3631613721' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T01:13:31.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:31 smithi031 ceph-mon[28053]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T01:13:31.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:31 smithi031 ceph-mon[28053]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T01:13:31.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.769+0000 7f4eb3e00640 1 -- 172.21.15.31:0/2514993231 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4eac0051d0 con 0x7f4ebc0ffe30 2024-04-25T01:13:31.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:31 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2572078141' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T01:13:31.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:31 smithi169 ceph-mon[32759]: pgmap v313: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:31.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:31 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2572078141' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T01:13:31.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:31 smithi169 ceph-mon[32759]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T01:13:31.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:31 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:31.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:31 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:31.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:31 smithi169 ceph-mon[32759]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T01:13:31.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:31 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:31.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:31 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:13:31.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:31 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:31.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:31 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3631613721' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T01:13:31.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:31 smithi169 ceph-mon[32759]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T01:13:31.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:31 smithi169 ceph-mon[32759]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T01:13:31.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:31 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2572078141' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T01:13:31.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:31 smithi040 ceph-mon[32518]: pgmap v313: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:13:31.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:31 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2572078141' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T01:13:31.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:31 smithi040 ceph-mon[32518]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T01:13:31.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:31 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:31.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:31 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:31.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:31 smithi040 ceph-mon[32518]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T01:13:31.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:31 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:13:31.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:31 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:13:31.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:31 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:13:31.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:31 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3631613721' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T01:13:31.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:31 smithi040 ceph-mon[32518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T01:13:31.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:31 smithi040 ceph-mon[32518]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T01:13:31.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.976+0000 7f4ebbe00640 1 -- 172.21.15.31:0/2514993231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ebc02d340 con 0x7f4ebc0ffe30 2024-04-25T01:13:31.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.978+0000 7f4eb3e00640 1 -- 172.21.15.31:0/2514993231 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22185 (secure 0 0 0) 0x7f4eac09bf00 con 0x7f4ebc0ffe30 2024-04-25T01:13:31.965 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:13:31.966 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":86,"fsid":"95849b94-029f-11ef-bc93-c7b262605968","created":"2024-04-25T01:04:23.310856+0000","modified":"2024-04-25T01:13:30.975448+0000","last_up_change":"2024-04-25T01:12:29.393907+0000","last_in_change":"2024-04-25T01:12:08.978148+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:18.662391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:13:29.258771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:13:30.480996+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa08e4c-fe6b-4116-bdf5-1adf5b465d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6803","nonce":788174091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6805","nonce":788174091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6809","nonce":788174091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6807","nonce":788174091}]},"public_addr":"172.21.15.31:6803/788174091","cluster_addr":"172.21.15.31:6805/788174091","heartbeat_back_addr":"172.21.15.31:6809/788174091","heartbeat_front_addr":"172.21.15.31:6807/788174091","state":["exists","up"]},{"osd":1,"uuid":"85339fe7-0dbc-4485-9334-2b039993c1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6801","nonce":3196394509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6803","nonce":3196394509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6807","nonce":3196394509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6805","nonce":3196394509}]},"public_addr":"172.21.15.40:6801/3196394509","cluster_addr":"172.21.15.40:6803/3196394509","heartbeat_back_addr":"172.21.15.40:6807/3196394509","heartbeat_front_addr":"172.21.15.40:6805/3196394509","state":["exists","up"]},{"osd":2,"uuid":"ab62ecb2-8aae-4987-9e2e-589e0f2fee5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6801","nonce":1955211197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6803","nonce":1955211197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6807","nonce":1955211197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6805","nonce":1955211197}]},"public_addr":"172.21.15.169:6801/1955211197","cluster_addr":"172.21.15.169:6803/1955211197","heartbeat_back_addr":"172.21.15.169:6807/1955211197","heartbeat_front_addr":"172.21.15.169:6805/1955211197","state":["exists","up"]},{"osd":3,"uuid":"8b5a7881-3401-495c-8ca4-c90685919d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6811","nonce":1228688643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6813","nonce":1228688643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6817","nonce":1228688643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6815","nonce":1228688643}]},"public_addr":"172.21.15.31:6811/1228688643","cluster_addr":"172.21.15.31:6813/1228688643","heartbeat_back_addr":"172.21.15.31:6817/1228688643","heartbeat_front_addr":"172.21.15.31:6815/1228688643","state":["exists","up"]},{"osd":4,"uuid":"56deb61d-a334-4cf2-8ac7-fbb6e6947092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6809","nonce":1913779091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6811","nonce":1913779091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6815","nonce":1913779091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6813","nonce":1913779091}]},"public_addr":"172.21.15.40:6809/1913779091","cluster_addr":"172.21.15.40:6811/1913779091","heartbeat_back_addr":"172.21.15.40:6815/1913779091","heartbeat_front_addr":"172.21.15.40:6813/1913779091","state":["exists","up"]},{"osd":5,"uuid":"8da55693-f9c7-49a7-99e0-165078f8001b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6809","nonce":2549650781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6811","nonce":2549650781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6815","nonce":2549650781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6813","nonce":2549650781}]},"public_addr":"172.21.15.169:6809/2549650781","cluster_addr":"172.21.15.169:6811/2549650781","heartbeat_back_addr":"172.21.15.169:6815/2549650781","heartbeat_front_addr":"172.21.15.169:6813/2549650781","state":["exists","up"]},{"osd":6,"uuid":"fc8350d2-1ad6-43cb-89f4-d336afc99ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6819","nonce":611507624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6821","nonce":611507624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6825","nonce":611507624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6823","nonce":611507624}]},"public_addr":"172.21.15.31:6819/611507624","cluster_addr":"172.21.15.31:6821/611507624","heartbeat_back_addr":"172.21.15.31:6825/611507624","heartbeat_front_addr":"172.21.15.31:6823/611507624","state":["exists","up"]},{"osd":7,"uuid":"3121b58b-8c94-40b5-9d48-f56552d7c39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6817","nonce":3776085849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6819","nonce":3776085849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6823","nonce":3776085849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6821","nonce":3776085849}]},"public_addr":"172.21.15.40:6817/3776085849","cluster_addr":"172.21.15.40:6819/3776085849","heartbeat_back_addr":"172.21.15.40:6823/3776085849","heartbeat_front_addr":"172.21.15.40:6821/3776085849","state":["exists","up"]},{"osd":8,"uuid":"f60407df-91f5-477a-9702-ddaef24faf82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6817","nonce":2736546122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6819","nonce":2736546122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6823","nonce":2736546122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6821","nonce":2736546122}]},"public_addr":"172.21.15.169:6817/2736546122","cluster_addr":"172.21.15.169:6819/2736546122","heartbeat_back_addr":"172.21.15.169:6823/2736546122","heartbeat_front_addr":"172.21.15.169:6821/2736546122","state":["exists","up"]},{"osd":9,"uuid":"7506a407-1960-49bc-9e38-7f2a8bf996ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6827","nonce":1187207880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6829","nonce":1187207880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6833","nonce":1187207880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6831","nonce":1187207880}]},"public_addr":"172.21.15.31:6827/1187207880","cluster_addr":"172.21.15.31:6829/1187207880","heartbeat_back_addr":"172.21.15.31:6833/1187207880","heartbeat_front_addr":"172.21.15.31:6831/1187207880","state":["exists","up"]},{"osd":10,"uuid":"831c48e2-b0d0-451f-959b-228e927542cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6825","nonce":3050148001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6827","nonce":3050148001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6831","nonce":3050148001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6829","nonce":3050148001}]},"public_addr":"172.21.15.40:6825/3050148001","cluster_addr":"172.21.15.40:6827/3050148001","heartbeat_back_addr":"172.21.15.40:6831/3050148001","heartbeat_front_addr":"172.21.15.40:6829/3050148001","state":["exists","up"]},{"osd":11,"uuid":"144fe0aa-f757-4b3c-90ad-99775e17d57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6825","nonce":262401944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6827","nonce":262401944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6831","nonce":262401944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6829","nonce":262401944}]},"public_addr":"172.21.15.169:6825/262401944","cluster_addr":"172.21.15.169:6827/262401944","heartbeat_back_addr":"172.21.15.169:6831/262401944","heartbeat_front_addr":"172.21.15.169:6829/262401944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:25.640995+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:51.036909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.635633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:48.467912+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:14.454971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:37.609315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:12.441807+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:38.239215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:02.230758+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:35.762553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:02.051405+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:26.569593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6801/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4004014066":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/1930101646":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/2638309002":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2238638505":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2902841144":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4190696323":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/480558521":"2024-04-26T01:04:57.149272+0000","172.21.15.31:6801/2638309002":"2024-04-26T01:04:57.149272+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:13:31.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.981+0000 7f4ebbe00640 1 -- 172.21.15.31:0/2514993231 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f4ea00a2b90 msgr2=0x7f4ea00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:31.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.981+0000 7f4ebbe00640 1 --2- 172.21.15.31:0/2514993231 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f4ea00a2b90 0x7f4ea00a5050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4ea40099a0 tx=0x7f4ea4009500 comp rx=0 tx=0).stop 2024-04-25T01:13:31.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.981+0000 7f4ebbe00640 1 -- 172.21.15.31:0/2514993231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ebc0ffe30 msgr2=0x7f4ebc199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:31.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.981+0000 7f4ebbe00640 1 --2- 172.21.15.31:0/2514993231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ebc0ffe30 0x7f4ebc199ac0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00e9f0 tx=0x7f4eac00eeb0 comp rx=0 tx=0).stop 2024-04-25T01:13:31.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.982+0000 7f4ebbe00640 1 -- 172.21.15.31:0/2514993231 shutdown_connections 2024-04-25T01:13:31.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.982+0000 7f4ebbe00640 1 --2- 172.21.15.31:0/2514993231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ebc101130 0x7f4ebc1a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:31.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.982+0000 7f4ebbe00640 1 --2- 172.21.15.31:0/2514993231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ebc100790 0x7f4ebc19a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:31.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.982+0000 7f4ebbe00640 1 --2- 172.21.15.31:0/2514993231 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f4ea00a2b90 0x7f4ea00a5050 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:31.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.982+0000 7f4ebbe00640 1 --2- 172.21.15.31:0/2514993231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ebc0ffe30 0x7f4ebc199ac0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:31.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.982+0000 7f4ebbe00640 1 -- 172.21.15.31:0/2514993231 >> 172.21.15.31:0/2514993231 conn(0x7f4ebc0fba10 msgr2=0x7f4ebc0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:31.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.982+0000 7f4ebbe00640 1 -- 172.21.15.31:0/2514993231 shutdown_connections 2024-04-25T01:13:31.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:31.982+0000 7f4ebbe00640 1 -- 172.21.15.31:0/2514993231 wait complete. 2024-04-25T01:13:31.988 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:13:32.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.204+0000 7f1d9f200640 1 -- 172.21.15.31:0/1143895194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1da0109770 msgr2=0x7f1da0110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:32.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.204+0000 7f1d9f200640 1 --2- 172.21.15.31:0/1143895194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1da0109770 0x7f1da0110000 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009be0 tx=0x7f1d9802a6c0 comp rx=0 tx=0).stop 2024-04-25T01:13:32.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.204+0000 7f1d9f200640 1 -- 172.21.15.31:0/1143895194 shutdown_connections 2024-04-25T01:13:32.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.204+0000 7f1d9f200640 1 --2- 172.21.15.31:0/1143895194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1da0109770 0x7f1da0110000 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:32.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.204+0000 7f1d9f200640 1 --2- 172.21.15.31:0/1143895194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1da0108db0 0x7f1da0109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:32.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.204+0000 7f1d9f200640 1 --2- 172.21.15.31:0/1143895194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da0107bc0 0x7f1da0107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:32.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.204+0000 7f1d9f200640 1 -- 172.21.15.31:0/1143895194 >> 172.21.15.31:0/1143895194 conn(0x7f1da0076b80 msgr2=0x7f1da0078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:32.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.204+0000 7f1d9f200640 1 -- 172.21.15.31:0/1143895194 shutdown_connections 2024-04-25T01:13:32.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.204+0000 7f1d9f200640 1 -- 172.21.15.31:0/1143895194 wait complete. 2024-04-25T01:13:32.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.205+0000 7f1d9f200640 1 Processor -- start 2024-04-25T01:13:32.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.206+0000 7f1d9f200640 1 -- start start 2024-04-25T01:13:32.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.206+0000 7f1d9f200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da0107bc0 0x7f1da019de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:32.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.206+0000 7f1d9f200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1da0108db0 0x7f1da019e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:32.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.206+0000 7f1d9f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1da0109770 0x7f1da01a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:32.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.206+0000 7f1d9f200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1da0113820 con 0x7f1da0107bc0 2024-04-25T01:13:32.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.206+0000 7f1d9f200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01136a0 con 0x7f1da0108db0 2024-04-25T01:13:32.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.206+0000 7f1d9f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01139a0 con 0x7f1da0109770 2024-04-25T01:13:32.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.206+0000 7f1d9de00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da0107bc0 0x7f1da019de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:32.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.206+0000 7f1d97e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1da0108db0 0x7f1da019e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:32.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.207+0000 7f1d9de00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da0107bc0 0x7f1da019de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43498/0 (socket says 172.21.15.31:43498) 2024-04-25T01:13:32.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.207+0000 7f1d9de00640 1 -- 172.21.15.31:0/136723451 learned_addr learned my addr 172.21.15.31:0/136723451 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:32.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.207+0000 7f1d9e800640 1 --2- 172.21.15.31:0/136723451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1da0109770 0x7f1da01a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:32.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.207+0000 7f1d97e00640 1 -- 172.21.15.31:0/136723451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1da0109770 msgr2=0x7f1da01a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:32.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.207+0000 7f1d97e00640 1 --2- 172.21.15.31:0/136723451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1da0109770 0x7f1da01a5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.207+0000 7f1d97e00640 1 -- 172.21.15.31:0/136723451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da0107bc0 msgr2=0x7f1da019de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.207+0000 7f1d97e00640 1 --2- 172.21.15.31:0/136723451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da0107bc0 0x7f1da019de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.207+0000 7f1d97e00640 1 -- 172.21.15.31:0/136723451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da01a5a80 con 0x7f1da0108db0 2024-04-25T01:13:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.207+0000 7f1d9de00640 1 --2- 172.21.15.31:0/136723451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da0107bc0 0x7f1da019de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:13:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.207+0000 7f1d97e00640 1 --2- 172.21.15.31:0/136723451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1da0108db0 0x7f1da019e3a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1d88005fd0 tx=0x7f1d8800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.208+0000 7f1d96a00640 1 -- 172.21.15.31:0/136723451 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88009c30 con 0x7f1da0108db0 2024-04-25T01:13:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.208+0000 7f1d96a00640 1 -- 172.21.15.31:0/136723451 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d88009dd0 con 0x7f1da0108db0 2024-04-25T01:13:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.208+0000 7f1d96a00640 1 -- 172.21.15.31:0/136723451 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88010460 con 0x7f1da0108db0 2024-04-25T01:13:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.208+0000 7f1d9f200640 1 -- 172.21.15.31:0/136723451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da01a5d70 con 0x7f1da0108db0 2024-04-25T01:13:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.208+0000 7f1d9f200640 1 -- 172.21.15.31:0/136723451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da01a6260 con 0x7f1da0108db0 2024-04-25T01:13:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.209+0000 7f1d9f200640 1 -- 172.21.15.31:0/136723451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da010ee80 con 0x7f1da0108db0 2024-04-25T01:13:32.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.211+0000 7f1d96a00640 1 -- 172.21.15.31:0/136723451 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1d88003ef0 con 0x7f1da0108db0 2024-04-25T01:13:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.213+0000 7f1d96a00640 1 --2- 172.21.15.31:0/136723451 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1d840a2b90 0x7f1d840a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.213+0000 7f1d96a00640 1 -- 172.21.15.31:0/136723451 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f1d880cea70 con 0x7f1da0108db0 2024-04-25T01:13:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.213+0000 7f1d9de00640 1 --2- 172.21.15.31:0/136723451 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1d840a2b90 0x7f1d840a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.214+0000 7f1d9de00640 1 --2- 172.21.15.31:0/136723451 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1d840a2b90 0x7f1d840a5050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1d90011490 tx=0x7f1d90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:32.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.215+0000 7f1d96a00640 1 -- 172.21.15.31:0/136723451 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1d88017020 con 0x7f1da0108db0 2024-04-25T01:13:32.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.423+0000 7f1d9f200640 1 -- 172.21.15.31:0/136723451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1da002d340 con 0x7f1da0108db0 2024-04-25T01:13:32.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.426+0000 7f1d96a00640 1 -- 172.21.15.31:0/136723451 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22185 (secure 0 0 0) 0x7f1d8809b430 con 0x7f1da0108db0 2024-04-25T01:13:32.413 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:13:32.413 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":86,"fsid":"95849b94-029f-11ef-bc93-c7b262605968","created":"2024-04-25T01:04:23.310856+0000","modified":"2024-04-25T01:13:30.975448+0000","last_up_change":"2024-04-25T01:12:29.393907+0000","last_in_change":"2024-04-25T01:12:08.978148+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:18.662391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:13:29.258771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:13:30.480996+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa08e4c-fe6b-4116-bdf5-1adf5b465d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6803","nonce":788174091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6805","nonce":788174091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6809","nonce":788174091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6807","nonce":788174091}]},"public_addr":"172.21.15.31:6803/788174091","cluster_addr":"172.21.15.31:6805/788174091","heartbeat_back_addr":"172.21.15.31:6809/788174091","heartbeat_front_addr":"172.21.15.31:6807/788174091","state":["exists","up"]},{"osd":1,"uuid":"85339fe7-0dbc-4485-9334-2b039993c1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6801","nonce":3196394509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6803","nonce":3196394509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6807","nonce":3196394509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6805","nonce":3196394509}]},"public_addr":"172.21.15.40:6801/3196394509","cluster_addr":"172.21.15.40:6803/3196394509","heartbeat_back_addr":"172.21.15.40:6807/3196394509","heartbeat_front_addr":"172.21.15.40:6805/3196394509","state":["exists","up"]},{"osd":2,"uuid":"ab62ecb2-8aae-4987-9e2e-589e0f2fee5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6801","nonce":1955211197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6803","nonce":1955211197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6807","nonce":1955211197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6805","nonce":1955211197}]},"public_addr":"172.21.15.169:6801/1955211197","cluster_addr":"172.21.15.169:6803/1955211197","heartbeat_back_addr":"172.21.15.169:6807/1955211197","heartbeat_front_addr":"172.21.15.169:6805/1955211197","state":["exists","up"]},{"osd":3,"uuid":"8b5a7881-3401-495c-8ca4-c90685919d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6811","nonce":1228688643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6813","nonce":1228688643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6817","nonce":1228688643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6815","nonce":1228688643}]},"public_addr":"172.21.15.31:6811/1228688643","cluster_addr":"172.21.15.31:6813/1228688643","heartbeat_back_addr":"172.21.15.31:6817/1228688643","heartbeat_front_addr":"172.21.15.31:6815/1228688643","state":["exists","up"]},{"osd":4,"uuid":"56deb61d-a334-4cf2-8ac7-fbb6e6947092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6809","nonce":1913779091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6811","nonce":1913779091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6815","nonce":1913779091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6813","nonce":1913779091}]},"public_addr":"172.21.15.40:6809/1913779091","cluster_addr":"172.21.15.40:6811/1913779091","heartbeat_back_addr":"172.21.15.40:6815/1913779091","heartbeat_front_addr":"172.21.15.40:6813/1913779091","state":["exists","up"]},{"osd":5,"uuid":"8da55693-f9c7-49a7-99e0-165078f8001b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6809","nonce":2549650781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6811","nonce":2549650781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6815","nonce":2549650781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6813","nonce":2549650781}]},"public_addr":"172.21.15.169:6809/2549650781","cluster_addr":"172.21.15.169:6811/2549650781","heartbeat_back_addr":"172.21.15.169:6815/2549650781","heartbeat_front_addr":"172.21.15.169:6813/2549650781","state":["exists","up"]},{"osd":6,"uuid":"fc8350d2-1ad6-43cb-89f4-d336afc99ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6819","nonce":611507624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6821","nonce":611507624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6825","nonce":611507624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6823","nonce":611507624}]},"public_addr":"172.21.15.31:6819/611507624","cluster_addr":"172.21.15.31:6821/611507624","heartbeat_back_addr":"172.21.15.31:6825/611507624","heartbeat_front_addr":"172.21.15.31:6823/611507624","state":["exists","up"]},{"osd":7,"uuid":"3121b58b-8c94-40b5-9d48-f56552d7c39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6817","nonce":3776085849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6819","nonce":3776085849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6823","nonce":3776085849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6821","nonce":3776085849}]},"public_addr":"172.21.15.40:6817/3776085849","cluster_addr":"172.21.15.40:6819/3776085849","heartbeat_back_addr":"172.21.15.40:6823/3776085849","heartbeat_front_addr":"172.21.15.40:6821/3776085849","state":["exists","up"]},{"osd":8,"uuid":"f60407df-91f5-477a-9702-ddaef24faf82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6817","nonce":2736546122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6819","nonce":2736546122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6823","nonce":2736546122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6821","nonce":2736546122}]},"public_addr":"172.21.15.169:6817/2736546122","cluster_addr":"172.21.15.169:6819/2736546122","heartbeat_back_addr":"172.21.15.169:6823/2736546122","heartbeat_front_addr":"172.21.15.169:6821/2736546122","state":["exists","up"]},{"osd":9,"uuid":"7506a407-1960-49bc-9e38-7f2a8bf996ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6827","nonce":1187207880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6829","nonce":1187207880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6833","nonce":1187207880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6831","nonce":1187207880}]},"public_addr":"172.21.15.31:6827/1187207880","cluster_addr":"172.21.15.31:6829/1187207880","heartbeat_back_addr":"172.21.15.31:6833/1187207880","heartbeat_front_addr":"172.21.15.31:6831/1187207880","state":["exists","up"]},{"osd":10,"uuid":"831c48e2-b0d0-451f-959b-228e927542cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6825","nonce":3050148001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6827","nonce":3050148001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6831","nonce":3050148001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6829","nonce":3050148001}]},"public_addr":"172.21.15.40:6825/3050148001","cluster_addr":"172.21.15.40:6827/3050148001","heartbeat_back_addr":"172.21.15.40:6831/3050148001","heartbeat_front_addr":"172.21.15.40:6829/3050148001","state":["exists","up"]},{"osd":11,"uuid":"144fe0aa-f757-4b3c-90ad-99775e17d57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6825","nonce":262401944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6827","nonce":262401944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6831","nonce":262401944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6829","nonce":262401944}]},"public_addr":"172.21.15.169:6825/262401944","cluster_addr":"172.21.15.169:6827/262401944","heartbeat_back_addr":"172.21.15.169:6831/262401944","heartbeat_front_addr":"172.21.15.169:6829/262401944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:25.640995+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:51.036909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.635633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:48.467912+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:14.454971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:37.609315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:12.441807+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:38.239215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:02.230758+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:35.762553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:02.051405+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:26.569593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6801/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4004014066":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/1930101646":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/2638309002":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2238638505":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2902841144":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4190696323":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/480558521":"2024-04-26T01:04:57.149272+0000","172.21.15.31:6801/2638309002":"2024-04-26T01:04:57.149272+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:13:32.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.428+0000 7f1d9f200640 1 -- 172.21.15.31:0/136723451 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1d840a2b90 msgr2=0x7f1d840a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:32.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.428+0000 7f1d9f200640 1 --2- 172.21.15.31:0/136723451 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1d840a2b90 0x7f1d840a5050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1d90011490 tx=0x7f1d90009260 comp rx=0 tx=0).stop 2024-04-25T01:13:32.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.428+0000 7f1d9f200640 1 -- 172.21.15.31:0/136723451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1da0108db0 msgr2=0x7f1da019e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:32.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.428+0000 7f1d9f200640 1 --2- 172.21.15.31:0/136723451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1da0108db0 0x7f1da019e3a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1d88005fd0 tx=0x7f1d8800dde0 comp rx=0 tx=0).stop 2024-04-25T01:13:32.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.429+0000 7f1d9f200640 1 -- 172.21.15.31:0/136723451 shutdown_connections 2024-04-25T01:13:32.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.429+0000 7f1d9f200640 1 --2- 172.21.15.31:0/136723451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1da0109770 0x7f1da01a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:32.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.429+0000 7f1d9f200640 1 --2- 172.21.15.31:0/136723451 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1d840a2b90 0x7f1d840a5050 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:32.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.429+0000 7f1d9f200640 1 --2- 172.21.15.31:0/136723451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1da0108db0 0x7f1da019e3a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:32.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.429+0000 7f1d9f200640 1 --2- 172.21.15.31:0/136723451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da0107bc0 0x7f1da019de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:32.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.429+0000 7f1d9f200640 1 -- 172.21.15.31:0/136723451 >> 172.21.15.31:0/136723451 conn(0x7f1da0076b80 msgr2=0x7f1da010c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:32.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.429+0000 7f1d9f200640 1 -- 172.21.15.31:0/136723451 shutdown_connections 2024-04-25T01:13:32.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.429+0000 7f1d9f200640 1 -- 172.21.15.31:0/136723451 wait complete. 2024-04-25T01:13:32.436 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T01:13:32.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.665+0000 7ff7ba600640 1 -- 172.21.15.31:0/3466145642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b4105430 msgr2=0x7ff7b410bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:32.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.665+0000 7ff7ba600640 1 --2- 172.21.15.31:0/3466145642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b4105430 0x7ff7b410bcc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009ca0 tx=0x7ff7ac02a8c0 comp rx=0 tx=0).stop 2024-04-25T01:13:32.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.666+0000 7ff7ba600640 1 -- 172.21.15.31:0/3466145642 shutdown_connections 2024-04-25T01:13:32.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.666+0000 7ff7ba600640 1 --2- 172.21.15.31:0/3466145642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b4105430 0x7ff7b410bcc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:32.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.666+0000 7ff7ba600640 1 --2- 172.21.15.31:0/3466145642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7b4104a70 0x7ff7b4104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:32.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.666+0000 7ff7ba600640 1 --2- 172.21.15.31:0/3466145642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7b4103880 0x7ff7b4103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:32.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.666+0000 7ff7ba600640 1 -- 172.21.15.31:0/3466145642 >> 172.21.15.31:0/3466145642 conn(0x7ff7b40fedf0 msgr2=0x7ff7b4101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:32.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.666+0000 7ff7ba600640 1 -- 172.21.15.31:0/3466145642 shutdown_connections 2024-04-25T01:13:32.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.666+0000 7ff7ba600640 1 -- 172.21.15.31:0/3466145642 wait complete. 2024-04-25T01:13:32.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.667+0000 7ff7ba600640 1 Processor -- start 2024-04-25T01:13:32.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.667+0000 7ff7ba600640 1 -- start start 2024-04-25T01:13:32.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.667+0000 7ff7ba600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7b4103880 0x7ff7b4199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:32.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.667+0000 7ff7ba600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b4104a70 0x7ff7b419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:32.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.667+0000 7ff7ba600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7b4105430 0x7ff7b41a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:32.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.667+0000 7ff7ba600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b410f4e0 con 0x7ff7b4104a70 2024-04-25T01:13:32.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.667+0000 7ff7ba600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b410f360 con 0x7ff7b4103880 2024-04-25T01:13:32.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.667+0000 7ff7ba600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b410f660 con 0x7ff7b4105430 2024-04-25T01:13:32.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.668+0000 7ff7b9200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7b4103880 0x7ff7b4199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:32.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.668+0000 7ff7b9200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7b4103880 0x7ff7b4199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:44846/0 (socket says 172.21.15.31:44846) 2024-04-25T01:13:32.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.668+0000 7ff7b3e00640 1 --2- 172.21.15.31:0/1543374523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b4104a70 0x7ff7b419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:32.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.668+0000 7ff7b9200640 1 -- 172.21.15.31:0/1543374523 learned_addr learned my addr 172.21.15.31:0/1543374523 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:32.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.668+0000 7ff7b9c00640 1 --2- 172.21.15.31:0/1543374523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7b4105430 0x7ff7b41a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:32.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.668+0000 7ff7b9200640 1 -- 172.21.15.31:0/1543374523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7b4105430 msgr2=0x7ff7b41a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:32.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.668+0000 7ff7b9200640 1 --2- 172.21.15.31:0/1543374523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7b4105430 0x7ff7b41a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:32.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.668+0000 7ff7b9200640 1 -- 172.21.15.31:0/1543374523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b4104a70 msgr2=0x7ff7b419a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:32.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.668+0000 7ff7b9200640 1 --2- 172.21.15.31:0/1543374523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b4104a70 0x7ff7b419a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:32.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.668+0000 7ff7b9200640 1 -- 172.21.15.31:0/1543374523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b41a1710 con 0x7ff7b4103880 2024-04-25T01:13:32.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.669+0000 7ff7b9200640 1 --2- 172.21.15.31:0/1543374523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7b4103880 0x7ff7b4199af0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff7a800e9c0 tx=0x7ff7a800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.669+0000 7ff7b1600640 1 -- 172.21.15.31:0/1543374523 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a800cd80 con 0x7ff7b4103880 2024-04-25T01:13:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.669+0000 7ff7b1600640 1 -- 172.21.15.31:0/1543374523 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7a80035f0 con 0x7ff7b4103880 2024-04-25T01:13:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.669+0000 7ff7ba600640 1 -- 172.21.15.31:0/1543374523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b41a1a00 con 0x7ff7b4103880 2024-04-25T01:13:32.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.669+0000 7ff7b1600640 1 -- 172.21.15.31:0/1543374523 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8010690 con 0x7ff7b4103880 2024-04-25T01:13:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.669+0000 7ff7ba600640 1 -- 172.21.15.31:0/1543374523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b41a1f50 con 0x7ff7b4103880 2024-04-25T01:13:32.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.670+0000 7ff7ba600640 1 -- 172.21.15.31:0/1543374523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b410ab40 con 0x7ff7b4103880 2024-04-25T01:13:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.672+0000 7ff7b1600640 1 -- 172.21.15.31:0/1543374523 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff7a80037a0 con 0x7ff7b4103880 2024-04-25T01:13:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.673+0000 7ff7b1600640 1 --2- 172.21.15.31:0/1543374523 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff79c0a2b90 0x7ff79c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.674+0000 7ff7b1600640 1 -- 172.21.15.31:0/1543374523 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7ff7a80cf350 con 0x7ff7b4103880 2024-04-25T01:13:32.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.674+0000 7ff7b3e00640 1 --2- 172.21.15.31:0/1543374523 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff79c0a2b90 0x7ff79c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:32.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.674+0000 7ff7b3e00640 1 --2- 172.21.15.31:0/1543374523 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff79c0a2b90 0x7ff79c0a5050 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00095f0 tx=0x7ff7a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:32.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.678+0000 7ff7b1600640 1 -- 172.21.15.31:0/1543374523 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff7a809c360 con 0x7ff7b4103880 2024-04-25T01:13:32.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:32 smithi031 ceph-mon[28053]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T01:13:32.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:32 smithi031 ceph-mon[28053]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T01:13:32.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:32 smithi031 ceph-mon[28053]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T01:13:32.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:32 smithi031 ceph-mon[28053]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T01:13:32.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:32 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2514993231' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:13:32.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:32 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/136723451' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:13:32.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:32.928+0000 7ff7ba600640 1 -- 172.21.15.31:0/1543374523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff7b402d170 con 0x7ff7b4103880 2024-04-25T01:13:32.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:32 smithi169 ceph-mon[32759]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T01:13:32.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:32 smithi169 ceph-mon[32759]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T01:13:32.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:32 smithi169 ceph-mon[32759]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T01:13:32.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:32 smithi169 ceph-mon[32759]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T01:13:32.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:32 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2514993231' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:13:32.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:32 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/136723451' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:13:32.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:32 smithi040 ceph-mon[32518]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T01:13:32.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:32 smithi040 ceph-mon[32518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T01:13:32.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:32 smithi040 ceph-mon[32518]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T01:13:32.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:32 smithi040 ceph-mon[32518]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T01:13:32.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:32 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2514993231' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:13:32.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:32 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/136723451' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:13:33.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.570+0000 7ff7b1600640 1 -- 172.21.15.31:0/1543374523 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7ff7b402d170 con 0x7ff7b4103880 2024-04-25T01:13:33.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.574+0000 7ff7ba600640 1 -- 172.21.15.31:0/1543374523 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff79c0a2b90 msgr2=0x7ff79c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:33.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.574+0000 7ff7ba600640 1 --2- 172.21.15.31:0/1543374523 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff79c0a2b90 0x7ff79c0a5050 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00095f0 tx=0x7ff7a0005f90 comp rx=0 tx=0).stop 2024-04-25T01:13:33.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.574+0000 7ff7ba600640 1 -- 172.21.15.31:0/1543374523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7b4103880 msgr2=0x7ff7b4199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:33.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.574+0000 7ff7ba600640 1 --2- 172.21.15.31:0/1543374523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7b4103880 0x7ff7b4199af0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff7a800e9c0 tx=0x7ff7a800ee80 comp rx=0 tx=0).stop 2024-04-25T01:13:33.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.574+0000 7ff7ba600640 1 -- 172.21.15.31:0/1543374523 shutdown_connections 2024-04-25T01:13:33.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.575+0000 7ff7ba600640 1 --2- 172.21.15.31:0/1543374523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7b4105430 0x7ff7b41a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:33.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.575+0000 7ff7ba600640 1 --2- 172.21.15.31:0/1543374523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b4104a70 0x7ff7b419a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:33.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.575+0000 7ff7ba600640 1 --2- 172.21.15.31:0/1543374523 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff79c0a2b90 0x7ff79c0a5050 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:33.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.575+0000 7ff7ba600640 1 --2- 172.21.15.31:0/1543374523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7b4103880 0x7ff7b4199af0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:33.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.575+0000 7ff7ba600640 1 -- 172.21.15.31:0/1543374523 >> 172.21.15.31:0/1543374523 conn(0x7ff7b40fedf0 msgr2=0x7ff7b4107d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:33.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.575+0000 7ff7ba600640 1 -- 172.21.15.31:0/1543374523 shutdown_connections 2024-04-25T01:13:33.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.575+0000 7ff7ba600640 1 -- 172.21.15.31:0/1543374523 wait complete. 2024-04-25T01:13:33.584 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-25T01:13:33.585 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-25T01:13:33.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:33 smithi031 ceph-mon[28053]: pgmap v316: 97 pgs: 27 active+clean, 15 creating+peering, 55 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-25T01:13:33.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:33 smithi031 ceph-mon[28053]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T01:13:33.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:33 smithi031 ceph-mon[28053]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T01:13:33.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:33 smithi031 ceph-mon[28053]: Cluster is now healthy 2024-04-25T01:13:33.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:33 smithi031 ceph-mon[28053]: mds.? [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] up:active 2024-04-25T01:13:33.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:33 smithi031 ceph-mon[28053]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:13:33.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:33 smithi031 ceph-mon[28053]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:13:33.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:33 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1543374523' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T01:13:33.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:33 smithi031 ceph-mon[28053]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T01:13:33.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.821+0000 7f3184c00640 1 -- 172.21.15.31:0/452342600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3180108e70 msgr2=0x7f31801092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:33.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.821+0000 7f3184c00640 1 --2- 172.21.15.31:0/452342600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3180108e70 0x7f31801092f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3168009900 tx=0x7f316802a3f0 comp rx=0 tx=0).stop 2024-04-25T01:13:33.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.822+0000 7f3184c00640 1 -- 172.21.15.31:0/452342600 shutdown_connections 2024-04-25T01:13:33.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.822+0000 7f3184c00640 1 --2- 172.21.15.31:0/452342600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3180109830 0x7f31801100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:33.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.822+0000 7f3184c00640 1 --2- 172.21.15.31:0/452342600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3180108e70 0x7f31801092f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:33.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.822+0000 7f3184c00640 1 --2- 172.21.15.31:0/452342600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3180107c80 0x7f31801080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:33.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.822+0000 7f3184c00640 1 -- 172.21.15.31:0/452342600 >> 172.21.15.31:0/452342600 conn(0x7f3180076c10 msgr2=0x7f3180079050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:33.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.822+0000 7f3184c00640 1 -- 172.21.15.31:0/452342600 shutdown_connections 2024-04-25T01:13:33.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.822+0000 7f3184c00640 1 -- 172.21.15.31:0/452342600 wait complete. 2024-04-25T01:13:33.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.823+0000 7f3184c00640 1 Processor -- start 2024-04-25T01:13:33.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.823+0000 7f3184c00640 1 -- start start 2024-04-25T01:13:33.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.823+0000 7f3184c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3180107c80 0x7f318019e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:33.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.823+0000 7f3184c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3180108e70 0x7f318019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:33.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.823+0000 7f3184c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3180109830 0x7f31801a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:33.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.823+0000 7f3184c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3180113950 con 0x7f3180107c80 2024-04-25T01:13:33.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.823+0000 7f3184c00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f31801137d0 con 0x7f3180108e70 2024-04-25T01:13:33.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.823+0000 7f3184c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3180113ad0 con 0x7f3180109830 2024-04-25T01:13:33.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.824+0000 7f317f400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3180107c80 0x7f318019e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:33.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.824+0000 7f317ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3180108e70 0x7f318019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:33.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.824+0000 7f317ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3180108e70 0x7f318019e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:44872/0 (socket says 172.21.15.31:44872) 2024-04-25T01:13:33.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.824+0000 7f317f400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3180107c80 0x7f318019e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43538/0 (socket says 172.21.15.31:43538) 2024-04-25T01:13:33.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.824+0000 7f317ea00640 1 -- 172.21.15.31:0/3230520627 learned_addr learned my addr 172.21.15.31:0/3230520627 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:33.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.824+0000 7f317fe00640 1 --2- 172.21.15.31:0/3230520627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3180109830 0x7f31801a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:33.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.824+0000 7f317ea00640 1 -- 172.21.15.31:0/3230520627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3180109830 msgr2=0x7f31801a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:33.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.824+0000 7f317ea00640 1 --2- 172.21.15.31:0/3230520627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3180109830 0x7f31801a54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:33.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.824+0000 7f317ea00640 1 -- 172.21.15.31:0/3230520627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3180107c80 msgr2=0x7f318019e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:33.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.824+0000 7f317ea00640 1 --2- 172.21.15.31:0/3230520627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3180107c80 0x7f318019e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:33.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.824+0000 7f317ea00640 1 -- 172.21.15.31:0/3230520627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31801a5b40 con 0x7f3180108e70 2024-04-25T01:13:33.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.824+0000 7f317f400640 1 --2- 172.21.15.31:0/3230520627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3180107c80 0x7f318019e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:13:33.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.825+0000 7f317ea00640 1 --2- 172.21.15.31:0/3230520627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3180108e70 0x7f318019e5a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3168031440 tx=0x7f3168032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:33.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.825+0000 7f3177e00640 1 -- 172.21.15.31:0/3230520627 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316803f030 con 0x7f3180108e70 2024-04-25T01:13:33.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.825+0000 7f3177e00640 1 -- 172.21.15.31:0/3230520627 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3168003ce0 con 0x7f3180108e70 2024-04-25T01:13:33.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.825+0000 7f3177e00640 1 -- 172.21.15.31:0/3230520627 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3168035070 con 0x7f3180108e70 2024-04-25T01:13:33.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.825+0000 7f3184c00640 1 -- 172.21.15.31:0/3230520627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31801a5dd0 con 0x7f3180108e70 2024-04-25T01:13:33.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.825+0000 7f3184c00640 1 -- 172.21.15.31:0/3230520627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31801a6210 con 0x7f3180108e70 2024-04-25T01:13:33.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.826+0000 7f3184c00640 1 -- 172.21.15.31:0/3230520627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f318010ef40 con 0x7f3180108e70 2024-04-25T01:13:33.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.828+0000 7f3177e00640 1 -- 172.21.15.31:0/3230520627 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3168042030 con 0x7f3180108e70 2024-04-25T01:13:33.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.830+0000 7f3177e00640 1 --2- 172.21.15.31:0/3230520627 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f31640a2c10 0x7f31640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:33.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.830+0000 7f3177e00640 1 -- 172.21.15.31:0/3230520627 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f31680ef3a0 con 0x7f3180108e70 2024-04-25T01:13:33.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.830+0000 7f317f400640 1 --2- 172.21.15.31:0/3230520627 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f31640a2c10 0x7f31640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:33.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.831+0000 7f317f400640 1 --2- 172.21.15.31:0/3230520627 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f31640a2c10 0x7f31640a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3180108cd0 tx=0x7f3170005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:33.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:33.834+0000 7f3177e00640 1 -- 172.21.15.31:0/3230520627 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f31680bbd60 con 0x7f3180108e70 2024-04-25T01:13:33.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:33 smithi169 ceph-mon[32759]: pgmap v316: 97 pgs: 27 active+clean, 15 creating+peering, 55 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-25T01:13:33.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:33 smithi169 ceph-mon[32759]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T01:13:33.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:33 smithi169 ceph-mon[32759]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T01:13:33.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:33 smithi169 ceph-mon[32759]: Cluster is now healthy 2024-04-25T01:13:33.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:33 smithi169 ceph-mon[32759]: mds.? [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] up:active 2024-04-25T01:13:33.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:33 smithi169 ceph-mon[32759]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:13:33.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:33 smithi169 ceph-mon[32759]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:13:33.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:33 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1543374523' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T01:13:33.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:33 smithi169 ceph-mon[32759]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T01:13:33.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:33 smithi040 ceph-mon[32518]: pgmap v316: 97 pgs: 27 active+clean, 15 creating+peering, 55 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-25T01:13:33.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:33 smithi040 ceph-mon[32518]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T01:13:33.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:33 smithi040 ceph-mon[32518]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T01:13:33.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:33 smithi040 ceph-mon[32518]: Cluster is now healthy 2024-04-25T01:13:33.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:33 smithi040 ceph-mon[32518]: mds.? [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] up:active 2024-04-25T01:13:33.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:33 smithi040 ceph-mon[32518]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:13:33.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:33 smithi040 ceph-mon[32518]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:13:33.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:33 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1543374523' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T01:13:33.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:33 smithi040 ceph-mon[32518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T01:13:34.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.081+0000 7f3184c00640 1 -- 172.21.15.31:0/3230520627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f31800623e0 con 0x7f3180108e70 2024-04-25T01:13:34.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.574+0000 7f3177e00640 1 -- 172.21.15.31:0/3230520627 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f31680c0060 con 0x7f3180108e70 2024-04-25T01:13:34.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.578+0000 7f3184c00640 1 -- 172.21.15.31:0/3230520627 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f31640a2c10 msgr2=0x7f31640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:34.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.578+0000 7f3184c00640 1 --2- 172.21.15.31:0/3230520627 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f31640a2c10 0x7f31640a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3180108cd0 tx=0x7f3170005f90 comp rx=0 tx=0).stop 2024-04-25T01:13:34.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.578+0000 7f3184c00640 1 -- 172.21.15.31:0/3230520627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3180108e70 msgr2=0x7f318019e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:34.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.578+0000 7f3184c00640 1 --2- 172.21.15.31:0/3230520627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3180108e70 0x7f318019e5a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3168031440 tx=0x7f3168032ce0 comp rx=0 tx=0).stop 2024-04-25T01:13:34.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.578+0000 7f3184c00640 1 -- 172.21.15.31:0/3230520627 shutdown_connections 2024-04-25T01:13:34.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.578+0000 7f3184c00640 1 --2- 172.21.15.31:0/3230520627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3180109830 0x7f31801a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:34.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.578+0000 7f3184c00640 1 --2- 172.21.15.31:0/3230520627 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f31640a2c10 0x7f31640a50d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:34.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.578+0000 7f3184c00640 1 --2- 172.21.15.31:0/3230520627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3180108e70 0x7f318019e5a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:34.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.578+0000 7f3184c00640 1 --2- 172.21.15.31:0/3230520627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3180107c80 0x7f318019e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:34.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.578+0000 7f3184c00640 1 -- 172.21.15.31:0/3230520627 >> 172.21.15.31:0/3230520627 conn(0x7f3180076c10 msgr2=0x7f318010c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:34.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.578+0000 7f3184c00640 1 -- 172.21.15.31:0/3230520627 shutdown_connections 2024-04-25T01:13:34.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.578+0000 7f3184c00640 1 -- 172.21.15.31:0/3230520627 wait complete. 2024-04-25T01:13:34.594 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T01:13:34.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.809+0000 7f348ac00640 1 -- 172.21.15.31:0/1903743093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3484076100 msgr2=0x7f3484076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:34.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.809+0000 7f348ac00640 1 --2- 172.21.15.31:0/1903743093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3484076100 0x7f3484076520 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3474009900 tx=0x7f347402a3f0 comp rx=0 tx=0).stop 2024-04-25T01:13:34.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.809+0000 7f348ac00640 1 -- 172.21.15.31:0/1903743093 shutdown_connections 2024-04-25T01:13:34.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.809+0000 7f348ac00640 1 --2- 172.21.15.31:0/1903743093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3484077ce0 0x7f3484078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:34.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.809+0000 7f348ac00640 1 --2- 172.21.15.31:0/1903743093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3484077340 0x7f34840777a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:34.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.809+0000 7f348ac00640 1 --2- 172.21.15.31:0/1903743093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3484076100 0x7f3484076520 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:34.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.809+0000 7f348ac00640 1 -- 172.21.15.31:0/1903743093 >> 172.21.15.31:0/1903743093 conn(0x7f34840fef30 msgr2=0x7f3484101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:34.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.810+0000 7f348ac00640 1 -- 172.21.15.31:0/1903743093 shutdown_connections 2024-04-25T01:13:34.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.810+0000 7f348ac00640 1 -- 172.21.15.31:0/1903743093 wait complete. 2024-04-25T01:13:34.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.811+0000 7f348ac00640 1 Processor -- start 2024-04-25T01:13:34.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.811+0000 7f348ac00640 1 -- start start 2024-04-25T01:13:34.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.811+0000 7f348ac00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3484076100 0x7f348419e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.811+0000 7f348ac00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3484077340 0x7f348419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.811+0000 7f348ac00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3484077ce0 0x7f34841a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.811+0000 7f348ac00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3484113a70 con 0x7f3484077340 2024-04-25T01:13:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.811+0000 7f348ac00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f34841138f0 con 0x7f3484076100 2024-04-25T01:13:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.811+0000 7f348ac00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3484113bf0 con 0x7f3484077ce0 2024-04-25T01:13:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.811+0000 7f3483400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3484077340 0x7f348419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.811+0000 7f3483400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3484077340 0x7f348419e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43568/0 (socket says 172.21.15.31:43568) 2024-04-25T01:13:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.811+0000 7f3483400640 1 -- 172.21.15.31:0/1827885600 learned_addr learned my addr 172.21.15.31:0/1827885600 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.811+0000 7f3483e00640 1 --2- 172.21.15.31:0/1827885600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3484076100 0x7f348419e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.811+0000 7f3488c00640 1 --2- 172.21.15.31:0/1827885600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3484077ce0 0x7f34841a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.812+0000 7f3483400640 1 -- 172.21.15.31:0/1827885600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3484077ce0 msgr2=0x7f34841a5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.812+0000 7f3483400640 1 --2- 172.21.15.31:0/1827885600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3484077ce0 0x7f34841a5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.812+0000 7f3483400640 1 -- 172.21.15.31:0/1827885600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3484076100 msgr2=0x7f348419e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:34.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.812+0000 7f3483400640 1 --2- 172.21.15.31:0/1827885600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3484076100 0x7f348419e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:34.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.812+0000 7f3483400640 1 -- 172.21.15.31:0/1827885600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34841a5d00 con 0x7f3484077340 2024-04-25T01:13:34.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.812+0000 7f3483e00640 1 --2- 172.21.15.31:0/1827885600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3484076100 0x7f348419e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:13:34.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.812+0000 7f3483400640 1 --2- 172.21.15.31:0/1827885600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3484077340 0x7f348419e670 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f346c00eb90 tx=0x7f346c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:34.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.812+0000 7f3480c00640 1 -- 172.21.15.31:0/1827885600 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c004d20 con 0x7f3484077340 2024-04-25T01:13:34.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.813+0000 7f3480c00640 1 -- 172.21.15.31:0/1827885600 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f346c00ee90 con 0x7f3484077340 2024-04-25T01:13:34.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.813+0000 7f348ac00640 1 -- 172.21.15.31:0/1827885600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34841a5ff0 con 0x7f3484077340 2024-04-25T01:13:34.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.813+0000 7f3480c00640 1 -- 172.21.15.31:0/1827885600 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c010830 con 0x7f3484077340 2024-04-25T01:13:34.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.813+0000 7f348ac00640 1 -- 172.21.15.31:0/1827885600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34841a6460 con 0x7f3484077340 2024-04-25T01:13:34.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.814+0000 7f348ac00640 1 -- 172.21.15.31:0/1827885600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3484110310 con 0x7f3484077340 2024-04-25T01:13:34.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.816+0000 7f3480c00640 1 -- 172.21.15.31:0/1827885600 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f346c005020 con 0x7f3484077340 2024-04-25T01:13:34.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.817+0000 7f3480c00640 1 --2- 172.21.15.31:0/1827885600 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f34680a2b90 0x7f34680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:34.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.818+0000 7f3480c00640 1 -- 172.21.15.31:0/1827885600 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f346c0cf5d0 con 0x7f3484077340 2024-04-25T01:13:34.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.818+0000 7f3483e00640 1 --2- 172.21.15.31:0/1827885600 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f34680a2b90 0x7f34680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:34.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.819+0000 7f3483e00640 1 --2- 172.21.15.31:0/1827885600 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f34680a2b90 0x7f34680a5050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3474031440 tx=0x7f34740315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:34.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:34.822+0000 7f3480c00640 1 -- 172.21.15.31:0/1827885600 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f346c09c5e0 con 0x7f3484077340 2024-04-25T01:13:34.886 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:34 smithi040 ceph-mon[32518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T01:13:34.886 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:34 smithi040 ceph-mon[32518]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:13:34.887 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:34 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3230520627' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T01:13:34.887 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:34 smithi040 ceph-mon[32518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T01:13:34.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:34 smithi169 ceph-mon[32759]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T01:13:34.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:34 smithi169 ceph-mon[32759]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:13:34.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:34 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3230520627' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T01:13:34.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:34 smithi169 ceph-mon[32759]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T01:13:35.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:34 smithi031 ceph-mon[28053]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T01:13:35.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:34 smithi031 ceph-mon[28053]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:13:35.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:34 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3230520627' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T01:13:35.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:34 smithi031 ceph-mon[28053]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T01:13:35.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.068+0000 7f348ac00640 1 -- 172.21.15.31:0/1827885600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f34840623e0 con 0x7f3484077340 2024-04-25T01:13:35.198 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 01:13:34 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-b[63226]: 2024-04-25T01:13:34.898+0000 7f2b40ea0700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T01:13:35.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.525+0000 7f3480c00640 1 -- 172.21.15.31:0/1827885600 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f346c09bee0 con 0x7f3484077340 2024-04-25T01:13:35.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.529+0000 7f348ac00640 1 -- 172.21.15.31:0/1827885600 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f34680a2b90 msgr2=0x7f34680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:35.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.529+0000 7f348ac00640 1 --2- 172.21.15.31:0/1827885600 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f34680a2b90 0x7f34680a5050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3474031440 tx=0x7f34740315b0 comp rx=0 tx=0).stop 2024-04-25T01:13:35.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.529+0000 7f348ac00640 1 -- 172.21.15.31:0/1827885600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3484077340 msgr2=0x7f348419e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:35.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.529+0000 7f348ac00640 1 --2- 172.21.15.31:0/1827885600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3484077340 0x7f348419e670 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f346c00eb90 tx=0x7f346c00c6a0 comp rx=0 tx=0).stop 2024-04-25T01:13:35.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.529+0000 7f348ac00640 1 -- 172.21.15.31:0/1827885600 shutdown_connections 2024-04-25T01:13:35.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.529+0000 7f348ac00640 1 --2- 172.21.15.31:0/1827885600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3484077ce0 0x7f34841a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:35.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.529+0000 7f348ac00640 1 --2- 172.21.15.31:0/1827885600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3484077340 0x7f348419e670 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:35.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.529+0000 7f348ac00640 1 --2- 172.21.15.31:0/1827885600 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f34680a2b90 0x7f34680a5050 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:35.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.529+0000 7f348ac00640 1 --2- 172.21.15.31:0/1827885600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3484076100 0x7f348419e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:35.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.529+0000 7f348ac00640 1 -- 172.21.15.31:0/1827885600 >> 172.21.15.31:0/1827885600 conn(0x7f34840fef30 msgr2=0x7f3484100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:35.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.529+0000 7f348ac00640 1 -- 172.21.15.31:0/1827885600 shutdown_connections 2024-04-25T01:13:35.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.529+0000 7f348ac00640 1 -- 172.21.15.31:0/1827885600 wait complete. 2024-04-25T01:13:35.551 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T01:13:35.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.761+0000 7fbb1f200640 1 -- 172.21.15.31:0/717635461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb201028c0 msgr2=0x7fbb20102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:35.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.761+0000 7fbb1f200640 1 --2- 172.21.15.31:0/717635461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb201028c0 0x7fbb20102d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbb08009900 tx=0x7fbb0802a3f0 comp rx=0 tx=0).stop 2024-04-25T01:13:35.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.761+0000 7fbb1f200640 1 -- 172.21.15.31:0/717635461 shutdown_connections 2024-04-25T01:13:35.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.761+0000 7fbb1f200640 1 --2- 172.21.15.31:0/717635461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb20103280 0x7fbb20109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:35.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.761+0000 7fbb1f200640 1 --2- 172.21.15.31:0/717635461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb201028c0 0x7fbb20102d40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:35.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.761+0000 7fbb1f200640 1 --2- 172.21.15.31:0/717635461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb201016d0 0x7fbb20101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:35.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.761+0000 7fbb1f200640 1 -- 172.21.15.31:0/717635461 >> 172.21.15.31:0/717635461 conn(0x7fbb200fcc40 msgr2=0x7fbb200ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:35.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.761+0000 7fbb1f200640 1 -- 172.21.15.31:0/717635461 shutdown_connections 2024-04-25T01:13:35.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.761+0000 7fbb1f200640 1 -- 172.21.15.31:0/717635461 wait complete. 2024-04-25T01:13:35.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.762+0000 7fbb1f200640 1 Processor -- start 2024-04-25T01:13:35.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.763+0000 7fbb1f200640 1 -- start start 2024-04-25T01:13:35.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.763+0000 7fbb1f200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb201016d0 0x7fbb20199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:35.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.763+0000 7fbb1f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb201028c0 0x7fbb2019a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:35.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.763+0000 7fbb1f200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb20103280 0x7fbb201a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:35.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.763+0000 7fbb1f200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2010d330 con 0x7fbb201016d0 2024-04-25T01:13:35.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.763+0000 7fbb1f200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2010d1b0 con 0x7fbb20103280 2024-04-25T01:13:35.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.763+0000 7fbb1f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2010d4b0 con 0x7fbb201028c0 2024-04-25T01:13:35.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.763+0000 7fbb1d400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb201028c0 0x7fbb2019a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:35.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.763+0000 7fbb1e800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb20103280 0x7fbb201a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:35.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.763+0000 7fbb1de00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb201016d0 0x7fbb20199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:35.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.763+0000 7fbb1d400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb201028c0 0x7fbb2019a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:60716/0 (socket says 172.21.15.31:60716) 2024-04-25T01:13:35.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.763+0000 7fbb1d400640 1 -- 172.21.15.31:0/2038948956 learned_addr learned my addr 172.21.15.31:0/2038948956 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:35.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.764+0000 7fbb1e800640 1 -- 172.21.15.31:0/2038948956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb201028c0 msgr2=0x7fbb2019a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:35.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.764+0000 7fbb1e800640 1 --2- 172.21.15.31:0/2038948956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb201028c0 0x7fbb2019a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:35.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.764+0000 7fbb1e800640 1 -- 172.21.15.31:0/2038948956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb201016d0 msgr2=0x7fbb20199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:35.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.764+0000 7fbb1e800640 1 --2- 172.21.15.31:0/2038948956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb201016d0 0x7fbb20199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:35.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.764+0000 7fbb1e800640 1 -- 172.21.15.31:0/2038948956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb201a17d0 con 0x7fbb20103280 2024-04-25T01:13:35.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.764+0000 7fbb1e800640 1 --2- 172.21.15.31:0/2038948956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb20103280 0x7fbb201a1160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800e970 tx=0x7fbb1800ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:35.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.764+0000 7fbb16a00640 1 -- 172.21.15.31:0/2038948956 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1800ccc0 con 0x7fbb20103280 2024-04-25T01:13:35.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.764+0000 7fbb16a00640 1 -- 172.21.15.31:0/2038948956 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb18005020 con 0x7fbb20103280 2024-04-25T01:13:35.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.764+0000 7fbb1f200640 1 -- 172.21.15.31:0/2038948956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb201a1ac0 con 0x7fbb20103280 2024-04-25T01:13:35.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.764+0000 7fbb16a00640 1 -- 172.21.15.31:0/2038948956 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18004a00 con 0x7fbb20103280 2024-04-25T01:13:35.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.765+0000 7fbb1f200640 1 -- 172.21.15.31:0/2038948956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb201a1fc0 con 0x7fbb20103280 2024-04-25T01:13:35.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.766+0000 7fbb1f200640 1 -- 172.21.15.31:0/2038948956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb201a21f0 con 0x7fbb20103280 2024-04-25T01:13:35.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.767+0000 7fbb16a00640 1 -- 172.21.15.31:0/2038948956 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbb180051d0 con 0x7fbb20103280 2024-04-25T01:13:35.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.769+0000 7fbb16a00640 1 --2- 172.21.15.31:0/2038948956 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbb040a2b90 0x7fbb040a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:35.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.769+0000 7fbb16a00640 1 -- 172.21.15.31:0/2038948956 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fbb180cef00 con 0x7fbb20103280 2024-04-25T01:13:35.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.769+0000 7fbb1de00640 1 --2- 172.21.15.31:0/2038948956 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbb040a2b90 0x7fbb040a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:35.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.770+0000 7fbb1de00640 1 --2- 172.21.15.31:0/2038948956 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbb040a2b90 0x7fbb040a5050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbb20102720 tx=0x7fbb10009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:35.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:35.773+0000 7fbb16a00640 1 -- 172.21.15.31:0/2038948956 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbb1809bf10 con 0x7fbb20103280 2024-04-25T01:13:35.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:35 smithi169 ceph-mon[32759]: pgmap v318: 97 pgs: 53 active+clean, 15 creating+peering, 29 unknown; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 3 op/s 2024-04-25T01:13:35.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:35 smithi169 ceph-mon[32759]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T01:13:35.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:35 smithi169 ceph-mon[32759]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T01:13:35.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:35 smithi169 ceph-mon[32759]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T01:13:35.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:35 smithi169 ceph-mon[32759]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:13:35.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:35 smithi169 ceph-mon[32759]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T01:13:35.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:35 smithi169 ceph-mon[32759]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T01:13:35.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:35 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1827885600' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T01:13:35.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:35 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1827885600' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T01:13:35.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:35 smithi169 ceph-mon[32759]: mds.? [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] up:active 2024-04-25T01:13:35.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:35 smithi169 ceph-mon[32759]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T01:13:35.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:35 smithi169 ceph-mon[32759]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T01:13:35.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:35 smithi169 ceph-mon[32759]: Cluster is now healthy 2024-04-25T01:13:35.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:35 smithi169 ceph-mon[32759]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2024-04-25T01:13:35.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:35 smithi169 ceph-mon[32759]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-04-25T01:13:35.946 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:35 smithi169 ceph-mon[32759]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-04-25T01:13:35.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:35 smithi040 ceph-mon[32518]: pgmap v318: 97 pgs: 53 active+clean, 15 creating+peering, 29 unknown; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 3 op/s 2024-04-25T01:13:35.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:35 smithi040 ceph-mon[32518]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T01:13:35.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:35 smithi040 ceph-mon[32518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T01:13:35.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:35 smithi040 ceph-mon[32518]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T01:13:35.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:35 smithi040 ceph-mon[32518]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:13:35.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:35 smithi040 ceph-mon[32518]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T01:13:35.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:35 smithi040 ceph-mon[32518]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T01:13:35.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:35 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1827885600' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T01:13:35.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:35 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1827885600' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T01:13:35.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:35 smithi040 ceph-mon[32518]: mds.? [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] up:active 2024-04-25T01:13:35.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:35 smithi040 ceph-mon[32518]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T01:13:35.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:35 smithi040 ceph-mon[32518]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T01:13:35.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:35 smithi040 ceph-mon[32518]: Cluster is now healthy 2024-04-25T01:13:35.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:35 smithi040 ceph-mon[32518]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2024-04-25T01:13:35.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:35 smithi040 ceph-mon[32518]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-04-25T01:13:35.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:35 smithi040 ceph-mon[32518]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-04-25T01:13:36.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:35 smithi031 ceph-mon[28053]: pgmap v318: 97 pgs: 53 active+clean, 15 creating+peering, 29 unknown; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 3 op/s 2024-04-25T01:13:36.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:35 smithi031 ceph-mon[28053]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T01:13:36.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:35 smithi031 ceph-mon[28053]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T01:13:36.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:35 smithi031 ceph-mon[28053]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T01:13:36.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:35 smithi031 ceph-mon[28053]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:13:36.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:35 smithi031 ceph-mon[28053]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T01:13:36.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:35 smithi031 ceph-mon[28053]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T01:13:36.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:35 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1827885600' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T01:13:36.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:35 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1827885600' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T01:13:36.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:35 smithi031 ceph-mon[28053]: mds.? [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] up:active 2024-04-25T01:13:36.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:35 smithi031 ceph-mon[28053]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T01:13:36.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:35 smithi031 ceph-mon[28053]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T01:13:36.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:35 smithi031 ceph-mon[28053]: Cluster is now healthy 2024-04-25T01:13:36.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:35 smithi031 ceph-mon[28053]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2024-04-25T01:13:36.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:35 smithi031 ceph-mon[28053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-04-25T01:13:36.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:35 smithi031 ceph-mon[28053]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-04-25T01:13:36.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.020+0000 7fbb1f200640 1 -- 172.21.15.31:0/2038948956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fbb201a25e0 con 0x7fbb20103280 2024-04-25T01:13:36.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.541+0000 7fbb16a00640 1 -- 172.21.15.31:0/2038948956 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7fbb1809b810 con 0x7fbb20103280 2024-04-25T01:13:36.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.546+0000 7fbb1f200640 1 -- 172.21.15.31:0/2038948956 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbb040a2b90 msgr2=0x7fbb040a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:36.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.546+0000 7fbb1f200640 1 --2- 172.21.15.31:0/2038948956 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbb040a2b90 0x7fbb040a5050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbb20102720 tx=0x7fbb10009500 comp rx=0 tx=0).stop 2024-04-25T01:13:36.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.546+0000 7fbb1f200640 1 -- 172.21.15.31:0/2038948956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb20103280 msgr2=0x7fbb201a1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:36.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.546+0000 7fbb1f200640 1 --2- 172.21.15.31:0/2038948956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb20103280 0x7fbb201a1160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800e970 tx=0x7fbb1800ee30 comp rx=0 tx=0).stop 2024-04-25T01:13:36.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.546+0000 7fbb1f200640 1 -- 172.21.15.31:0/2038948956 shutdown_connections 2024-04-25T01:13:36.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.546+0000 7fbb1f200640 1 --2- 172.21.15.31:0/2038948956 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbb040a2b90 0x7fbb040a5050 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:36.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.546+0000 7fbb1f200640 1 --2- 172.21.15.31:0/2038948956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb20103280 0x7fbb201a1160 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:36.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.546+0000 7fbb1f200640 1 --2- 172.21.15.31:0/2038948956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb201028c0 0x7fbb2019a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:36.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.546+0000 7fbb1f200640 1 --2- 172.21.15.31:0/2038948956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb201016d0 0x7fbb20199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:36.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.546+0000 7fbb1f200640 1 -- 172.21.15.31:0/2038948956 >> 172.21.15.31:0/2038948956 conn(0x7fbb200fcc40 msgr2=0x7fbb20073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:36.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.546+0000 7fbb1f200640 1 -- 172.21.15.31:0/2038948956 shutdown_connections 2024-04-25T01:13:36.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.546+0000 7fbb1f200640 1 -- 172.21.15.31:0/2038948956 wait complete. 2024-04-25T01:13:36.559 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-25T01:13:36.559 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-25T01:13:36.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.795+0000 7fbac4e00640 1 -- 172.21.15.31:0/3190307836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbac0104a70 msgr2=0x7fbac0104ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:36.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.795+0000 7fbac4e00640 1 --2- 172.21.15.31:0/3190307836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbac0104a70 0x7fbac0104ef0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8009900 tx=0x7fbaa802a410 comp rx=0 tx=0).stop 2024-04-25T01:13:36.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.795+0000 7fbac4e00640 1 -- 172.21.15.31:0/3190307836 shutdown_connections 2024-04-25T01:13:36.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.795+0000 7fbac4e00640 1 --2- 172.21.15.31:0/3190307836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbac0105430 0x7fbac010bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:36.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.795+0000 7fbac4e00640 1 --2- 172.21.15.31:0/3190307836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbac0104a70 0x7fbac0104ef0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:36.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.795+0000 7fbac4e00640 1 --2- 172.21.15.31:0/3190307836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbac0103880 0x7fbac0103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:36.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.795+0000 7fbac4e00640 1 -- 172.21.15.31:0/3190307836 >> 172.21.15.31:0/3190307836 conn(0x7fbac00fedf0 msgr2=0x7fbac0101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:36.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.795+0000 7fbac4e00640 1 -- 172.21.15.31:0/3190307836 shutdown_connections 2024-04-25T01:13:36.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.795+0000 7fbac4e00640 1 -- 172.21.15.31:0/3190307836 wait complete. 2024-04-25T01:13:36.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.796+0000 7fbac4e00640 1 Processor -- start 2024-04-25T01:13:36.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.796+0000 7fbac4e00640 1 -- start start 2024-04-25T01:13:36.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.797+0000 7fbac4e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbac0103880 0x7fbac0199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:36.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.797+0000 7fbac4e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbac0104a70 0x7fbac019a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:36.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.797+0000 7fbac4e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbac0105430 0x7fbac01a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:36.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.797+0000 7fbac4e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbac010f530 con 0x7fbac0105430 2024-04-25T01:13:36.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.797+0000 7fbac4e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbac010f3b0 con 0x7fbac0103880 2024-04-25T01:13:36.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.797+0000 7fbac4e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbac010f6b0 con 0x7fbac0104a70 2024-04-25T01:13:36.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.797+0000 7fbabf400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbac0103880 0x7fbac0199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:36.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.797+0000 7fbabfe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbac0105430 0x7fbac01a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:36.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.797+0000 7fbabf400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbac0103880 0x7fbac0199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:40762/0 (socket says 172.21.15.31:40762) 2024-04-25T01:13:36.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.797+0000 7fbabf400640 1 -- 172.21.15.31:0/320075743 learned_addr learned my addr 172.21.15.31:0/320075743 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:36.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.797+0000 7fbab7e00640 1 --2- 172.21.15.31:0/320075743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbac0104a70 0x7fbac019a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:36.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.797+0000 7fbabf400640 1 -- 172.21.15.31:0/320075743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbac0104a70 msgr2=0x7fbac019a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:36.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.797+0000 7fbabf400640 1 --2- 172.21.15.31:0/320075743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbac0104a70 0x7fbac019a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:36.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.797+0000 7fbabf400640 1 -- 172.21.15.31:0/320075743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbac0105430 msgr2=0x7fbac01a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:36.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.797+0000 7fbabf400640 1 --2- 172.21.15.31:0/320075743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbac0105430 0x7fbac01a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:36.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.797+0000 7fbabf400640 1 -- 172.21.15.31:0/320075743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac01a1770 con 0x7fbac0103880 2024-04-25T01:13:36.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.798+0000 7fbabf400640 1 --2- 172.21.15.31:0/320075743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbac0103880 0x7fbac0199b50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbab000e9c0 tx=0x7fbab000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:36.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.798+0000 7fbabcc00640 1 -- 172.21.15.31:0/320075743 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab000cd80 con 0x7fbac0103880 2024-04-25T01:13:36.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.798+0000 7fbabcc00640 1 -- 172.21.15.31:0/320075743 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbab00035f0 con 0x7fbac0103880 2024-04-25T01:13:36.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.798+0000 7fbabcc00640 1 -- 172.21.15.31:0/320075743 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0010690 con 0x7fbac0103880 2024-04-25T01:13:36.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.798+0000 7fbac4e00640 1 -- 172.21.15.31:0/320075743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac01a1a60 con 0x7fbac0103880 2024-04-25T01:13:36.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.798+0000 7fbac4e00640 1 -- 172.21.15.31:0/320075743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac01a1fd0 con 0x7fbac0103880 2024-04-25T01:13:36.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.799+0000 7fbac4e00640 1 -- 172.21.15.31:0/320075743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbac00688d0 con 0x7fbac0103880 2024-04-25T01:13:36.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.801+0000 7fbabcc00640 1 -- 172.21.15.31:0/320075743 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbab00037a0 con 0x7fbac0103880 2024-04-25T01:13:36.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.802+0000 7fbabcc00640 1 --2- 172.21.15.31:0/320075743 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbaa40a2c10 0x7fbaa40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:36.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.802+0000 7fbabcc00640 1 -- 172.21.15.31:0/320075743 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fbab00cf2f0 con 0x7fbac0103880 2024-04-25T01:13:36.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.802+0000 7fbab7e00640 1 --2- 172.21.15.31:0/320075743 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbaa40a2c10 0x7fbaa40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:36.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.803+0000 7fbab7e00640 1 --2- 172.21.15.31:0/320075743 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbaa40a2c10 0x7fbaa40a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8031440 tx=0x7fbaa80315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:36.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:36.806+0000 7fbabcc00640 1 -- 172.21.15.31:0/320075743 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbab009c300 con 0x7fbac0103880 2024-04-25T01:13:36.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:36 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2038948956' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T01:13:36.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:36 smithi169 ceph-mon[32759]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T01:13:36.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:36 smithi169 ceph-mon[32759]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T01:13:36.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:36 smithi169 ceph-mon[32759]: mds.? [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] up:active 2024-04-25T01:13:36.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:36 smithi169 ceph-mon[32759]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T01:13:36.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:36 smithi169 ceph-mon[32759]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:13:36.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:36 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2038948956' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T01:13:36.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:36 smithi040 ceph-mon[32518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T01:13:36.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:36 smithi040 ceph-mon[32518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T01:13:36.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:36 smithi040 ceph-mon[32518]: mds.? [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] up:active 2024-04-25T01:13:36.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:36 smithi040 ceph-mon[32518]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T01:13:36.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:36 smithi040 ceph-mon[32518]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:13:37.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:36 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2038948956' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T01:13:37.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:36 smithi031 ceph-mon[28053]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T01:13:37.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:36 smithi031 ceph-mon[28053]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T01:13:37.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:36 smithi031 ceph-mon[28053]: mds.? [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] up:active 2024-04-25T01:13:37.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:36 smithi031 ceph-mon[28053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T01:13:37.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:36 smithi031 ceph-mon[28053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:13:37.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.039+0000 7fbac4e00640 1 -- 172.21.15.31:0/320075743 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fbac0061b80 con 0x7fbaa40a2c10 2024-04-25T01:13:37.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.076+0000 7fbabcc00640 1 -- 172.21.15.31:0/320075743 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbac0061b80 con 0x7fbaa40a2c10 2024-04-25T01:13:37.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.078+0000 7fbac4e00640 1 -- 172.21.15.31:0/320075743 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbaa40a2c10 msgr2=0x7fbaa40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:37.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.078+0000 7fbac4e00640 1 --2- 172.21.15.31:0/320075743 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbaa40a2c10 0x7fbaa40a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8031440 tx=0x7fbaa80315b0 comp rx=0 tx=0).stop 2024-04-25T01:13:37.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.078+0000 7fbac4e00640 1 -- 172.21.15.31:0/320075743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbac0103880 msgr2=0x7fbac0199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:37.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.079+0000 7fbac4e00640 1 --2- 172.21.15.31:0/320075743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbac0103880 0x7fbac0199b50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbab000e9c0 tx=0x7fbab000ee80 comp rx=0 tx=0).stop 2024-04-25T01:13:37.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.079+0000 7fbac4e00640 1 -- 172.21.15.31:0/320075743 shutdown_connections 2024-04-25T01:13:37.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.079+0000 7fbac4e00640 1 --2- 172.21.15.31:0/320075743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbac0105430 0x7fbac01a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:37.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.079+0000 7fbac4e00640 1 --2- 172.21.15.31:0/320075743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbac0104a70 0x7fbac019a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:37.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.079+0000 7fbac4e00640 1 --2- 172.21.15.31:0/320075743 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbaa40a2c10 0x7fbaa40a50d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:37.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.079+0000 7fbac4e00640 1 --2- 172.21.15.31:0/320075743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbac0103880 0x7fbac0199b50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:37.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.079+0000 7fbac4e00640 1 -- 172.21.15.31:0/320075743 >> 172.21.15.31:0/320075743 conn(0x7fbac00fedf0 msgr2=0x7fbac0107d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:37.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.079+0000 7fbac4e00640 1 -- 172.21.15.31:0/320075743 shutdown_connections 2024-04-25T01:13:37.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.079+0000 7fbac4e00640 1 -- 172.21.15.31:0/320075743 wait complete. 2024-04-25T01:13:37.087 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T01:13:37.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.303+0000 7fe99f200640 1 -- 172.21.15.31:0/732706244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe998109830 msgr2=0x7fe9981100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:37.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.303+0000 7fe99f200640 1 --2- 172.21.15.31:0/732706244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe998109830 0x7fe9981100c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe994009be0 tx=0x7fe99402a6d0 comp rx=0 tx=0).stop 2024-04-25T01:13:37.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.303+0000 7fe99f200640 1 -- 172.21.15.31:0/732706244 shutdown_connections 2024-04-25T01:13:37.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.303+0000 7fe99f200640 1 --2- 172.21.15.31:0/732706244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe998109830 0x7fe9981100c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:37.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.303+0000 7fe99f200640 1 --2- 172.21.15.31:0/732706244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe998108e70 0x7fe9981092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:37.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.303+0000 7fe99f200640 1 --2- 172.21.15.31:0/732706244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe998107c80 0x7fe9981080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:37.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.303+0000 7fe99f200640 1 -- 172.21.15.31:0/732706244 >> 172.21.15.31:0/732706244 conn(0x7fe998076c10 msgr2=0x7fe998079050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:37.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.303+0000 7fe99f200640 1 -- 172.21.15.31:0/732706244 shutdown_connections 2024-04-25T01:13:37.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.303+0000 7fe99f200640 1 -- 172.21.15.31:0/732706244 wait complete. 2024-04-25T01:13:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.304+0000 7fe99f200640 1 Processor -- start 2024-04-25T01:13:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.304+0000 7fe99f200640 1 -- start start 2024-04-25T01:13:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.304+0000 7fe99f200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe998107c80 0x7fe99819df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.304+0000 7fe99f200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe998108e70 0x7fe99819e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.304+0000 7fe99f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe998109830 0x7fe9981a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.304+0000 7fe99f200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe998113900 con 0x7fe998107c80 2024-04-25T01:13:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.304+0000 7fe99f200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe998113780 con 0x7fe998108e70 2024-04-25T01:13:37.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.304+0000 7fe99f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe998113a80 con 0x7fe998109830 2024-04-25T01:13:37.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.305+0000 7fe99de00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe998107c80 0x7fe99819df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:37.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.305+0000 7fe99d400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe998108e70 0x7fe99819e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:37.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.305+0000 7fe99de00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe998107c80 0x7fe99819df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44060/0 (socket says 172.21.15.31:44060) 2024-04-25T01:13:37.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.305+0000 7fe99de00640 1 -- 172.21.15.31:0/1608601500 learned_addr learned my addr 172.21.15.31:0/1608601500 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:37.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.305+0000 7fe99e800640 1 --2- 172.21.15.31:0/1608601500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe998109830 0x7fe9981a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:37.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.305+0000 7fe99d400640 1 -- 172.21.15.31:0/1608601500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe998109830 msgr2=0x7fe9981a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:37.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.305+0000 7fe99d400640 1 --2- 172.21.15.31:0/1608601500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe998109830 0x7fe9981a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:37.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.305+0000 7fe99d400640 1 -- 172.21.15.31:0/1608601500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe998107c80 msgr2=0x7fe99819df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:37.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.305+0000 7fe99d400640 1 --2- 172.21.15.31:0/1608601500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe998107c80 0x7fe99819df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:37.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.305+0000 7fe99d400640 1 -- 172.21.15.31:0/1608601500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9981a5b30 con 0x7fe998108e70 2024-04-25T01:13:37.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.305+0000 7fe99e800640 1 --2- 172.21.15.31:0/1608601500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe998109830 0x7fe9981a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:13:37.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.306+0000 7fe99d400640 1 --2- 172.21.15.31:0/1608601500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe998108e70 0x7fe99819e450 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe984005fd0 tx=0x7fe98400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:37.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.306+0000 7fe992a00640 1 -- 172.21.15.31:0/1608601500 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98400cd00 con 0x7fe998108e70 2024-04-25T01:13:37.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.306+0000 7fe992a00640 1 -- 172.21.15.31:0/1608601500 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9840035f0 con 0x7fe998108e70 2024-04-25T01:13:37.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.306+0000 7fe992a00640 1 -- 172.21.15.31:0/1608601500 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984010690 con 0x7fe998108e70 2024-04-25T01:13:37.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.306+0000 7fe99f200640 1 -- 172.21.15.31:0/1608601500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9981a5e20 con 0x7fe998108e70 2024-04-25T01:13:37.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.306+0000 7fe99f200640 1 -- 172.21.15.31:0/1608601500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9981a6330 con 0x7fe998108e70 2024-04-25T01:13:37.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.307+0000 7fe99f200640 1 -- 172.21.15.31:0/1608601500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe99810ef40 con 0x7fe998108e70 2024-04-25T01:13:37.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.309+0000 7fe992a00640 1 -- 172.21.15.31:0/1608601500 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe9840037a0 con 0x7fe998108e70 2024-04-25T01:13:37.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.310+0000 7fe992a00640 1 --2- 172.21.15.31:0/1608601500 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe9800a2c10 0x7fe9800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:37.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.310+0000 7fe992a00640 1 -- 172.21.15.31:0/1608601500 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fe9840cf220 con 0x7fe998108e70 2024-04-25T01:13:37.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.311+0000 7fe99de00640 1 --2- 172.21.15.31:0/1608601500 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe9800a2c10 0x7fe9800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:37.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.311+0000 7fe99de00640 1 --2- 172.21.15.31:0/1608601500 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe9800a2c10 0x7fe9800a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe98c005fd0 tx=0x7fe98c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:37.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.314+0000 7fe992a00640 1 -- 172.21.15.31:0/1608601500 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe984009510 con 0x7fe998108e70 2024-04-25T01:13:37.435 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:13:37 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:13:37.050+0000 7fda1b000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T01:13:37.435 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:13:37 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:13:37.050+0000 7fda1b000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T01:13:37.435 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:13:37 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:13:37.050+0000 7fda1b000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T01:13:37.435 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:13:37 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:13:37.050+0000 7fda1b000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T01:13:37.436 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:13:37 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:13:37.050+0000 7fda1b000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T01:13:37.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.538+0000 7fe99f200640 1 -- 172.21.15.31:0/1608601500 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe998061b80 con 0x7fe9800a2c10 2024-04-25T01:13:37.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.577+0000 7fe992a00640 1 -- 172.21.15.31:0/1608601500 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe998061b80 con 0x7fe9800a2c10 2024-04-25T01:13:37.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.581+0000 7fe99f200640 1 -- 172.21.15.31:0/1608601500 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe9800a2c10 msgr2=0x7fe9800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:37.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.581+0000 7fe99f200640 1 --2- 172.21.15.31:0/1608601500 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe9800a2c10 0x7fe9800a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe98c005fd0 tx=0x7fe98c00b410 comp rx=0 tx=0).stop 2024-04-25T01:13:37.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.581+0000 7fe99f200640 1 -- 172.21.15.31:0/1608601500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe998108e70 msgr2=0x7fe99819e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:37.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.581+0000 7fe99f200640 1 --2- 172.21.15.31:0/1608601500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe998108e70 0x7fe99819e450 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe984005fd0 tx=0x7fe98400ee50 comp rx=0 tx=0).stop 2024-04-25T01:13:37.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.582+0000 7fe99f200640 1 -- 172.21.15.31:0/1608601500 shutdown_connections 2024-04-25T01:13:37.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.582+0000 7fe99f200640 1 --2- 172.21.15.31:0/1608601500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe998109830 0x7fe9981a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:37.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.582+0000 7fe99f200640 1 --2- 172.21.15.31:0/1608601500 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe9800a2c10 0x7fe9800a50d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:37.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.582+0000 7fe99f200640 1 --2- 172.21.15.31:0/1608601500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe998108e70 0x7fe99819e450 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:37.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.582+0000 7fe99f200640 1 --2- 172.21.15.31:0/1608601500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe998107c80 0x7fe99819df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:37.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.582+0000 7fe99f200640 1 -- 172.21.15.31:0/1608601500 >> 172.21.15.31:0/1608601500 conn(0x7fe998076c10 msgr2=0x7fe99810c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:37.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.582+0000 7fe99f200640 1 -- 172.21.15.31:0/1608601500 shutdown_connections 2024-04-25T01:13:37.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.582+0000 7fe99f200640 1 -- 172.21.15.31:0/1608601500 wait complete. 2024-04-25T01:13:37.588 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T01:13:37.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:37 smithi031 ceph-mon[28053]: pgmap v319: 97 pgs: 91 active+clean, 6 creating+peering; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-04-25T01:13:37.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.812+0000 7f5032800640 1 -- 172.21.15.31:0/614245654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f502c1039a0 msgr2=0x7f502c103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:37.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.812+0000 7f5032800640 1 --2- 172.21.15.31:0/614245654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f502c1039a0 0x7f502c103dc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f501c009ca0 tx=0x7f501c02a8c0 comp rx=0 tx=0).stop 2024-04-25T01:13:37.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.812+0000 7f5032800640 1 -- 172.21.15.31:0/614245654 shutdown_connections 2024-04-25T01:13:37.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.812+0000 7f5032800640 1 --2- 172.21.15.31:0/614245654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f502c105550 0x7f502c10bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:37.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.812+0000 7f5032800640 1 --2- 172.21.15.31:0/614245654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f502c104b90 0x7f502c105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:37.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.812+0000 7f5032800640 1 --2- 172.21.15.31:0/614245654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f502c1039a0 0x7f502c103dc0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:37.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.812+0000 7f5032800640 1 -- 172.21.15.31:0/614245654 >> 172.21.15.31:0/614245654 conn(0x7f502c0fef30 msgr2=0x7f502c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:37.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.812+0000 7f5032800640 1 -- 172.21.15.31:0/614245654 shutdown_connections 2024-04-25T01:13:37.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.812+0000 7f5032800640 1 -- 172.21.15.31:0/614245654 wait complete. 2024-04-25T01:13:37.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.813+0000 7f5032800640 1 Processor -- start 2024-04-25T01:13:37.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.813+0000 7f5032800640 1 -- start start 2024-04-25T01:13:37.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.813+0000 7f5032800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f502c1039a0 0x7f502c199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:37.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.813+0000 7f5032800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f502c104b90 0x7f502c19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:37.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.813+0000 7f5032800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f502c105550 0x7f502c1a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:37.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.813+0000 7f5032800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f502c10f660 con 0x7f502c105550 2024-04-25T01:13:37.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.813+0000 7f5032800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f502c10f4e0 con 0x7f502c1039a0 2024-04-25T01:13:37.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.813+0000 7f5032800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f502c10f7e0 con 0x7f502c104b90 2024-04-25T01:13:37.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.813+0000 7f502b400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f502c1039a0 0x7f502c199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:37.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.814+0000 7f502b400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f502c1039a0 0x7f502c199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:40842/0 (socket says 172.21.15.31:40842) 2024-04-25T01:13:37.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.814+0000 7f502b400640 1 -- 172.21.15.31:0/4216560675 learned_addr learned my addr 172.21.15.31:0/4216560675 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:37.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.814+0000 7f502be00640 1 --2- 172.21.15.31:0/4216560675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f502c105550 0x7f502c1a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:37.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.814+0000 7f502aa00640 1 --2- 172.21.15.31:0/4216560675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f502c104b90 0x7f502c19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:37.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.814+0000 7f502b400640 1 -- 172.21.15.31:0/4216560675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f502c104b90 msgr2=0x7f502c19a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:37.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.814+0000 7f502b400640 1 --2- 172.21.15.31:0/4216560675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f502c104b90 0x7f502c19a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:37.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.814+0000 7f502b400640 1 -- 172.21.15.31:0/4216560675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f502c105550 msgr2=0x7f502c1a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:37.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.814+0000 7f502b400640 1 --2- 172.21.15.31:0/4216560675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f502c105550 0x7f502c1a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:37.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.814+0000 7f502b400640 1 -- 172.21.15.31:0/4216560675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502c1a1960 con 0x7f502c1039a0 2024-04-25T01:13:37.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.814+0000 7f502aa00640 1 --2- 172.21.15.31:0/4216560675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f502c104b90 0x7f502c19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:13:37.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.814+0000 7f502b400640 1 --2- 172.21.15.31:0/4216560675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f502c1039a0 0x7f502c199d50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f501c000f80 tx=0x7f501c0319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:37.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.815+0000 7f5023e00640 1 -- 172.21.15.31:0/4216560675 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c003bf0 con 0x7f502c1039a0 2024-04-25T01:13:37.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.815+0000 7f5023e00640 1 -- 172.21.15.31:0/4216560675 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f501c003d90 con 0x7f502c1039a0 2024-04-25T01:13:37.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.815+0000 7f5023e00640 1 -- 172.21.15.31:0/4216560675 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c02fd30 con 0x7f502c1039a0 2024-04-25T01:13:37.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.815+0000 7f5032800640 1 -- 172.21.15.31:0/4216560675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f502c1a1bf0 con 0x7f502c1039a0 2024-04-25T01:13:37.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.815+0000 7f5032800640 1 -- 172.21.15.31:0/4216560675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f502c1a2190 con 0x7f502c1039a0 2024-04-25T01:13:37.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.816+0000 7f5032800640 1 -- 172.21.15.31:0/4216560675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f502c10ad60 con 0x7f502c1039a0 2024-04-25T01:13:37.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.818+0000 7f5023e00640 1 -- 172.21.15.31:0/4216560675 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f501c004040 con 0x7f502c1039a0 2024-04-25T01:13:37.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.819+0000 7f5023e00640 1 --2- 172.21.15.31:0/4216560675 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f50100a2c10 0x7f50100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:37.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.819+0000 7f5023e00640 1 -- 172.21.15.31:0/4216560675 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f501c0f0460 con 0x7f502c1039a0 2024-04-25T01:13:37.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.819+0000 7f502aa00640 1 --2- 172.21.15.31:0/4216560675 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f50100a2c10 0x7f50100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:37.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.820+0000 7f502aa00640 1 --2- 172.21.15.31:0/4216560675 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f50100a2c10 0x7f50100a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5014009c20 tx=0x7f5014005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:37.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:37.823+0000 7f5023e00640 1 -- 172.21.15.31:0/4216560675 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f501c0bd470 con 0x7f502c1039a0 2024-04-25T01:13:37.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:37 smithi169 ceph-mon[32759]: pgmap v319: 97 pgs: 91 active+clean, 6 creating+peering; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-04-25T01:13:37.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:37 smithi040 ceph-mon[32518]: pgmap v319: 97 pgs: 91 active+clean, 6 creating+peering; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-04-25T01:13:38.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.048+0000 7f5032800640 1 -- 172.21.15.31:0/4216560675 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f502c061b80 con 0x7f50100a2c10 2024-04-25T01:13:38.044 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/qa/sv_0/7e634fd0-3a32-4519-9ebf-9c0417b88cc5 2024-04-25T01:13:38.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.057+0000 7f5023e00640 1 -- 172.21.15.31:0/4216560675 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f502c061b80 con 0x7f50100a2c10 2024-04-25T01:13:38.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.060+0000 7f5032800640 1 -- 172.21.15.31:0/4216560675 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f50100a2c10 msgr2=0x7f50100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:38.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.060+0000 7f5032800640 1 --2- 172.21.15.31:0/4216560675 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f50100a2c10 0x7f50100a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5014009c20 tx=0x7f5014005f90 comp rx=0 tx=0).stop 2024-04-25T01:13:38.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.060+0000 7f5032800640 1 -- 172.21.15.31:0/4216560675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f502c1039a0 msgr2=0x7f502c199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:38.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.060+0000 7f5032800640 1 --2- 172.21.15.31:0/4216560675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f502c1039a0 0x7f502c199d50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f501c000f80 tx=0x7f501c0319f0 comp rx=0 tx=0).stop 2024-04-25T01:13:38.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.060+0000 7f5032800640 1 -- 172.21.15.31:0/4216560675 shutdown_connections 2024-04-25T01:13:38.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.060+0000 7f5032800640 1 --2- 172.21.15.31:0/4216560675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f502c105550 0x7f502c1a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:38.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.060+0000 7f5032800640 1 --2- 172.21.15.31:0/4216560675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f502c104b90 0x7f502c19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:38.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.060+0000 7f5032800640 1 --2- 172.21.15.31:0/4216560675 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f50100a2c10 0x7f50100a50d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:38.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.060+0000 7f5032800640 1 --2- 172.21.15.31:0/4216560675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f502c1039a0 0x7f502c199d50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:38.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.060+0000 7f5032800640 1 -- 172.21.15.31:0/4216560675 >> 172.21.15.31:0/4216560675 conn(0x7f502c0fef30 msgr2=0x7f502c107f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:38.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.060+0000 7f5032800640 1 -- 172.21.15.31:0/4216560675 shutdown_connections 2024-04-25T01:13:38.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.060+0000 7f5032800640 1 -- 172.21.15.31:0/4216560675 wait complete. 2024-04-25T01:13:38.067 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T01:13:38.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.302+0000 7f9315000640 1 -- 172.21.15.31:0/500133653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93100ffec0 msgr2=0x7f93101002e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:38.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.302+0000 7f9315000640 1 --2- 172.21.15.31:0/500133653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93100ffec0 0x7f93101002e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9300009be0 tx=0x7f930002a6c0 comp rx=0 tx=0).stop 2024-04-25T01:13:38.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.302+0000 7f9315000640 1 -- 172.21.15.31:0/500133653 shutdown_connections 2024-04-25T01:13:38.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.302+0000 7f9315000640 1 --2- 172.21.15.31:0/500133653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93101011c0 0x7f9310101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:38.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.302+0000 7f9315000640 1 --2- 172.21.15.31:0/500133653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9310100820 0x7f9310100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:38.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.302+0000 7f9315000640 1 --2- 172.21.15.31:0/500133653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93100ffec0 0x7f93101002e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:38.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.302+0000 7f9315000640 1 -- 172.21.15.31:0/500133653 >> 172.21.15.31:0/500133653 conn(0x7f93100fbae0 msgr2=0x7f93100fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:38.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.302+0000 7f9315000640 1 -- 172.21.15.31:0/500133653 shutdown_connections 2024-04-25T01:13:38.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.302+0000 7f9315000640 1 -- 172.21.15.31:0/500133653 wait complete. 2024-04-25T01:13:38.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.303+0000 7f9315000640 1 Processor -- start 2024-04-25T01:13:38.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.303+0000 7f9315000640 1 -- start start 2024-04-25T01:13:38.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.303+0000 7f9315000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93100ffec0 0x7f9310199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:38.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.303+0000 7f9315000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9310100820 0x7f931019a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:38.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.303+0000 7f9315000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93101011c0 0x7f93101a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:38.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.303+0000 7f9315000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f931010f490 con 0x7f93101011c0 2024-04-25T01:13:38.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.303+0000 7f9315000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f931010f310 con 0x7f9310100820 2024-04-25T01:13:38.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.303+0000 7f9315000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f931010f610 con 0x7f93100ffec0 2024-04-25T01:13:38.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.304+0000 7f930ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9310100820 0x7f931019a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:38.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.304+0000 7f930ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9310100820 0x7f931019a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:40870/0 (socket says 172.21.15.31:40870) 2024-04-25T01:13:38.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.304+0000 7f930fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93101011c0 0x7f93101a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:38.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.304+0000 7f930ea00640 1 -- 172.21.15.31:0/964355537 learned_addr learned my addr 172.21.15.31:0/964355537 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:38.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.304+0000 7f930f400640 1 --2- 172.21.15.31:0/964355537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93100ffec0 0x7f9310199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:38.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.304+0000 7f930ea00640 1 -- 172.21.15.31:0/964355537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93100ffec0 msgr2=0x7f9310199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:38.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.304+0000 7f930ea00640 1 --2- 172.21.15.31:0/964355537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93100ffec0 0x7f9310199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:38.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.304+0000 7f930ea00640 1 -- 172.21.15.31:0/964355537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93101011c0 msgr2=0x7f93101a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:38.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.304+0000 7f930ea00640 1 --2- 172.21.15.31:0/964355537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93101011c0 0x7f93101a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:38.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.304+0000 7f930ea00640 1 -- 172.21.15.31:0/964355537 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93101a1760 con 0x7f9310100820 2024-04-25T01:13:38.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.304+0000 7f930fe00640 1 --2- 172.21.15.31:0/964355537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93101011c0 0x7f93101a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:13:38.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.305+0000 7f930ea00640 1 --2- 172.21.15.31:0/964355537 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9310100820 0x7f931019a080 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f92f800e9c0 tx=0x7f92f800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:38.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.305+0000 7f9307e00640 1 -- 172.21.15.31:0/964355537 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f800cd80 con 0x7f9310100820 2024-04-25T01:13:38.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.305+0000 7f9307e00640 1 -- 172.21.15.31:0/964355537 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92f80035f0 con 0x7f9310100820 2024-04-25T01:13:38.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.305+0000 7f9307e00640 1 -- 172.21.15.31:0/964355537 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f8010670 con 0x7f9310100820 2024-04-25T01:13:38.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.305+0000 7f9315000640 1 -- 172.21.15.31:0/964355537 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93101a1a50 con 0x7f9310100820 2024-04-25T01:13:38.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.305+0000 7f9315000640 1 -- 172.21.15.31:0/964355537 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93101a1ff0 con 0x7f9310100820 2024-04-25T01:13:38.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.306+0000 7f9315000640 1 -- 172.21.15.31:0/964355537 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f931010adf0 con 0x7f9310100820 2024-04-25T01:13:38.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.308+0000 7f9307e00640 1 -- 172.21.15.31:0/964355537 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f92f80037a0 con 0x7f9310100820 2024-04-25T01:13:38.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.310+0000 7f9307e00640 1 --2- 172.21.15.31:0/964355537 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f92f40a2c10 0x7f92f40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:38.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.310+0000 7f9307e00640 1 -- 172.21.15.31:0/964355537 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f92f80cf130 con 0x7f9310100820 2024-04-25T01:13:38.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.310+0000 7f930f400640 1 --2- 172.21.15.31:0/964355537 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f92f40a2c10 0x7f92f40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:38.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.311+0000 7f930f400640 1 --2- 172.21.15.31:0/964355537 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f92f40a2c10 0x7f92f40a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f930002acb0 tx=0x7f9300005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:38.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.314+0000 7f9307e00640 1 -- 172.21.15.31:0/964355537 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f92f809c140 con 0x7f9310100820 2024-04-25T01:13:38.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.552+0000 7f9315000640 1 -- 172.21.15.31:0/964355537 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9310061b80 con 0x7f92f40a2c10 2024-04-25T01:13:38.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.589+0000 7f9307e00640 1 -- 172.21.15.31:0/964355537 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9310061b80 con 0x7f92f40a2c10 2024-04-25T01:13:38.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.592+0000 7f9315000640 1 -- 172.21.15.31:0/964355537 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f92f40a2c10 msgr2=0x7f92f40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:38.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.592+0000 7f9315000640 1 --2- 172.21.15.31:0/964355537 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f92f40a2c10 0x7f92f40a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f930002acb0 tx=0x7f9300005fd0 comp rx=0 tx=0).stop 2024-04-25T01:13:38.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.592+0000 7f9315000640 1 -- 172.21.15.31:0/964355537 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9310100820 msgr2=0x7f931019a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:38.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.592+0000 7f9315000640 1 --2- 172.21.15.31:0/964355537 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9310100820 0x7f931019a080 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f92f800e9c0 tx=0x7f92f800edc0 comp rx=0 tx=0).stop 2024-04-25T01:13:38.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.592+0000 7f9315000640 1 -- 172.21.15.31:0/964355537 shutdown_connections 2024-04-25T01:13:38.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.592+0000 7f9315000640 1 --2- 172.21.15.31:0/964355537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93101011c0 0x7f93101a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:38.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.592+0000 7f9315000640 1 --2- 172.21.15.31:0/964355537 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f92f40a2c10 0x7f92f40a50d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:38.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.592+0000 7f9315000640 1 --2- 172.21.15.31:0/964355537 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9310100820 0x7f931019a080 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:38.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.592+0000 7f9315000640 1 --2- 172.21.15.31:0/964355537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93100ffec0 0x7f9310199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:38.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.592+0000 7f9315000640 1 -- 172.21.15.31:0/964355537 >> 172.21.15.31:0/964355537 conn(0x7f93100fbae0 msgr2=0x7f93100fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:38.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.593+0000 7f9315000640 1 -- 172.21.15.31:0/964355537 shutdown_connections 2024-04-25T01:13:38.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.593+0000 7f9315000640 1 -- 172.21.15.31:0/964355537 wait complete. 2024-04-25T01:13:38.605 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T01:13:38.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.819+0000 7fe161600640 1 -- 172.21.15.31:0/1921227819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe15c076100 msgr2=0x7fe15c076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:38.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.819+0000 7fe161600640 1 --2- 172.21.15.31:0/1921227819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe15c076100 0x7fe15c076520 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe14c009900 tx=0x7fe14c02a400 comp rx=0 tx=0).stop 2024-04-25T01:13:38.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.819+0000 7fe161600640 1 -- 172.21.15.31:0/1921227819 shutdown_connections 2024-04-25T01:13:38.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.819+0000 7fe161600640 1 --2- 172.21.15.31:0/1921227819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe15c077ce0 0x7fe15c078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:38.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.819+0000 7fe161600640 1 --2- 172.21.15.31:0/1921227819 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe15c077340 0x7fe15c0777a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:38.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.819+0000 7fe161600640 1 --2- 172.21.15.31:0/1921227819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe15c076100 0x7fe15c076520 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:38.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.819+0000 7fe161600640 1 -- 172.21.15.31:0/1921227819 >> 172.21.15.31:0/1921227819 conn(0x7fe15c0fedf0 msgr2=0x7fe15c101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:38.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.819+0000 7fe161600640 1 -- 172.21.15.31:0/1921227819 shutdown_connections 2024-04-25T01:13:38.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.819+0000 7fe161600640 1 -- 172.21.15.31:0/1921227819 wait complete. 2024-04-25T01:13:38.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.820+0000 7fe161600640 1 Processor -- start 2024-04-25T01:13:38.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.821+0000 7fe161600640 1 -- start start 2024-04-25T01:13:38.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.821+0000 7fe161600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe15c076100 0x7fe15c19df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:38.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.821+0000 7fe161600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe15c077340 0x7fe15c19e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:38.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.821+0000 7fe161600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe15c077ce0 0x7fe15c1a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:38.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.821+0000 7fe161600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c113910 con 0x7fe15c077340 2024-04-25T01:13:38.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.821+0000 7fe161600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c113790 con 0x7fe15c076100 2024-04-25T01:13:38.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.821+0000 7fe161600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c113a90 con 0x7fe15c077ce0 2024-04-25T01:13:38.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.821+0000 7fe15b400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe15c077340 0x7fe15c19e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:38.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.821+0000 7fe15b400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe15c077340 0x7fe15c19e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44108/0 (socket says 172.21.15.31:44108) 2024-04-25T01:13:38.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.821+0000 7fe15b400640 1 -- 172.21.15.31:0/1828531107 learned_addr learned my addr 172.21.15.31:0/1828531107 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:38.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.821+0000 7fe15be00640 1 --2- 172.21.15.31:0/1828531107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe15c076100 0x7fe15c19df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:38.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.821+0000 7fe160c00640 1 --2- 172.21.15.31:0/1828531107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe15c077ce0 0x7fe15c1a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:38.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.822+0000 7fe15b400640 1 -- 172.21.15.31:0/1828531107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe15c077ce0 msgr2=0x7fe15c1a5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:38.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.822+0000 7fe15b400640 1 --2- 172.21.15.31:0/1828531107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe15c077ce0 0x7fe15c1a5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:38.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.822+0000 7fe15b400640 1 -- 172.21.15.31:0/1828531107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe15c076100 msgr2=0x7fe15c19df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:38.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.822+0000 7fe15b400640 1 --2- 172.21.15.31:0/1828531107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe15c076100 0x7fe15c19df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:38.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.822+0000 7fe15b400640 1 -- 172.21.15.31:0/1828531107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15c1a5b90 con 0x7fe15c077340 2024-04-25T01:13:38.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.822+0000 7fe15be00640 1 --2- 172.21.15.31:0/1828531107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe15c076100 0x7fe15c19df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:13:38.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.822+0000 7fe15b400640 1 --2- 172.21.15.31:0/1828531107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe15c077340 0x7fe15c19e4b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe14800ea60 tx=0x7fe14800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:38.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.822+0000 7fe158c00640 1 -- 172.21.15.31:0/1828531107 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14800cde0 con 0x7fe15c077340 2024-04-25T01:13:38.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.822+0000 7fe158c00640 1 -- 172.21.15.31:0/1828531107 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1480035f0 con 0x7fe15c077340 2024-04-25T01:13:38.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.822+0000 7fe161600640 1 -- 172.21.15.31:0/1828531107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe15c1a5e80 con 0x7fe15c077340 2024-04-25T01:13:38.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.822+0000 7fe158c00640 1 -- 172.21.15.31:0/1828531107 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148010670 con 0x7fe15c077340 2024-04-25T01:13:38.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.822+0000 7fe161600640 1 -- 172.21.15.31:0/1828531107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe15c1a6340 con 0x7fe15c077340 2024-04-25T01:13:38.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.824+0000 7fe161600640 1 -- 172.21.15.31:0/1828531107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe15c1101f0 con 0x7fe15c077340 2024-04-25T01:13:38.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.826+0000 7fe158c00640 1 -- 172.21.15.31:0/1828531107 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe1480037a0 con 0x7fe15c077340 2024-04-25T01:13:38.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.828+0000 7fe158c00640 1 --2- 172.21.15.31:0/1828531107 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe1400a2c20 0x7fe1400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:38.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.828+0000 7fe158c00640 1 -- 172.21.15.31:0/1828531107 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fe1480cf3f0 con 0x7fe15c077340 2024-04-25T01:13:38.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.828+0000 7fe15be00640 1 --2- 172.21.15.31:0/1828531107 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe1400a2c20 0x7fe1400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:38.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.829+0000 7fe15be00640 1 --2- 172.21.15.31:0/1828531107 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe1400a2c20 0x7fe1400a50e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0314e0 tx=0x7fe14c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:38.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:38.833+0000 7fe158c00640 1 -- 172.21.15.31:0/1828531107 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe14809c2a0 con 0x7fe15c077340 2024-04-25T01:13:38.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:38 smithi169 ceph-mon[32759]: from='client.24686 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:13:38.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:38 smithi169 ceph-mon[32759]: from='client.24698 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:13:38.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:38 smithi040 ceph-mon[32518]: from='client.24686 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:13:38.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:38 smithi040 ceph-mon[32518]: from='client.24698 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:13:39.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:38 smithi031 ceph-mon[28053]: from='client.24686 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:13:39.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:38 smithi031 ceph-mon[28053]: from='client.24698 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:13:39.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.073+0000 7fe161600640 1 -- 172.21.15.31:0/1828531107 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fe15c061b80 con 0x7fe1400a2c20 2024-04-25T01:13:39.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.084+0000 7fe158c00640 1 -- 172.21.15.31:0/1828531107 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fe15c061b80 con 0x7fe1400a2c20 2024-04-25T01:13:39.071 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/qa/sv_1/1c94ca6f-03d3-430b-88f5-fb1cad13c507 2024-04-25T01:13:39.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.087+0000 7fe161600640 1 -- 172.21.15.31:0/1828531107 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe1400a2c20 msgr2=0x7fe1400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:39.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.087+0000 7fe161600640 1 --2- 172.21.15.31:0/1828531107 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe1400a2c20 0x7fe1400a50e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0314e0 tx=0x7fe14c000bc0 comp rx=0 tx=0).stop 2024-04-25T01:13:39.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.087+0000 7fe161600640 1 -- 172.21.15.31:0/1828531107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe15c077340 msgr2=0x7fe15c19e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:39.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.087+0000 7fe161600640 1 --2- 172.21.15.31:0/1828531107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe15c077340 0x7fe15c19e4b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe14800ea60 tx=0x7fe14800ef20 comp rx=0 tx=0).stop 2024-04-25T01:13:39.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.087+0000 7fe161600640 1 -- 172.21.15.31:0/1828531107 shutdown_connections 2024-04-25T01:13:39.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.087+0000 7fe161600640 1 --2- 172.21.15.31:0/1828531107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe15c077ce0 0x7fe15c1a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:39.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.087+0000 7fe161600640 1 --2- 172.21.15.31:0/1828531107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe15c077340 0x7fe15c19e4b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:39.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.087+0000 7fe161600640 1 --2- 172.21.15.31:0/1828531107 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fe1400a2c20 0x7fe1400a50e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:39.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.087+0000 7fe161600640 1 --2- 172.21.15.31:0/1828531107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe15c076100 0x7fe15c19df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:39.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.087+0000 7fe161600640 1 -- 172.21.15.31:0/1828531107 >> 172.21.15.31:0/1828531107 conn(0x7fe15c0fedf0 msgr2=0x7fe15c102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:39.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.087+0000 7fe161600640 1 -- 172.21.15.31:0/1828531107 shutdown_connections 2024-04-25T01:13:39.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.087+0000 7fe161600640 1 -- 172.21.15.31:0/1828531107 wait complete. 2024-04-25T01:13:39.094 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:13:39.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.322+0000 7f0181800640 1 -- 172.21.15.31:0/2133045843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017c0741a0 msgr2=0x7f017c074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:39.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.322+0000 7f0181800640 1 --2- 172.21.15.31:0/2133045843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017c0741a0 0x7f017c074620 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0164009930 tx=0x7f016402a440 comp rx=0 tx=0).stop 2024-04-25T01:13:39.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.322+0000 7f0181800640 1 -- 172.21.15.31:0/2133045843 shutdown_connections 2024-04-25T01:13:39.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.322+0000 7f0181800640 1 --2- 172.21.15.31:0/2133045843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f017c10db80 0x7f017c10ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:39.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.322+0000 7f0181800640 1 --2- 172.21.15.31:0/2133045843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017c0741a0 0x7f017c074620 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:39.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.322+0000 7f0181800640 1 --2- 172.21.15.31:0/2133045843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f017c075780 0x7f017c073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:39.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.322+0000 7f0181800640 1 -- 172.21.15.31:0/2133045843 >> 172.21.15.31:0/2133045843 conn(0x7f017c0fcb20 msgr2=0x7f017c0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:39.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.322+0000 7f0181800640 1 -- 172.21.15.31:0/2133045843 shutdown_connections 2024-04-25T01:13:39.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.322+0000 7f0181800640 1 -- 172.21.15.31:0/2133045843 wait complete. 2024-04-25T01:13:39.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.323+0000 7f0181800640 1 Processor -- start 2024-04-25T01:13:39.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.324+0000 7f0181800640 1 -- start start 2024-04-25T01:13:39.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.324+0000 7f0181800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017c0741a0 0x7f017c19df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:39.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.324+0000 7f0181800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f017c075780 0x7f017c19e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:39.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.324+0000 7f0181800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f017c10db80 0x7f017c1a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:39.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.324+0000 7f0181800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f017c113870 con 0x7f017c0741a0 2024-04-25T01:13:39.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.324+0000 7f0181800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f017c1136f0 con 0x7f017c10db80 2024-04-25T01:13:39.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.324+0000 7f0181800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f017c1139f0 con 0x7f017c075780 2024-04-25T01:13:39.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.324+0000 7f017be00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017c0741a0 0x7f017c19df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:39.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.324+0000 7f017be00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017c0741a0 0x7f017c19df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44136/0 (socket says 172.21.15.31:44136) 2024-04-25T01:13:39.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.324+0000 7f017be00640 1 -- 172.21.15.31:0/311670140 learned_addr learned my addr 172.21.15.31:0/311670140 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:39.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.324+0000 7f017b400640 1 --2- 172.21.15.31:0/311670140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f017c075780 0x7f017c19e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:39.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.324+0000 7f0180e00640 1 --2- 172.21.15.31:0/311670140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f017c10db80 0x7f017c1a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:39.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.325+0000 7f017be00640 1 -- 172.21.15.31:0/311670140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f017c075780 msgr2=0x7f017c19e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:39.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.325+0000 7f017be00640 1 --2- 172.21.15.31:0/311670140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f017c075780 0x7f017c19e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:39.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.325+0000 7f017be00640 1 -- 172.21.15.31:0/311670140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f017c10db80 msgr2=0x7f017c1a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:39.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.325+0000 7f017be00640 1 --2- 172.21.15.31:0/311670140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f017c10db80 0x7f017c1a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:39.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.325+0000 7f017be00640 1 -- 172.21.15.31:0/311670140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017c1a5b30 con 0x7f017c0741a0 2024-04-25T01:13:39.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.325+0000 7f017be00640 1 --2- 172.21.15.31:0/311670140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017c0741a0 0x7f017c19df10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f016c00d950 tx=0x7f016c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:39.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.325+0000 7f0178c00640 1 -- 172.21.15.31:0/311670140 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c017030 con 0x7f017c0741a0 2024-04-25T01:13:39.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.325+0000 7f0178c00640 1 -- 172.21.15.31:0/311670140 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f016c004020 con 0x7f017c0741a0 2024-04-25T01:13:39.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.325+0000 7f0181800640 1 -- 172.21.15.31:0/311670140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f017c1a5e20 con 0x7f017c0741a0 2024-04-25T01:13:39.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.325+0000 7f0178c00640 1 -- 172.21.15.31:0/311670140 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c0047a0 con 0x7f017c0741a0 2024-04-25T01:13:39.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.325+0000 7f0181800640 1 -- 172.21.15.31:0/311670140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f017c1a6280 con 0x7f017c0741a0 2024-04-25T01:13:39.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.327+0000 7f0181800640 1 -- 172.21.15.31:0/311670140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f017c103d50 con 0x7f017c0741a0 2024-04-25T01:13:39.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.329+0000 7f0178c00640 1 -- 172.21.15.31:0/311670140 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f016c0041d0 con 0x7f017c0741a0 2024-04-25T01:13:39.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.330+0000 7f0178c00640 1 --2- 172.21.15.31:0/311670140 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f01600a2c20 0x7f01600a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:39.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.330+0000 7f0178c00640 1 -- 172.21.15.31:0/311670140 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f016c0cead0 con 0x7f017c0741a0 2024-04-25T01:13:39.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.330+0000 7f017b400640 1 --2- 172.21.15.31:0/311670140 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f01600a2c20 0x7f01600a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:39.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.331+0000 7f017b400640 1 --2- 172.21.15.31:0/311670140 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f01600a2c20 0x7f01600a50e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0164009900 tx=0x7f01640315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:39.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.334+0000 7f0178c00640 1 -- 172.21.15.31:0/311670140 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f016c09b980 con 0x7f017c0741a0 2024-04-25T01:13:39.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.579+0000 7f0181800640 1 -- 172.21.15.31:0/311670140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f017c02d400 con 0x7f017c0741a0 2024-04-25T01:13:39.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.581+0000 7f0178c00640 1 -- 172.21.15.31:0/311670140 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11854 (secure 0 0 0) 0x7f016c09b280 con 0x7f017c0741a0 2024-04-25T01:13:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:13:39.568 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/498218714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":498218714},{"type":"v1","addr":"172.21.15.31:6835","nonce":498218714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2647199475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2647199475},{"type":"v1","addr":"172.21.15.31:6839","nonce":2647199475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/30518311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":30518311},{"type":"v1","addr":"172.21.15.31:6841","nonce":30518311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/420362823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":420362823},{"type":"v1","addr":"172.21.15.169:6833","nonce":420362823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1943710438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1943710438},{"type":"v1","addr":"172.21.15.40:6833","nonce":1943710438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/4191344115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":4191344115},{"type":"v1","addr":"172.21.15.31:6837","nonce":4191344115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:13:30.975848+0000","modified":"2024-04-25T01:13:36.545846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24565,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6835/2916305801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2916305801},{"type":"v1","addr":"172.21.15.40:6835","nonce":2916305801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.169:6835/2253264784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2253264784},{"type":"v1","addr":"172.21.15.169:6835","nonce":2253264784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6837/2883308641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2883308641},{"type":"v1","addr":"172.21.15.169:6837","nonce":2883308641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/590109562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":590109562},{"type":"v1","addr":"172.21.15.40:6837","nonce":590109562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/4185710218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":4185710218},{"type":"v1","addr":"172.21.15.169:6839","nonce":4185710218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6839/927194004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":927194004},{"type":"v1","addr":"172.21.15.40:6839","nonce":927194004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24532,24565,24596]},"id":1}]} 2024-04-25T01:13:39.568 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 23 2024-04-25T01:13:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.583+0000 7f0181800640 1 -- 172.21.15.31:0/311670140 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f01600a2c20 msgr2=0x7f01600a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.583+0000 7f0181800640 1 --2- 172.21.15.31:0/311670140 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f01600a2c20 0x7f01600a50e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0164009900 tx=0x7f01640315b0 comp rx=0 tx=0).stop 2024-04-25T01:13:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.583+0000 7f0181800640 1 -- 172.21.15.31:0/311670140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017c0741a0 msgr2=0x7f017c19df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.583+0000 7f0181800640 1 --2- 172.21.15.31:0/311670140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017c0741a0 0x7f017c19df10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f016c00d950 tx=0x7f016c00de10 comp rx=0 tx=0).stop 2024-04-25T01:13:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.583+0000 7f0181800640 1 -- 172.21.15.31:0/311670140 shutdown_connections 2024-04-25T01:13:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.583+0000 7f0181800640 1 --2- 172.21.15.31:0/311670140 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f01600a2c20 0x7f01600a50e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.583+0000 7f0181800640 1 --2- 172.21.15.31:0/311670140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f017c10db80 0x7f017c1a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.583+0000 7f0181800640 1 --2- 172.21.15.31:0/311670140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f017c075780 0x7f017c19e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.584+0000 7f0181800640 1 --2- 172.21.15.31:0/311670140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017c0741a0 0x7f017c19df10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.584+0000 7f0181800640 1 -- 172.21.15.31:0/311670140 >> 172.21.15.31:0/311670140 conn(0x7f017c0fcb20 msgr2=0x7f017c100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:39.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.584+0000 7f0181800640 1 -- 172.21.15.31:0/311670140 shutdown_connections 2024-04-25T01:13:39.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.584+0000 7f0181800640 1 -- 172.21.15.31:0/311670140 wait complete. 2024-04-25T01:13:39.592 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:13:39.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.818+0000 7fb1dfe00640 1 -- 172.21.15.31:0/2762024170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1d8101790 msgr2=0x7fb1d8101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:39.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.818+0000 7fb1dfe00640 1 --2- 172.21.15.31:0/2762024170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1d8101790 0x7fb1d8101bb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc009900 tx=0x7fb1cc02a410 comp rx=0 tx=0).stop 2024-04-25T01:13:39.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.819+0000 7fb1dfe00640 1 -- 172.21.15.31:0/2762024170 shutdown_connections 2024-04-25T01:13:39.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.819+0000 7fb1dfe00640 1 --2- 172.21.15.31:0/2762024170 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1d8103340 0x7fb1d8109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:39.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.819+0000 7fb1dfe00640 1 --2- 172.21.15.31:0/2762024170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d8102980 0x7fb1d8102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:39.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.819+0000 7fb1dfe00640 1 --2- 172.21.15.31:0/2762024170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1d8101790 0x7fb1d8101bb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:39.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.819+0000 7fb1dfe00640 1 -- 172.21.15.31:0/2762024170 >> 172.21.15.31:0/2762024170 conn(0x7fb1d80fcd20 msgr2=0x7fb1d80ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:39.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.819+0000 7fb1dfe00640 1 -- 172.21.15.31:0/2762024170 shutdown_connections 2024-04-25T01:13:39.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.819+0000 7fb1dfe00640 1 -- 172.21.15.31:0/2762024170 wait complete. 2024-04-25T01:13:39.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.820+0000 7fb1dfe00640 1 Processor -- start 2024-04-25T01:13:39.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.820+0000 7fb1dfe00640 1 -- start start 2024-04-25T01:13:39.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.820+0000 7fb1dfe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1d8101790 0x7fb1d8199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:39.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.820+0000 7fb1dfe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d8102980 0x7fb1d819a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:39.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.820+0000 7fb1dfe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1d8103340 0x7fb1d81a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:39.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.820+0000 7fb1dfe00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d810d430 con 0x7fb1d8102980 2024-04-25T01:13:39.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.820+0000 7fb1dfe00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d810d2b0 con 0x7fb1d8103340 2024-04-25T01:13:39.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.820+0000 7fb1dfe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d810d5b0 con 0x7fb1d8101790 2024-04-25T01:13:39.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.821+0000 7fb1dca00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d8102980 0x7fb1d819a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:39.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.821+0000 7fb1dde00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1d8103340 0x7fb1d81a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:39.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.821+0000 7fb1dd400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1d8101790 0x7fb1d8199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:39.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.821+0000 7fb1dde00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1d8103340 0x7fb1d81a1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:40910/0 (socket says 172.21.15.31:40910) 2024-04-25T01:13:39.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.821+0000 7fb1dd400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1d8101790 0x7fb1d8199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:44302/0 (socket says 172.21.15.31:44302) 2024-04-25T01:13:39.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.821+0000 7fb1dde00640 1 -- 172.21.15.31:0/3471414325 learned_addr learned my addr 172.21.15.31:0/3471414325 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:39.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.821+0000 7fb1dca00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d8102980 0x7fb1d819a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44148/0 (socket says 172.21.15.31:44148) 2024-04-25T01:13:39.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.821+0000 7fb1dca00640 1 -- 172.21.15.31:0/3471414325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1d8101790 msgr2=0x7fb1d8199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:39.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.821+0000 7fb1dca00640 1 --2- 172.21.15.31:0/3471414325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1d8101790 0x7fb1d8199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:39.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.821+0000 7fb1dca00640 1 -- 172.21.15.31:0/3471414325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1d8103340 msgr2=0x7fb1d81a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:39.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.821+0000 7fb1dca00640 1 --2- 172.21.15.31:0/3471414325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1d8103340 0x7fb1d81a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:39.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.821+0000 7fb1dca00640 1 -- 172.21.15.31:0/3471414325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d81a1900 con 0x7fb1d8102980 2024-04-25T01:13:39.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.821+0000 7fb1dde00640 1 --2- 172.21.15.31:0/3471414325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1d8103340 0x7fb1d81a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:13:39.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.822+0000 7fb1dca00640 1 --2- 172.21.15.31:0/3471414325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d8102980 0x7fb1d819a200 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400e9f0 tx=0x7fb1c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:39.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.822+0000 7fb1d6000640 1 -- 172.21.15.31:0/3471414325 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c400ccc0 con 0x7fb1d8102980 2024-04-25T01:13:39.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.822+0000 7fb1d6000640 1 -- 172.21.15.31:0/3471414325 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1c4010910 con 0x7fb1d8102980 2024-04-25T01:13:39.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.822+0000 7fb1dfe00640 1 -- 172.21.15.31:0/3471414325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d81a1bf0 con 0x7fb1d8102980 2024-04-25T01:13:39.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.822+0000 7fb1d6000640 1 -- 172.21.15.31:0/3471414325 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4018960 con 0x7fb1d8102980 2024-04-25T01:13:39.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.822+0000 7fb1dfe00640 1 -- 172.21.15.31:0/3471414325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d81a2160 con 0x7fb1d8102980 2024-04-25T01:13:39.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.823+0000 7fb1dfe00640 1 -- 172.21.15.31:0/3471414325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d81a2330 con 0x7fb1d8102980 2024-04-25T01:13:39.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.826+0000 7fb1d6000640 1 -- 172.21.15.31:0/3471414325 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb1c4010ac0 con 0x7fb1d8102980 2024-04-25T01:13:39.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.828+0000 7fb1d6000640 1 --2- 172.21.15.31:0/3471414325 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb1c00a2ca0 0x7fb1c00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:39.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.828+0000 7fb1d6000640 1 -- 172.21.15.31:0/3471414325 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fb1c40cf430 con 0x7fb1d8102980 2024-04-25T01:13:39.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.829+0000 7fb1dd400640 1 --2- 172.21.15.31:0/3471414325 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb1c00a2ca0 0x7fb1c00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:39.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.829+0000 7fb1dd400640 1 --2- 172.21.15.31:0/3471414325 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb1c00a2ca0 0x7fb1c00a5160 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc02aa00 tx=0x7fb1cc0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:39.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:39.833+0000 7fb1d6000640 1 -- 172.21.15.31:0/3471414325 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb1c409bc90 con 0x7fb1d8102980 2024-04-25T01:13:39.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:39 smithi169 ceph-mon[32759]: from='client.24704 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:13:39.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:39 smithi169 ceph-mon[32759]: pgmap v320: 97 pgs: 97 active+clean; 580 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-04-25T01:13:39.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:39 smithi169 ceph-mon[32759]: from='client.24710 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:13:39.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:39 smithi169 ceph-mon[32759]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-25T01:13:39.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:39 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/311670140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:39.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:39 smithi040 ceph-mon[32518]: from='client.24704 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:13:39.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:39 smithi040 ceph-mon[32518]: pgmap v320: 97 pgs: 97 active+clean; 580 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-04-25T01:13:39.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:39 smithi040 ceph-mon[32518]: from='client.24710 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:13:39.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:39 smithi040 ceph-mon[32518]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-25T01:13:39.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:39 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/311670140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:40.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:39 smithi031 ceph-mon[28053]: from='client.24704 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:13:40.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:39 smithi031 ceph-mon[28053]: pgmap v320: 97 pgs: 97 active+clean; 580 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-04-25T01:13:40.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:39 smithi031 ceph-mon[28053]: from='client.24710 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:13:40.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:39 smithi031 ceph-mon[28053]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-25T01:13:40.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:39 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/311670140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:13:40.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.039+0000 7fb1dfe00640 1 -- 172.21.15.31:0/3471414325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1d81a26c0 con 0x7fb1d8102980 2024-04-25T01:13:40.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.041+0000 7fb1d6000640 1 -- 172.21.15.31:0/3471414325 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22172 (secure 0 0 0) 0x7fb1c409ff90 con 0x7fb1d8102980 2024-04-25T01:13:40.028 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:13:40.029 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"95849b94-029f-11ef-bc93-c7b262605968","created":"2024-04-25T01:04:23.310856+0000","modified":"2024-04-25T01:13:32.489971+0000","last_up_change":"2024-04-25T01:12:29.393907+0000","last_in_change":"2024-04-25T01:12:08.978148+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:18.662391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:13:29.258771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:13:30.480996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa08e4c-fe6b-4116-bdf5-1adf5b465d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6803","nonce":788174091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6805","nonce":788174091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6809","nonce":788174091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6807","nonce":788174091}]},"public_addr":"172.21.15.31:6803/788174091","cluster_addr":"172.21.15.31:6805/788174091","heartbeat_back_addr":"172.21.15.31:6809/788174091","heartbeat_front_addr":"172.21.15.31:6807/788174091","state":["exists","up"]},{"osd":1,"uuid":"85339fe7-0dbc-4485-9334-2b039993c1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6801","nonce":3196394509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6803","nonce":3196394509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6807","nonce":3196394509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6805","nonce":3196394509}]},"public_addr":"172.21.15.40:6801/3196394509","cluster_addr":"172.21.15.40:6803/3196394509","heartbeat_back_addr":"172.21.15.40:6807/3196394509","heartbeat_front_addr":"172.21.15.40:6805/3196394509","state":["exists","up"]},{"osd":2,"uuid":"ab62ecb2-8aae-4987-9e2e-589e0f2fee5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6801","nonce":1955211197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6803","nonce":1955211197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6807","nonce":1955211197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6805","nonce":1955211197}]},"public_addr":"172.21.15.169:6801/1955211197","cluster_addr":"172.21.15.169:6803/1955211197","heartbeat_back_addr":"172.21.15.169:6807/1955211197","heartbeat_front_addr":"172.21.15.169:6805/1955211197","state":["exists","up"]},{"osd":3,"uuid":"8b5a7881-3401-495c-8ca4-c90685919d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6811","nonce":1228688643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6813","nonce":1228688643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6817","nonce":1228688643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6815","nonce":1228688643}]},"public_addr":"172.21.15.31:6811/1228688643","cluster_addr":"172.21.15.31:6813/1228688643","heartbeat_back_addr":"172.21.15.31:6817/1228688643","heartbeat_front_addr":"172.21.15.31:6815/1228688643","state":["exists","up"]},{"osd":4,"uuid":"56deb61d-a334-4cf2-8ac7-fbb6e6947092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6809","nonce":1913779091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6811","nonce":1913779091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6815","nonce":1913779091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6813","nonce":1913779091}]},"public_addr":"172.21.15.40:6809/1913779091","cluster_addr":"172.21.15.40:6811/1913779091","heartbeat_back_addr":"172.21.15.40:6815/1913779091","heartbeat_front_addr":"172.21.15.40:6813/1913779091","state":["exists","up"]},{"osd":5,"uuid":"8da55693-f9c7-49a7-99e0-165078f8001b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6809","nonce":2549650781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6811","nonce":2549650781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6815","nonce":2549650781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6813","nonce":2549650781}]},"public_addr":"172.21.15.169:6809/2549650781","cluster_addr":"172.21.15.169:6811/2549650781","heartbeat_back_addr":"172.21.15.169:6815/2549650781","heartbeat_front_addr":"172.21.15.169:6813/2549650781","state":["exists","up"]},{"osd":6,"uuid":"fc8350d2-1ad6-43cb-89f4-d336afc99ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6819","nonce":611507624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6821","nonce":611507624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6825","nonce":611507624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6823","nonce":611507624}]},"public_addr":"172.21.15.31:6819/611507624","cluster_addr":"172.21.15.31:6821/611507624","heartbeat_back_addr":"172.21.15.31:6825/611507624","heartbeat_front_addr":"172.21.15.31:6823/611507624","state":["exists","up"]},{"osd":7,"uuid":"3121b58b-8c94-40b5-9d48-f56552d7c39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6817","nonce":3776085849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6819","nonce":3776085849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6823","nonce":3776085849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6821","nonce":3776085849}]},"public_addr":"172.21.15.40:6817/3776085849","cluster_addr":"172.21.15.40:6819/3776085849","heartbeat_back_addr":"172.21.15.40:6823/3776085849","heartbeat_front_addr":"172.21.15.40:6821/3776085849","state":["exists","up"]},{"osd":8,"uuid":"f60407df-91f5-477a-9702-ddaef24faf82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6817","nonce":2736546122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6819","nonce":2736546122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6823","nonce":2736546122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6821","nonce":2736546122}]},"public_addr":"172.21.15.169:6817/2736546122","cluster_addr":"172.21.15.169:6819/2736546122","heartbeat_back_addr":"172.21.15.169:6823/2736546122","heartbeat_front_addr":"172.21.15.169:6821/2736546122","state":["exists","up"]},{"osd":9,"uuid":"7506a407-1960-49bc-9e38-7f2a8bf996ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6827","nonce":1187207880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6829","nonce":1187207880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6833","nonce":1187207880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6831","nonce":1187207880}]},"public_addr":"172.21.15.31:6827/1187207880","cluster_addr":"172.21.15.31:6829/1187207880","heartbeat_back_addr":"172.21.15.31:6833/1187207880","heartbeat_front_addr":"172.21.15.31:6831/1187207880","state":["exists","up"]},{"osd":10,"uuid":"831c48e2-b0d0-451f-959b-228e927542cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6825","nonce":3050148001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6827","nonce":3050148001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6831","nonce":3050148001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6829","nonce":3050148001}]},"public_addr":"172.21.15.40:6825/3050148001","cluster_addr":"172.21.15.40:6827/3050148001","heartbeat_back_addr":"172.21.15.40:6831/3050148001","heartbeat_front_addr":"172.21.15.40:6829/3050148001","state":["exists","up"]},{"osd":11,"uuid":"144fe0aa-f757-4b3c-90ad-99775e17d57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6825","nonce":262401944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6827","nonce":262401944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6831","nonce":262401944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6829","nonce":262401944}]},"public_addr":"172.21.15.169:6825/262401944","cluster_addr":"172.21.15.169:6827/262401944","heartbeat_back_addr":"172.21.15.169:6831/262401944","heartbeat_front_addr":"172.21.15.169:6829/262401944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:25.640995+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:51.036909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.635633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:48.467912+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:14.454971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:37.609315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:12.441807+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:38.239215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:02.230758+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:35.762553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:02.051405+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:26.569593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6801/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4004014066":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/1930101646":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/2638309002":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2238638505":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2902841144":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4190696323":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/480558521":"2024-04-26T01:04:57.149272+0000","172.21.15.31:6801/2638309002":"2024-04-26T01:04:57.149272+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:13:40.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.044+0000 7fb1dfe00640 1 -- 172.21.15.31:0/3471414325 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb1c00a2ca0 msgr2=0x7fb1c00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:40.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.044+0000 7fb1dfe00640 1 --2- 172.21.15.31:0/3471414325 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb1c00a2ca0 0x7fb1c00a5160 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc02aa00 tx=0x7fb1cc0315b0 comp rx=0 tx=0).stop 2024-04-25T01:13:40.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.044+0000 7fb1dfe00640 1 -- 172.21.15.31:0/3471414325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d8102980 msgr2=0x7fb1d819a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:40.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.044+0000 7fb1dfe00640 1 --2- 172.21.15.31:0/3471414325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d8102980 0x7fb1d819a200 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400e9f0 tx=0x7fb1c400eeb0 comp rx=0 tx=0).stop 2024-04-25T01:13:40.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.045+0000 7fb1dfe00640 1 -- 172.21.15.31:0/3471414325 shutdown_connections 2024-04-25T01:13:40.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.045+0000 7fb1dfe00640 1 --2- 172.21.15.31:0/3471414325 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb1c00a2ca0 0x7fb1c00a5160 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:40.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.045+0000 7fb1dfe00640 1 --2- 172.21.15.31:0/3471414325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1d8103340 0x7fb1d81a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:40.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.045+0000 7fb1dfe00640 1 --2- 172.21.15.31:0/3471414325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d8102980 0x7fb1d819a200 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:40.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.045+0000 7fb1dfe00640 1 --2- 172.21.15.31:0/3471414325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1d8101790 0x7fb1d8199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:40.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.045+0000 7fb1dfe00640 1 -- 172.21.15.31:0/3471414325 >> 172.21.15.31:0/3471414325 conn(0x7fb1d80fcd20 msgr2=0x7fb1d8073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:40.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.045+0000 7fb1dfe00640 1 -- 172.21.15.31:0/3471414325 shutdown_connections 2024-04-25T01:13:40.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.045+0000 7fb1dfe00640 1 -- 172.21.15.31:0/3471414325 wait complete. 2024-04-25T01:13:40.053 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T01:13:40.054 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:13:40.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.280+0000 7fa80ae00640 1 -- 172.21.15.31:0/488899584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa804077340 msgr2=0x7fa8040777a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:40.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.280+0000 7fa80ae00640 1 --2- 172.21.15.31:0/488899584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa804077340 0x7fa8040777a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0009860 tx=0x7fa7f002a340 comp rx=0 tx=0).stop 2024-04-25T01:13:40.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.280+0000 7fa80ae00640 1 -- 172.21.15.31:0/488899584 shutdown_connections 2024-04-25T01:13:40.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.280+0000 7fa80ae00640 1 --2- 172.21.15.31:0/488899584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa804077ce0 0x7fa804078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:40.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.280+0000 7fa80ae00640 1 --2- 172.21.15.31:0/488899584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa804077340 0x7fa8040777a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:40.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.280+0000 7fa80ae00640 1 --2- 172.21.15.31:0/488899584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa804076100 0x7fa804076520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:40.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.280+0000 7fa80ae00640 1 -- 172.21.15.31:0/488899584 >> 172.21.15.31:0/488899584 conn(0x7fa8040fed50 msgr2=0x7fa804101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:40.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.280+0000 7fa80ae00640 1 -- 172.21.15.31:0/488899584 shutdown_connections 2024-04-25T01:13:40.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.280+0000 7fa80ae00640 1 -- 172.21.15.31:0/488899584 wait complete. 2024-04-25T01:13:40.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.281+0000 7fa80ae00640 1 Processor -- start 2024-04-25T01:13:40.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.282+0000 7fa80ae00640 1 -- start start 2024-04-25T01:13:40.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.282+0000 7fa80ae00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa804076100 0x7fa80419dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:40.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.282+0000 7fa80ae00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa804077340 0x7fa80419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:40.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.282+0000 7fa80ae00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa804077ce0 0x7fa8041a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:40.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.282+0000 7fa80ae00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa804113870 con 0x7fa804077ce0 2024-04-25T01:13:40.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.282+0000 7fa80ae00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa8041136f0 con 0x7fa804077340 2024-04-25T01:13:40.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.282+0000 7fa80ae00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa8041139f0 con 0x7fa804076100 2024-04-25T01:13:40.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.282+0000 7fa80a400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa804077ce0 0x7fa8041a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:40.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.282+0000 7fa809a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa804076100 0x7fa80419dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:40.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.282+0000 7fa80a400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa804077ce0 0x7fa8041a5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44164/0 (socket says 172.21.15.31:44164) 2024-04-25T01:13:40.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.282+0000 7fa80a400640 1 -- 172.21.15.31:0/1447627950 learned_addr learned my addr 172.21.15.31:0/1447627950 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:40.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.282+0000 7fa809000640 1 --2- 172.21.15.31:0/1447627950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa804077340 0x7fa80419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:40.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.283+0000 7fa809a00640 1 -- 172.21.15.31:0/1447627950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa804077340 msgr2=0x7fa80419e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:40.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.283+0000 7fa809a00640 1 --2- 172.21.15.31:0/1447627950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa804077340 0x7fa80419e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:40.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.283+0000 7fa809a00640 1 -- 172.21.15.31:0/1447627950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa804077ce0 msgr2=0x7fa8041a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:40.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.283+0000 7fa809a00640 1 --2- 172.21.15.31:0/1447627950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa804077ce0 0x7fa8041a5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:40.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.283+0000 7fa809a00640 1 -- 172.21.15.31:0/1447627950 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8041a5ac0 con 0x7fa804076100 2024-04-25T01:13:40.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.283+0000 7fa809000640 1 --2- 172.21.15.31:0/1447627950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa804077340 0x7fa80419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:13:40.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.283+0000 7fa809a00640 1 --2- 172.21.15.31:0/1447627950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa804076100 0x7fa80419dea0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800ef60 tx=0x7fa7f800ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:40.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.283+0000 7fa802000640 1 -- 172.21.15.31:0/1447627950 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f801b070 con 0x7fa804076100 2024-04-25T01:13:40.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.284+0000 7fa80ae00640 1 -- 172.21.15.31:0/1447627950 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8041a5db0 con 0x7fa804076100 2024-04-25T01:13:40.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.284+0000 7fa802000640 1 -- 172.21.15.31:0/1447627950 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7f800ba50 con 0x7fa804076100 2024-04-25T01:13:40.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.284+0000 7fa802000640 1 -- 172.21.15.31:0/1447627950 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8016bf0 con 0x7fa804076100 2024-04-25T01:13:40.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.284+0000 7fa80ae00640 1 -- 172.21.15.31:0/1447627950 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8041a6270 con 0x7fa804076100 2024-04-25T01:13:40.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.285+0000 7fa80ae00640 1 -- 172.21.15.31:0/1447627950 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8041102f0 con 0x7fa804076100 2024-04-25T01:13:40.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.287+0000 7fa802000640 1 -- 172.21.15.31:0/1447627950 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa7f801e070 con 0x7fa804076100 2024-04-25T01:13:40.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.288+0000 7fa802000640 1 --2- 172.21.15.31:0/1447627950 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa7ec0a2ca0 0x7fa7ec0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:40.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.288+0000 7fa802000640 1 -- 172.21.15.31:0/1447627950 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fa7f80ceff0 con 0x7fa804076100 2024-04-25T01:13:40.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.289+0000 7fa809000640 1 --2- 172.21.15.31:0/1447627950 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa7ec0a2ca0 0x7fa7ec0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:40.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.289+0000 7fa809000640 1 --2- 172.21.15.31:0/1447627950 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa7ec0a2ca0 0x7fa7ec0a5160 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0009830 tx=0x7fa7f0030550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:40.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.293+0000 7fa802000640 1 -- 172.21.15.31:0/1447627950 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa7f809b8b0 con 0x7fa804076100 2024-04-25T01:13:40.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.492+0000 7fa80ae00640 1 -- 172.21.15.31:0/1447627950 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa804061b80 con 0x7fa7ec0a2ca0 2024-04-25T01:13:40.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.507+0000 7fa802000640 1 -- 172.21.15.31:0/1447627950 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258773 (secure 0 0 0) 0x7fa804061b80 con 0x7fa7ec0a2ca0 2024-04-25T01:13:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:13:40.500 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-04-25T01:13:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.513+0000 7fa80ae00640 1 -- 172.21.15.31:0/1447627950 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa7ec0a2ca0 msgr2=0x7fa7ec0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.513+0000 7fa80ae00640 1 --2- 172.21.15.31:0/1447627950 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa7ec0a2ca0 0x7fa7ec0a5160 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0009830 tx=0x7fa7f0030550 comp rx=0 tx=0).stop 2024-04-25T01:13:40.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.513+0000 7fa80ae00640 1 -- 172.21.15.31:0/1447627950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa804076100 msgr2=0x7fa80419dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:40.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.513+0000 7fa80ae00640 1 --2- 172.21.15.31:0/1447627950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa804076100 0x7fa80419dea0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800ef60 tx=0x7fa7f800ef90 comp rx=0 tx=0).stop 2024-04-25T01:13:40.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.513+0000 7fa80ae00640 1 -- 172.21.15.31:0/1447627950 shutdown_connections 2024-04-25T01:13:40.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.513+0000 7fa80ae00640 1 --2- 172.21.15.31:0/1447627950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa804077ce0 0x7fa8041a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:40.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.513+0000 7fa80ae00640 1 --2- 172.21.15.31:0/1447627950 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa7ec0a2ca0 0x7fa7ec0a5160 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:40.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.513+0000 7fa80ae00640 1 --2- 172.21.15.31:0/1447627950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa804077340 0x7fa80419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:40.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.513+0000 7fa80ae00640 1 --2- 172.21.15.31:0/1447627950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa804076100 0x7fa80419dea0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:40.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.513+0000 7fa80ae00640 1 -- 172.21.15.31:0/1447627950 >> 172.21.15.31:0/1447627950 conn(0x7fa8040fed50 msgr2=0x7fa804100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:40.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.513+0000 7fa80ae00640 1 -- 172.21.15.31:0/1447627950 shutdown_connections 2024-04-25T01:13:40.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.513+0000 7fa80ae00640 1 -- 172.21.15.31:0/1447627950 wait complete. 2024-04-25T01:13:40.521 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-04-25T01:13:40.072760+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632288,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":165,"num_read_kb":157,"num_write":317,"num_write_kb":3016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":256,"ondisk_log_size":256,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374556,"kb_used_data":12908,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567268,"statfs":{"total":1151940427776,"available":1151556882432,"internally_reserved":0,"allocated":13217792,"data_stored":7167707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":41920,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":39,"num_read_kb":48,"num_write":136,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.581218"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.642563+0000","last_change":"2024-04-25T01:13:31.536782+0000","last_active":"2024-04-25T01:13:32.642563+0000","last_peered":"2024-04-25T01:13:32.642563+0000","last_clean":"2024-04-25T01:13:32.642563+0000","last_became_active":"2024-04-25T01:13:31.536426+0000","last_became_peered":"2024-04-25T01:13:31.536426+0000","last_unstale":"2024-04-25T01:13:32.642563+0000","last_undegraded":"2024-04-25T01:13:32.642563+0000","last_fullsized":"2024-04-25T01:13:32.642563+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:07.489179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.786932+0000","last_change":"2024-04-25T01:13:31.543521+0000","last_active":"2024-04-25T01:13:33.786932+0000","last_peered":"2024-04-25T01:13:33.786932+0000","last_clean":"2024-04-25T01:13:33.786932+0000","last_became_active":"2024-04-25T01:13:31.542767+0000","last_became_peered":"2024-04-25T01:13:31.542767+0000","last_unstale":"2024-04-25T01:13:33.786932+0000","last_undegraded":"2024-04-25T01:13:33.786932+0000","last_fullsized":"2024-04-25T01:13:33.786932+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:33.287564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.787132+0000","last_change":"2024-04-25T01:13:31.537297+0000","last_active":"2024-04-25T01:13:33.787132+0000","last_peered":"2024-04-25T01:13:33.787132+0000","last_clean":"2024-04-25T01:13:33.787132+0000","last_became_active":"2024-04-25T01:13:31.536930+0000","last_became_peered":"2024-04-25T01:13:31.536930+0000","last_unstale":"2024-04-25T01:13:33.787132+0000","last_undegraded":"2024-04-25T01:13:33.787132+0000","last_fullsized":"2024-04-25T01:13:33.787132+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:58:35.465287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784311+0000","last_change":"2024-04-25T01:13:31.516514+0000","last_active":"2024-04-25T01:13:33.784311+0000","last_peered":"2024-04-25T01:13:33.784311+0000","last_clean":"2024-04-25T01:13:33.784311+0000","last_became_active":"2024-04-25T01:13:31.516192+0000","last_became_peered":"2024-04-25T01:13:31.516192+0000","last_unstale":"2024-04-25T01:13:33.784311+0000","last_undegraded":"2024-04-25T01:13:33.784311+0000","last_fullsized":"2024-04-25T01:13:33.784311+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:27.416626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.495010+0000","last_change":"2024-04-25T01:13:31.538662+0000","last_active":"2024-04-25T01:13:32.495010+0000","last_peered":"2024-04-25T01:13:32.495010+0000","last_clean":"2024-04-25T01:13:32.495010+0000","last_became_active":"2024-04-25T01:13:31.538362+0000","last_became_peered":"2024-04-25T01:13:31.538362+0000","last_unstale":"2024-04-25T01:13:32.495010+0000","last_undegraded":"2024-04-25T01:13:32.495010+0000","last_fullsized":"2024-04-25T01:13:32.495010+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:10.744891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.667900+0000","last_change":"2024-04-25T01:13:31.533051+0000","last_active":"2024-04-25T01:13:32.667900+0000","last_peered":"2024-04-25T01:13:32.667900+0000","last_clean":"2024-04-25T01:13:32.667900+0000","last_became_active":"2024-04-25T01:13:31.532652+0000","last_became_peered":"2024-04-25T01:13:31.532652+0000","last_unstale":"2024-04-25T01:13:32.667900+0000","last_undegraded":"2024-04-25T01:13:32.667900+0000","last_fullsized":"2024-04-25T01:13:32.667900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:41.998434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784902+0000","last_change":"2024-04-25T01:13:31.530362+0000","last_active":"2024-04-25T01:13:33.784902+0000","last_peered":"2024-04-25T01:13:33.784902+0000","last_clean":"2024-04-25T01:13:33.784902+0000","last_became_active":"2024-04-25T01:13:31.529869+0000","last_became_peered":"2024-04-25T01:13:31.529869+0000","last_unstale":"2024-04-25T01:13:33.784902+0000","last_undegraded":"2024-04-25T01:13:33.784902+0000","last_fullsized":"2024-04-25T01:13:33.784902+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:54.499230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.496815+0000","last_change":"2024-04-25T01:13:31.522133+0000","last_active":"2024-04-25T01:13:32.496815+0000","last_peered":"2024-04-25T01:13:32.496815+0000","last_clean":"2024-04-25T01:13:32.496815+0000","last_became_active":"2024-04-25T01:13:31.521552+0000","last_became_peered":"2024-04-25T01:13:31.521552+0000","last_unstale":"2024-04-25T01:13:32.496815+0000","last_undegraded":"2024-04-25T01:13:32.496815+0000","last_fullsized":"2024-04-25T01:13:32.496815+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:06.651854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.495576+0000","last_change":"2024-04-25T01:13:31.518589+0000","last_active":"2024-04-25T01:13:32.495576+0000","last_peered":"2024-04-25T01:13:32.495576+0000","last_clean":"2024-04-25T01:13:32.495576+0000","last_became_active":"2024-04-25T01:13:31.518183+0000","last_became_peered":"2024-04-25T01:13:31.518183+0000","last_unstale":"2024-04-25T01:13:32.495576+0000","last_undegraded":"2024-04-25T01:13:32.495576+0000","last_fullsized":"2024-04-25T01:13:32.495576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:06:17.534019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.668063+0000","last_change":"2024-04-25T01:13:31.522281+0000","last_active":"2024-04-25T01:13:32.668063+0000","last_peered":"2024-04-25T01:13:32.668063+0000","last_clean":"2024-04-25T01:13:32.668063+0000","last_became_active":"2024-04-25T01:13:31.521906+0000","last_became_peered":"2024-04-25T01:13:31.521906+0000","last_unstale":"2024-04-25T01:13:32.668063+0000","last_undegraded":"2024-04-25T01:13:32.668063+0000","last_fullsized":"2024-04-25T01:13:32.668063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:58.761284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.785685+0000","last_change":"2024-04-25T01:13:31.514766+0000","last_active":"2024-04-25T01:13:33.785685+0000","last_peered":"2024-04-25T01:13:33.785685+0000","last_clean":"2024-04-25T01:13:33.785685+0000","last_became_active":"2024-04-25T01:13:31.514391+0000","last_became_peered":"2024-04-25T01:13:31.514391+0000","last_unstale":"2024-04-25T01:13:33.785685+0000","last_undegraded":"2024-04-25T01:13:33.785685+0000","last_fullsized":"2024-04-25T01:13:33.785685+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:07.398446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784305+0000","last_change":"2024-04-25T01:13:31.518976+0000","last_active":"2024-04-25T01:13:33.784305+0000","last_peered":"2024-04-25T01:13:33.784305+0000","last_clean":"2024-04-25T01:13:33.784305+0000","last_became_active":"2024-04-25T01:13:31.518566+0000","last_became_peered":"2024-04-25T01:13:31.518566+0000","last_unstale":"2024-04-25T01:13:33.784305+0000","last_undegraded":"2024-04-25T01:13:33.784305+0000","last_fullsized":"2024-04-25T01:13:33.784305+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:59.477996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.787769+0000","last_change":"2024-04-25T01:13:31.532880+0000","last_active":"2024-04-25T01:13:33.787769+0000","last_peered":"2024-04-25T01:13:33.787769+0000","last_clean":"2024-04-25T01:13:33.787769+0000","last_became_active":"2024-04-25T01:13:31.531470+0000","last_became_peered":"2024-04-25T01:13:31.531470+0000","last_unstale":"2024-04-25T01:13:33.787769+0000","last_undegraded":"2024-04-25T01:13:33.787769+0000","last_fullsized":"2024-04-25T01:13:33.787769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:55.431431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.785850+0000","last_change":"2024-04-25T01:13:31.532472+0000","last_active":"2024-04-25T01:13:33.785850+0000","last_peered":"2024-04-25T01:13:33.785850+0000","last_clean":"2024-04-25T01:13:33.785850+0000","last_became_active":"2024-04-25T01:13:31.530850+0000","last_became_peered":"2024-04-25T01:13:31.530850+0000","last_unstale":"2024-04-25T01:13:33.785850+0000","last_undegraded":"2024-04-25T01:13:33.785850+0000","last_fullsized":"2024-04-25T01:13:33.785850+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:50:26.890858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.785818+0000","last_change":"2024-04-25T01:13:31.539930+0000","last_active":"2024-04-25T01:13:33.785818+0000","last_peered":"2024-04-25T01:13:33.785818+0000","last_clean":"2024-04-25T01:13:33.785818+0000","last_became_active":"2024-04-25T01:13:31.539523+0000","last_became_peered":"2024-04-25T01:13:31.539523+0000","last_unstale":"2024-04-25T01:13:33.785818+0000","last_undegraded":"2024-04-25T01:13:33.785818+0000","last_fullsized":"2024-04-25T01:13:33.785818+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:07.983893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.494913+0000","last_change":"2024-04-25T01:13:31.536437+0000","last_active":"2024-04-25T01:13:32.494913+0000","last_peered":"2024-04-25T01:13:32.494913+0000","last_clean":"2024-04-25T01:13:32.494913+0000","last_became_active":"2024-04-25T01:13:31.536066+0000","last_became_peered":"2024-04-25T01:13:31.536066+0000","last_unstale":"2024-04-25T01:13:32.494913+0000","last_undegraded":"2024-04-25T01:13:32.494913+0000","last_fullsized":"2024-04-25T01:13:32.494913+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:50.986971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.642681+0000","last_change":"2024-04-25T01:13:31.536706+0000","last_active":"2024-04-25T01:13:32.642681+0000","last_peered":"2024-04-25T01:13:32.642681+0000","last_clean":"2024-04-25T01:13:32.642681+0000","last_became_active":"2024-04-25T01:13:31.536318+0000","last_became_peered":"2024-04-25T01:13:31.536318+0000","last_unstale":"2024-04-25T01:13:32.642681+0000","last_undegraded":"2024-04-25T01:13:32.642681+0000","last_fullsized":"2024-04-25T01:13:32.642681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:35.526515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.560216+0000","last_change":"2024-04-25T01:13:31.524462+0000","last_active":"2024-04-25T01:13:32.560216+0000","last_peered":"2024-04-25T01:13:32.560216+0000","last_clean":"2024-04-25T01:13:32.560216+0000","last_became_active":"2024-04-25T01:13:31.524018+0000","last_became_peered":"2024-04-25T01:13:31.524018+0000","last_unstale":"2024-04-25T01:13:32.560216+0000","last_undegraded":"2024-04-25T01:13:32.560216+0000","last_fullsized":"2024-04-25T01:13:32.560216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:43.767053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784645+0000","last_change":"2024-04-25T01:13:31.531553+0000","last_active":"2024-04-25T01:13:33.784645+0000","last_peered":"2024-04-25T01:13:33.784645+0000","last_clean":"2024-04-25T01:13:33.784645+0000","last_became_active":"2024-04-25T01:13:31.531237+0000","last_became_peered":"2024-04-25T01:13:31.531237+0000","last_unstale":"2024-04-25T01:13:33.784645+0000","last_undegraded":"2024-04-25T01:13:33.784645+0000","last_fullsized":"2024-04-25T01:13:33.784645+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:41.839370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:35.560238+0000","last_change":"2024-04-25T01:13:30.516847+0000","last_active":"2024-04-25T01:13:35.560238+0000","last_peered":"2024-04-25T01:13:35.560238+0000","last_clean":"2024-04-25T01:13:35.560238+0000","last_became_active":"2024-04-25T01:13:30.516500+0000","last_became_peered":"2024-04-25T01:13:30.516500+0000","last_unstale":"2024-04-25T01:13:35.560238+0000","last_undegraded":"2024-04-25T01:13:35.560238+0000","last_fullsized":"2024-04-25T01:13:35.560238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:37.703075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.497385+0000","last_change":"2024-04-25T01:13:31.518822+0000","last_active":"2024-04-25T01:13:32.497385+0000","last_peered":"2024-04-25T01:13:32.497385+0000","last_clean":"2024-04-25T01:13:32.497385+0000","last_became_active":"2024-04-25T01:13:31.516726+0000","last_became_peered":"2024-04-25T01:13:31.516726+0000","last_unstale":"2024-04-25T01:13:32.497385+0000","last_undegraded":"2024-04-25T01:13:32.497385+0000","last_fullsized":"2024-04-25T01:13:32.497385+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:02.360104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.497986+0000","last_change":"2024-04-25T01:13:30.531875+0000","last_active":"2024-04-25T01:13:32.497986+0000","last_peered":"2024-04-25T01:13:32.497986+0000","last_clean":"2024-04-25T01:13:32.497986+0000","last_became_active":"2024-04-25T01:13:30.531532+0000","last_became_peered":"2024-04-25T01:13:30.531532+0000","last_unstale":"2024-04-25T01:13:32.497986+0000","last_undegraded":"2024-04-25T01:13:32.497986+0000","last_fullsized":"2024-04-25T01:13:32.497986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:30.820946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.788544+0000","last_change":"2024-04-25T01:13:31.534211+0000","last_active":"2024-04-25T01:13:33.788544+0000","last_peered":"2024-04-25T01:13:33.788544+0000","last_clean":"2024-04-25T01:13:33.788544+0000","last_became_active":"2024-04-25T01:13:31.533828+0000","last_became_peered":"2024-04-25T01:13:31.533828+0000","last_unstale":"2024-04-25T01:13:33.788544+0000","last_undegraded":"2024-04-25T01:13:33.788544+0000","last_fullsized":"2024-04-25T01:13:33.788544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:39.488524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:35.547492+0000","last_change":"2024-04-25T01:13:30.517922+0000","last_active":"2024-04-25T01:13:35.547492+0000","last_peered":"2024-04-25T01:13:35.547492+0000","last_clean":"2024-04-25T01:13:35.547492+0000","last_became_active":"2024-04-25T01:13:30.517571+0000","last_became_peered":"2024-04-25T01:13:30.517571+0000","last_unstale":"2024-04-25T01:13:35.547492+0000","last_undegraded":"2024-04-25T01:13:35.547492+0000","last_fullsized":"2024-04-25T01:13:35.547492+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:16:34.901210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.495220+0000","last_change":"2024-04-25T01:13:31.530460+0000","last_active":"2024-04-25T01:13:32.495220+0000","last_peered":"2024-04-25T01:13:32.495220+0000","last_clean":"2024-04-25T01:13:32.495220+0000","last_became_active":"2024-04-25T01:13:31.530084+0000","last_became_peered":"2024-04-25T01:13:31.530084+0000","last_unstale":"2024-04-25T01:13:32.495220+0000","last_undegraded":"2024-04-25T01:13:32.495220+0000","last_fullsized":"2024-04-25T01:13:32.495220+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:36.810301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.495042+0000","last_change":"2024-04-25T01:13:30.514320+0000","last_active":"2024-04-25T01:13:32.495042+0000","last_peered":"2024-04-25T01:13:32.495042+0000","last_clean":"2024-04-25T01:13:32.495042+0000","last_became_active":"2024-04-25T01:13:30.513752+0000","last_became_peered":"2024-04-25T01:13:30.513752+0000","last_unstale":"2024-04-25T01:13:32.495042+0000","last_undegraded":"2024-04-25T01:13:32.495042+0000","last_fullsized":"2024-04-25T01:13:32.495042+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:19.814454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784661+0000","last_change":"2024-04-25T01:13:31.514516+0000","last_active":"2024-04-25T01:13:33.784661+0000","last_peered":"2024-04-25T01:13:33.784661+0000","last_clean":"2024-04-25T01:13:33.784661+0000","last_became_active":"2024-04-25T01:13:31.514208+0000","last_became_peered":"2024-04-25T01:13:31.514208+0000","last_unstale":"2024-04-25T01:13:33.784661+0000","last_undegraded":"2024-04-25T01:13:33.784661+0000","last_fullsized":"2024-04-25T01:13:33.784661+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:10:11.204770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:37.538570+0000","last_change":"2024-04-25T01:13:30.519701+0000","last_active":"2024-04-25T01:13:37.538570+0000","last_peered":"2024-04-25T01:13:37.538570+0000","last_clean":"2024-04-25T01:13:37.538570+0000","last_became_active":"2024-04-25T01:13:30.519315+0000","last_became_peered":"2024-04-25T01:13:30.519315+0000","last_unstale":"2024-04-25T01:13:37.538570+0000","last_undegraded":"2024-04-25T01:13:37.538570+0000","last_fullsized":"2024-04-25T01:13:37.538570+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:34.613473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.495007+0000","last_change":"2024-04-25T01:13:31.528376+0000","last_active":"2024-04-25T01:13:32.495007+0000","last_peered":"2024-04-25T01:13:32.495007+0000","last_clean":"2024-04-25T01:13:32.495007+0000","last_became_active":"2024-04-25T01:13:31.527622+0000","last_became_peered":"2024-04-25T01:13:31.527622+0000","last_unstale":"2024-04-25T01:13:32.495007+0000","last_undegraded":"2024-04-25T01:13:32.495007+0000","last_fullsized":"2024-04-25T01:13:32.495007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:17.901898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:35.558911+0000","last_change":"2024-04-25T01:13:30.514798+0000","last_active":"2024-04-25T01:13:35.558911+0000","last_peered":"2024-04-25T01:13:35.558911+0000","last_clean":"2024-04-25T01:13:35.558911+0000","last_became_active":"2024-04-25T01:13:30.514422+0000","last_became_peered":"2024-04-25T01:13:30.514422+0000","last_unstale":"2024-04-25T01:13:35.558911+0000","last_undegraded":"2024-04-25T01:13:35.558911+0000","last_fullsized":"2024-04-25T01:13:35.558911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:27.584578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784646+0000","last_change":"2024-04-25T01:13:31.510479+0000","last_active":"2024-04-25T01:13:33.784646+0000","last_peered":"2024-04-25T01:13:33.784646+0000","last_clean":"2024-04-25T01:13:33.784646+0000","last_became_active":"2024-04-25T01:13:31.509166+0000","last_became_peered":"2024-04-25T01:13:31.509166+0000","last_unstale":"2024-04-25T01:13:33.784646+0000","last_undegraded":"2024-04-25T01:13:33.784646+0000","last_fullsized":"2024-04-25T01:13:33.784646+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:42.604266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:35.562022+0000","last_change":"2024-04-25T01:13:30.520435+0000","last_active":"2024-04-25T01:13:35.562022+0000","last_peered":"2024-04-25T01:13:35.562022+0000","last_clean":"2024-04-25T01:13:35.562022+0000","last_became_active":"2024-04-25T01:13:30.520042+0000","last_became_peered":"2024-04-25T01:13:30.520042+0000","last_unstale":"2024-04-25T01:13:35.562022+0000","last_undegraded":"2024-04-25T01:13:35.562022+0000","last_fullsized":"2024-04-25T01:13:35.562022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:37.818149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.497331+0000","last_change":"2024-04-25T01:13:31.524799+0000","last_active":"2024-04-25T01:13:32.497331+0000","last_peered":"2024-04-25T01:13:32.497331+0000","last_clean":"2024-04-25T01:13:32.497331+0000","last_became_active":"2024-04-25T01:13:31.524454+0000","last_became_peered":"2024-04-25T01:13:31.524454+0000","last_unstale":"2024-04-25T01:13:32.497331+0000","last_undegraded":"2024-04-25T01:13:32.497331+0000","last_fullsized":"2024-04-25T01:13:32.497331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:15.930922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:34.351397+0000","last_change":"2024-04-25T01:13:30.514104+0000","last_active":"2024-04-25T01:13:34.351397+0000","last_peered":"2024-04-25T01:13:34.351397+0000","last_clean":"2024-04-25T01:13:34.351397+0000","last_became_active":"2024-04-25T01:13:30.513742+0000","last_became_peered":"2024-04-25T01:13:30.513742+0000","last_unstale":"2024-04-25T01:13:34.351397+0000","last_undegraded":"2024-04-25T01:13:34.351397+0000","last_fullsized":"2024-04-25T01:13:34.351397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:42.189997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.559329+0000","last_change":"2024-04-25T01:13:31.528223+0000","last_active":"2024-04-25T01:13:32.559329+0000","last_peered":"2024-04-25T01:13:32.559329+0000","last_clean":"2024-04-25T01:13:32.559329+0000","last_became_active":"2024-04-25T01:13:31.526906+0000","last_became_peered":"2024-04-25T01:13:31.526906+0000","last_unstale":"2024-04-25T01:13:32.559329+0000","last_undegraded":"2024-04-25T01:13:32.559329+0000","last_fullsized":"2024-04-25T01:13:32.559329+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:12:16.276974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:37.537727+0000","last_change":"2024-04-25T01:13:30.519174+0000","last_active":"2024-04-25T01:13:37.537727+0000","last_peered":"2024-04-25T01:13:37.537727+0000","last_clean":"2024-04-25T01:13:37.537727+0000","last_became_active":"2024-04-25T01:13:30.518792+0000","last_became_peered":"2024-04-25T01:13:30.518792+0000","last_unstale":"2024-04-25T01:13:37.537727+0000","last_undegraded":"2024-04-25T01:13:37.537727+0000","last_fullsized":"2024-04-25T01:13:37.537727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:27.746435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.495454+0000","last_change":"2024-04-25T01:13:31.521482+0000","last_active":"2024-04-25T01:13:32.495454+0000","last_peered":"2024-04-25T01:13:32.495454+0000","last_clean":"2024-04-25T01:13:32.495454+0000","last_became_active":"2024-04-25T01:13:31.521114+0000","last_became_peered":"2024-04-25T01:13:31.521114+0000","last_unstale":"2024-04-25T01:13:32.495454+0000","last_undegraded":"2024-04-25T01:13:32.495454+0000","last_fullsized":"2024-04-25T01:13:32.495454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:48.748691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:34.591831+0000","last_change":"2024-04-25T01:13:30.518427+0000","last_active":"2024-04-25T01:13:34.591831+0000","last_peered":"2024-04-25T01:13:34.591831+0000","last_clean":"2024-04-25T01:13:34.591831+0000","last_became_active":"2024-04-25T01:13:30.518005+0000","last_became_peered":"2024-04-25T01:13:30.518005+0000","last_unstale":"2024-04-25T01:13:34.591831+0000","last_undegraded":"2024-04-25T01:13:34.591831+0000","last_fullsized":"2024-04-25T01:13:34.591831+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:29:21.087411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.788126+0000","last_change":"2024-04-25T01:13:31.532598+0000","last_active":"2024-04-25T01:13:33.788126+0000","last_peered":"2024-04-25T01:13:33.788126+0000","last_clean":"2024-04-25T01:13:33.788126+0000","last_became_active":"2024-04-25T01:13:31.530468+0000","last_became_peered":"2024-04-25T01:13:31.530468+0000","last_unstale":"2024-04-25T01:13:33.788126+0000","last_undegraded":"2024-04-25T01:13:33.788126+0000","last_fullsized":"2024-04-25T01:13:33.788126+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:28.254924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:34.593481+0000","last_change":"2024-04-25T01:13:30.520507+0000","last_active":"2024-04-25T01:13:34.593481+0000","last_peered":"2024-04-25T01:13:34.593481+0000","last_clean":"2024-04-25T01:13:34.593481+0000","last_became_active":"2024-04-25T01:13:30.520165+0000","last_became_peered":"2024-04-25T01:13:30.520165+0000","last_unstale":"2024-04-25T01:13:34.593481+0000","last_undegraded":"2024-04-25T01:13:34.593481+0000","last_fullsized":"2024-04-25T01:13:34.593481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:26.173090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.495294+0000","last_change":"2024-04-25T01:13:31.517021+0000","last_active":"2024-04-25T01:13:32.495294+0000","last_peered":"2024-04-25T01:13:32.495294+0000","last_clean":"2024-04-25T01:13:32.495294+0000","last_became_active":"2024-04-25T01:13:31.516676+0000","last_became_peered":"2024-04-25T01:13:31.516676+0000","last_unstale":"2024-04-25T01:13:32.495294+0000","last_undegraded":"2024-04-25T01:13:32.495294+0000","last_fullsized":"2024-04-25T01:13:32.495294+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:25.314544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'19","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:38.577943+0000","last_change":"2024-04-25T01:13:30.511963+0000","last_active":"2024-04-25T01:13:38.577943+0000","last_peered":"2024-04-25T01:13:38.577943+0000","last_clean":"2024-04-25T01:13:38.577943+0000","last_became_active":"2024-04-25T01:13:30.511142+0000","last_became_peered":"2024-04-25T01:13:30.511142+0000","last_unstale":"2024-04-25T01:13:38.577943+0000","last_undegraded":"2024-04-25T01:13:38.577943+0000","last_fullsized":"2024-04-25T01:13:38.577943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:53.545598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38193,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":12,"num_read_kb":26,"num_write":20,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.495404+0000","last_change":"2024-04-25T01:13:31.516236+0000","last_active":"2024-04-25T01:13:32.495404+0000","last_peered":"2024-04-25T01:13:32.495404+0000","last_clean":"2024-04-25T01:13:32.495404+0000","last_became_active":"2024-04-25T01:13:31.515871+0000","last_became_peered":"2024-04-25T01:13:31.515871+0000","last_unstale":"2024-04-25T01:13:32.495404+0000","last_undegraded":"2024-04-25T01:13:32.495404+0000","last_fullsized":"2024-04-25T01:13:32.495404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:24.464246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:35.560766+0000","last_change":"2024-04-25T01:13:30.513070+0000","last_active":"2024-04-25T01:13:35.560766+0000","last_peered":"2024-04-25T01:13:35.560766+0000","last_clean":"2024-04-25T01:13:35.560766+0000","last_became_active":"2024-04-25T01:13:30.512701+0000","last_became_peered":"2024-04-25T01:13:30.512701+0000","last_unstale":"2024-04-25T01:13:35.560766+0000","last_undegraded":"2024-04-25T01:13:35.560766+0000","last_fullsized":"2024-04-25T01:13:35.560766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:33.482272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.787225+0000","last_change":"2024-04-25T01:13:31.538313+0000","last_active":"2024-04-25T01:13:33.787225+0000","last_peered":"2024-04-25T01:13:33.787225+0000","last_clean":"2024-04-25T01:13:33.787225+0000","last_became_active":"2024-04-25T01:13:31.537907+0000","last_became_peered":"2024-04-25T01:13:31.537907+0000","last_unstale":"2024-04-25T01:13:33.787225+0000","last_undegraded":"2024-04-25T01:13:33.787225+0000","last_fullsized":"2024-04-25T01:13:33.787225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:19.484311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:34.596792+0000","last_change":"2024-04-25T01:13:30.520170+0000","last_active":"2024-04-25T01:13:34.596792+0000","last_peered":"2024-04-25T01:13:34.596792+0000","last_clean":"2024-04-25T01:13:34.596792+0000","last_became_active":"2024-04-25T01:13:30.519636+0000","last_became_peered":"2024-04-25T01:13:30.519636+0000","last_unstale":"2024-04-25T01:13:34.596792+0000","last_undegraded":"2024-04-25T01:13:34.596792+0000","last_fullsized":"2024-04-25T01:13:34.596792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:00:32.670127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.785623+0000","last_change":"2024-04-25T01:13:31.537630+0000","last_active":"2024-04-25T01:13:33.785623+0000","last_peered":"2024-04-25T01:13:33.785623+0000","last_clean":"2024-04-25T01:13:33.785623+0000","last_became_active":"2024-04-25T01:13:31.537164+0000","last_became_peered":"2024-04-25T01:13:31.537164+0000","last_unstale":"2024-04-25T01:13:33.785623+0000","last_undegraded":"2024-04-25T01:13:33.785623+0000","last_fullsized":"2024-04-25T01:13:33.785623+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:04.091194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:36.555909+0000","last_change":"2024-04-25T01:13:30.514322+0000","last_active":"2024-04-25T01:13:36.555909+0000","last_peered":"2024-04-25T01:13:36.555909+0000","last_clean":"2024-04-25T01:13:36.555909+0000","last_became_active":"2024-04-25T01:13:30.513990+0000","last_became_peered":"2024-04-25T01:13:30.513990+0000","last_unstale":"2024-04-25T01:13:36.555909+0000","last_undegraded":"2024-04-25T01:13:36.555909+0000","last_fullsized":"2024-04-25T01:13:36.555909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:59.995489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784873+0000","last_change":"2024-04-25T01:13:31.519730+0000","last_active":"2024-04-25T01:13:33.784873+0000","last_peered":"2024-04-25T01:13:33.784873+0000","last_clean":"2024-04-25T01:13:33.784873+0000","last_became_active":"2024-04-25T01:13:31.519363+0000","last_became_peered":"2024-04-25T01:13:31.519363+0000","last_unstale":"2024-04-25T01:13:33.784873+0000","last_undegraded":"2024-04-25T01:13:33.784873+0000","last_fullsized":"2024-04-25T01:13:33.784873+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:26.400754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:35.560473+0000","last_change":"2024-04-25T01:13:30.517220+0000","last_active":"2024-04-25T01:13:35.560473+0000","last_peered":"2024-04-25T01:13:35.560473+0000","last_clean":"2024-04-25T01:13:35.560473+0000","last_became_active":"2024-04-25T01:13:30.516825+0000","last_became_peered":"2024-04-25T01:13:30.516825+0000","last_unstale":"2024-04-25T01:13:35.560473+0000","last_undegraded":"2024-04-25T01:13:35.560473+0000","last_fullsized":"2024-04-25T01:13:35.560473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:04.452894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.783919+0000","last_change":"2024-04-25T01:13:31.517424+0000","last_active":"2024-04-25T01:13:33.783919+0000","last_peered":"2024-04-25T01:13:33.783919+0000","last_clean":"2024-04-25T01:13:33.783919+0000","last_became_active":"2024-04-25T01:13:31.517017+0000","last_became_peered":"2024-04-25T01:13:31.517017+0000","last_unstale":"2024-04-25T01:13:33.783919+0000","last_undegraded":"2024-04-25T01:13:33.783919+0000","last_fullsized":"2024-04-25T01:13:33.783919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:16.516450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:35.532404+0000","last_change":"2024-04-25T01:13:30.515249+0000","last_active":"2024-04-25T01:13:35.532404+0000","last_peered":"2024-04-25T01:13:35.532404+0000","last_clean":"2024-04-25T01:13:35.532404+0000","last_became_active":"2024-04-25T01:13:30.514819+0000","last_became_peered":"2024-04-25T01:13:30.514819+0000","last_unstale":"2024-04-25T01:13:35.532404+0000","last_undegraded":"2024-04-25T01:13:35.532404+0000","last_fullsized":"2024-04-25T01:13:35.532404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:52.503564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.642502+0000","last_change":"2024-04-25T01:13:31.519143+0000","last_active":"2024-04-25T01:13:32.642502+0000","last_peered":"2024-04-25T01:13:32.642502+0000","last_clean":"2024-04-25T01:13:32.642502+0000","last_became_active":"2024-04-25T01:13:31.518755+0000","last_became_peered":"2024-04-25T01:13:31.518755+0000","last_unstale":"2024-04-25T01:13:32.642502+0000","last_undegraded":"2024-04-25T01:13:32.642502+0000","last_fullsized":"2024-04-25T01:13:32.642502+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:02.870725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.559759+0000","last_change":"2024-04-25T01:13:30.522489+0000","last_active":"2024-04-25T01:13:32.559759+0000","last_peered":"2024-04-25T01:13:32.559759+0000","last_clean":"2024-04-25T01:13:32.559759+0000","last_became_active":"2024-04-25T01:13:30.522128+0000","last_became_peered":"2024-04-25T01:13:30.522128+0000","last_unstale":"2024-04-25T01:13:32.559759+0000","last_undegraded":"2024-04-25T01:13:32.559759+0000","last_fullsized":"2024-04-25T01:13:32.559759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:12.875627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.872438+0000","last_change":"2024-04-25T01:13:31.537536+0000","last_active":"2024-04-25T01:13:33.872438+0000","last_peered":"2024-04-25T01:13:33.872438+0000","last_clean":"2024-04-25T01:13:33.872438+0000","last_became_active":"2024-04-25T01:13:31.537169+0000","last_became_peered":"2024-04-25T01:13:31.537169+0000","last_unstale":"2024-04-25T01:13:33.872438+0000","last_undegraded":"2024-04-25T01:13:33.872438+0000","last_fullsized":"2024-04-25T01:13:33.872438+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:55.147719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784991+0000","last_change":"2024-04-25T01:13:30.523870+0000","last_active":"2024-04-25T01:13:33.784991+0000","last_peered":"2024-04-25T01:13:33.784991+0000","last_clean":"2024-04-25T01:13:33.784991+0000","last_became_active":"2024-04-25T01:13:30.523504+0000","last_became_peered":"2024-04-25T01:13:30.523504+0000","last_unstale":"2024-04-25T01:13:33.784991+0000","last_undegraded":"2024-04-25T01:13:33.784991+0000","last_fullsized":"2024-04-25T01:13:33.784991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:42.498898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.667722+0000","last_change":"2024-04-25T01:13:31.539691+0000","last_active":"2024-04-25T01:13:32.667722+0000","last_peered":"2024-04-25T01:13:32.667722+0000","last_clean":"2024-04-25T01:13:32.667722+0000","last_became_active":"2024-04-25T01:13:31.539287+0000","last_became_peered":"2024-04-25T01:13:31.539287+0000","last_unstale":"2024-04-25T01:13:32.667722+0000","last_undegraded":"2024-04-25T01:13:32.667722+0000","last_fullsized":"2024-04-25T01:13:32.667722+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:47.985167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784050+0000","last_change":"2024-04-25T01:13:31.513438+0000","last_active":"2024-04-25T01:13:33.784050+0000","last_peered":"2024-04-25T01:13:33.784050+0000","last_clean":"2024-04-25T01:13:33.784050+0000","last_became_active":"2024-04-25T01:13:31.513087+0000","last_became_peered":"2024-04-25T01:13:31.513087+0000","last_unstale":"2024-04-25T01:13:33.784050+0000","last_undegraded":"2024-04-25T01:13:33.784050+0000","last_fullsized":"2024-04-25T01:13:33.784050+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:12.999714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:35.561740+0000","last_change":"2024-04-25T01:13:30.527528+0000","last_active":"2024-04-25T01:13:35.561740+0000","last_peered":"2024-04-25T01:13:35.561740+0000","last_clean":"2024-04-25T01:13:35.561740+0000","last_became_active":"2024-04-25T01:13:30.526040+0000","last_became_peered":"2024-04-25T01:13:30.526040+0000","last_unstale":"2024-04-25T01:13:35.561740+0000","last_undegraded":"2024-04-25T01:13:35.561740+0000","last_fullsized":"2024-04-25T01:13:35.561740+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:31.943727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.497478+0000","last_change":"2024-04-25T01:13:31.523311+0000","last_active":"2024-04-25T01:13:32.497478+0000","last_peered":"2024-04-25T01:13:32.497478+0000","last_clean":"2024-04-25T01:13:32.497478+0000","last_became_active":"2024-04-25T01:13:31.518099+0000","last_became_peered":"2024-04-25T01:13:31.518099+0000","last_unstale":"2024-04-25T01:13:32.497478+0000","last_undegraded":"2024-04-25T01:13:32.497478+0000","last_fullsized":"2024-04-25T01:13:32.497478+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:02.947202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:35.553491+0000","last_change":"2024-04-25T01:13:30.512804+0000","last_active":"2024-04-25T01:13:35.553491+0000","last_peered":"2024-04-25T01:13:35.553491+0000","last_clean":"2024-04-25T01:13:35.553491+0000","last_became_active":"2024-04-25T01:13:30.512406+0000","last_became_peered":"2024-04-25T01:13:30.512406+0000","last_unstale":"2024-04-25T01:13:35.553491+0000","last_undegraded":"2024-04-25T01:13:35.553491+0000","last_fullsized":"2024-04-25T01:13:35.553491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:00.042244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":340,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784803+0000","last_change":"2024-04-25T01:12:07.906198+0000","last_active":"2024-04-25T01:13:33.784803+0000","last_peered":"2024-04-25T01:13:33.784803+0000","last_clean":"2024-04-25T01:13:33.784803+0000","last_became_active":"2024-04-25T01:12:07.905724+0000","last_became_peered":"2024-04-25T01:12:07.905724+0000","last_unstale":"2024-04-25T01:13:33.784803+0000","last_undegraded":"2024-04-25T01:13:33.784803+0000","last_fullsized":"2024-04-25T01:13:33.784803+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:18.712681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:18.712681+0000","last_clean_scrub_stamp":"2024-04-25T01:08:18.712681+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:13.899974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.785543+0000","last_change":"2024-04-25T01:13:31.534664+0000","last_active":"2024-04-25T01:13:33.785543+0000","last_peered":"2024-04-25T01:13:33.785543+0000","last_clean":"2024-04-25T01:13:33.785543+0000","last_became_active":"2024-04-25T01:13:31.530082+0000","last_became_peered":"2024-04-25T01:13:31.530082+0000","last_unstale":"2024-04-25T01:13:33.785543+0000","last_undegraded":"2024-04-25T01:13:33.785543+0000","last_fullsized":"2024-04-25T01:13:33.785543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:08:32.717997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.785037+0000","last_change":"2024-04-25T01:13:30.514295+0000","last_active":"2024-04-25T01:13:33.785037+0000","last_peered":"2024-04-25T01:13:33.785037+0000","last_clean":"2024-04-25T01:13:33.785037+0000","last_became_active":"2024-04-25T01:13:30.513844+0000","last_became_peered":"2024-04-25T01:13:30.513844+0000","last_unstale":"2024-04-25T01:13:33.785037+0000","last_undegraded":"2024-04-25T01:13:33.785037+0000","last_fullsized":"2024-04-25T01:13:33.785037+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:36.652771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:34.597140+0000","last_change":"2024-04-25T01:13:30.523088+0000","last_active":"2024-04-25T01:13:34.597140+0000","last_peered":"2024-04-25T01:13:34.597140+0000","last_clean":"2024-04-25T01:13:34.597140+0000","last_became_active":"2024-04-25T01:13:30.522642+0000","last_became_peered":"2024-04-25T01:13:30.522642+0000","last_unstale":"2024-04-25T01:13:34.597140+0000","last_undegraded":"2024-04-25T01:13:34.597140+0000","last_fullsized":"2024-04-25T01:13:34.597140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:55.501983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.787889+0000","last_change":"2024-04-25T01:13:31.531703+0000","last_active":"2024-04-25T01:13:33.787889+0000","last_peered":"2024-04-25T01:13:33.787889+0000","last_clean":"2024-04-25T01:13:33.787889+0000","last_became_active":"2024-04-25T01:13:31.531160+0000","last_became_peered":"2024-04-25T01:13:31.531160+0000","last_unstale":"2024-04-25T01:13:33.787889+0000","last_undegraded":"2024-04-25T01:13:33.787889+0000","last_fullsized":"2024-04-25T01:13:33.787889+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:21.993998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:35.562148+0000","last_change":"2024-04-25T01:13:30.517908+0000","last_active":"2024-04-25T01:13:35.562148+0000","last_peered":"2024-04-25T01:13:35.562148+0000","last_clean":"2024-04-25T01:13:35.562148+0000","last_became_active":"2024-04-25T01:13:30.517559+0000","last_became_peered":"2024-04-25T01:13:30.517559+0000","last_unstale":"2024-04-25T01:13:35.562148+0000","last_undegraded":"2024-04-25T01:13:35.562148+0000","last_fullsized":"2024-04-25T01:13:35.562148+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:44.357328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.497950+0000","last_change":"2024-04-25T01:13:31.515760+0000","last_active":"2024-04-25T01:13:32.497950+0000","last_peered":"2024-04-25T01:13:32.497950+0000","last_clean":"2024-04-25T01:13:32.497950+0000","last_became_active":"2024-04-25T01:13:31.514957+0000","last_became_peered":"2024-04-25T01:13:31.514957+0000","last_unstale":"2024-04-25T01:13:32.497950+0000","last_undegraded":"2024-04-25T01:13:32.497950+0000","last_fullsized":"2024-04-25T01:13:32.497950+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:46.169171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:34.592541+0000","last_change":"2024-04-25T01:13:30.523933+0000","last_active":"2024-04-25T01:13:34.592541+0000","last_peered":"2024-04-25T01:13:34.592541+0000","last_clean":"2024-04-25T01:13:34.592541+0000","last_became_active":"2024-04-25T01:13:30.523497+0000","last_became_peered":"2024-04-25T01:13:30.523497+0000","last_unstale":"2024-04-25T01:13:34.592541+0000","last_undegraded":"2024-04-25T01:13:34.592541+0000","last_fullsized":"2024-04-25T01:13:34.592541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:17:59.336812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.787296+0000","last_change":"2024-04-25T01:13:31.540188+0000","last_active":"2024-04-25T01:13:33.787296+0000","last_peered":"2024-04-25T01:13:33.787296+0000","last_clean":"2024-04-25T01:13:33.787296+0000","last_became_active":"2024-04-25T01:13:31.537121+0000","last_became_peered":"2024-04-25T01:13:31.537121+0000","last_unstale":"2024-04-25T01:13:33.787296+0000","last_undegraded":"2024-04-25T01:13:33.787296+0000","last_fullsized":"2024-04-25T01:13:33.787296+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:57.162639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:34.597355+0000","last_change":"2024-04-25T01:13:30.516975+0000","last_active":"2024-04-25T01:13:34.597355+0000","last_peered":"2024-04-25T01:13:34.597355+0000","last_clean":"2024-04-25T01:13:34.597355+0000","last_became_active":"2024-04-25T01:13:30.516391+0000","last_became_peered":"2024-04-25T01:13:30.516391+0000","last_unstale":"2024-04-25T01:13:34.597355+0000","last_undegraded":"2024-04-25T01:13:34.597355+0000","last_fullsized":"2024-04-25T01:13:34.597355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:57:53.486835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.494819+0000","last_change":"2024-04-25T01:13:31.514491+0000","last_active":"2024-04-25T01:13:32.494819+0000","last_peered":"2024-04-25T01:13:32.494819+0000","last_clean":"2024-04-25T01:13:32.494819+0000","last_became_active":"2024-04-25T01:13:31.514156+0000","last_became_peered":"2024-04-25T01:13:31.514156+0000","last_unstale":"2024-04-25T01:13:32.494819+0000","last_undegraded":"2024-04-25T01:13:32.494819+0000","last_fullsized":"2024-04-25T01:13:32.494819+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:18.588291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:34.598150+0000","last_change":"2024-04-25T01:13:30.521258+0000","last_active":"2024-04-25T01:13:34.598150+0000","last_peered":"2024-04-25T01:13:34.598150+0000","last_clean":"2024-04-25T01:13:34.598150+0000","last_became_active":"2024-04-25T01:13:30.518359+0000","last_became_peered":"2024-04-25T01:13:30.518359+0000","last_unstale":"2024-04-25T01:13:34.598150+0000","last_undegraded":"2024-04-25T01:13:34.598150+0000","last_fullsized":"2024-04-25T01:13:34.598150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:35.790798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.642277+0000","last_change":"2024-04-25T01:13:31.536755+0000","last_active":"2024-04-25T01:13:32.642277+0000","last_peered":"2024-04-25T01:13:32.642277+0000","last_clean":"2024-04-25T01:13:32.642277+0000","last_became_active":"2024-04-25T01:13:31.536303+0000","last_became_peered":"2024-04-25T01:13:31.536303+0000","last_unstale":"2024-04-25T01:13:32.642277+0000","last_undegraded":"2024-04-25T01:13:32.642277+0000","last_fullsized":"2024-04-25T01:13:32.642277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:44.793993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784439+0000","last_change":"2024-04-25T01:13:30.513635+0000","last_active":"2024-04-25T01:13:33.784439+0000","last_peered":"2024-04-25T01:13:33.784439+0000","last_clean":"2024-04-25T01:13:33.784439+0000","last_became_active":"2024-04-25T01:13:30.511902+0000","last_became_peered":"2024-04-25T01:13:30.511902+0000","last_unstale":"2024-04-25T01:13:33.784439+0000","last_undegraded":"2024-04-25T01:13:33.784439+0000","last_fullsized":"2024-04-25T01:13:33.784439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:52.506547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.497421+0000","last_change":"2024-04-25T01:13:31.543234+0000","last_active":"2024-04-25T01:13:32.497421+0000","last_peered":"2024-04-25T01:13:32.497421+0000","last_clean":"2024-04-25T01:13:32.497421+0000","last_became_active":"2024-04-25T01:13:31.542419+0000","last_became_peered":"2024-04-25T01:13:31.542419+0000","last_unstale":"2024-04-25T01:13:32.497421+0000","last_undegraded":"2024-04-25T01:13:32.497421+0000","last_fullsized":"2024-04-25T01:13:32.497421+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:57.226505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:34.592128+0000","last_change":"2024-04-25T01:13:30.516913+0000","last_active":"2024-04-25T01:13:34.592128+0000","last_peered":"2024-04-25T01:13:34.592128+0000","last_clean":"2024-04-25T01:13:34.592128+0000","last_became_active":"2024-04-25T01:13:30.516478+0000","last_became_peered":"2024-04-25T01:13:30.516478+0000","last_unstale":"2024-04-25T01:13:34.592128+0000","last_undegraded":"2024-04-25T01:13:34.592128+0000","last_fullsized":"2024-04-25T01:13:34.592128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:25.502072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.785184+0000","last_change":"2024-04-25T01:13:31.532194+0000","last_active":"2024-04-25T01:13:33.785184+0000","last_peered":"2024-04-25T01:13:33.785184+0000","last_clean":"2024-04-25T01:13:33.785184+0000","last_became_active":"2024-04-25T01:13:31.531798+0000","last_became_peered":"2024-04-25T01:13:31.531798+0000","last_unstale":"2024-04-25T01:13:33.785184+0000","last_undegraded":"2024-04-25T01:13:33.785184+0000","last_fullsized":"2024-04-25T01:13:33.785184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:36.872827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:35.561022+0000","last_change":"2024-04-25T01:13:30.522565+0000","last_active":"2024-04-25T01:13:35.561022+0000","last_peered":"2024-04-25T01:13:35.561022+0000","last_clean":"2024-04-25T01:13:35.561022+0000","last_became_active":"2024-04-25T01:13:30.522207+0000","last_became_peered":"2024-04-25T01:13:30.522207+0000","last_unstale":"2024-04-25T01:13:35.561022+0000","last_undegraded":"2024-04-25T01:13:35.561022+0000","last_fullsized":"2024-04-25T01:13:35.561022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:48.263665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.785683+0000","last_change":"2024-04-25T01:13:31.514330+0000","last_active":"2024-04-25T01:13:33.785683+0000","last_peered":"2024-04-25T01:13:33.785683+0000","last_clean":"2024-04-25T01:13:33.785683+0000","last_became_active":"2024-04-25T01:13:31.513948+0000","last_became_peered":"2024-04-25T01:13:31.513948+0000","last_unstale":"2024-04-25T01:13:33.785683+0000","last_undegraded":"2024-04-25T01:13:33.785683+0000","last_fullsized":"2024-04-25T01:13:33.785683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:57.544298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:34.351495+0000","last_change":"2024-04-25T01:13:30.515951+0000","last_active":"2024-04-25T01:13:34.351495+0000","last_peered":"2024-04-25T01:13:34.351495+0000","last_clean":"2024-04-25T01:13:34.351495+0000","last_became_active":"2024-04-25T01:13:30.515388+0000","last_became_peered":"2024-04-25T01:13:30.515388+0000","last_unstale":"2024-04-25T01:13:34.351495+0000","last_undegraded":"2024-04-25T01:13:34.351495+0000","last_fullsized":"2024-04-25T01:13:34.351495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:11.041466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.785615+0000","last_change":"2024-04-25T01:13:31.503386+0000","last_active":"2024-04-25T01:13:33.785615+0000","last_peered":"2024-04-25T01:13:33.785615+0000","last_clean":"2024-04-25T01:13:33.785615+0000","last_became_active":"2024-04-25T01:13:31.502966+0000","last_became_peered":"2024-04-25T01:13:31.502966+0000","last_unstale":"2024-04-25T01:13:33.785615+0000","last_undegraded":"2024-04-25T01:13:33.785615+0000","last_fullsized":"2024-04-25T01:13:33.785615+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:07.520811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.789001+0000","last_change":"2024-04-25T01:13:30.528033+0000","last_active":"2024-04-25T01:13:33.789001+0000","last_peered":"2024-04-25T01:13:33.789001+0000","last_clean":"2024-04-25T01:13:33.789001+0000","last_became_active":"2024-04-25T01:13:30.527609+0000","last_became_peered":"2024-04-25T01:13:30.527609+0000","last_unstale":"2024-04-25T01:13:33.789001+0000","last_undegraded":"2024-04-25T01:13:33.789001+0000","last_fullsized":"2024-04-25T01:13:33.789001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:08.182355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.785675+0000","last_change":"2024-04-25T01:13:31.516569+0000","last_active":"2024-04-25T01:13:33.785675+0000","last_peered":"2024-04-25T01:13:33.785675+0000","last_clean":"2024-04-25T01:13:33.785675+0000","last_became_active":"2024-04-25T01:13:31.515746+0000","last_became_peered":"2024-04-25T01:13:31.515746+0000","last_unstale":"2024-04-25T01:13:33.785675+0000","last_undegraded":"2024-04-25T01:13:33.785675+0000","last_fullsized":"2024-04-25T01:13:33.785675+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:27.860296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.642447+0000","last_change":"2024-04-25T01:13:31.518161+0000","last_active":"2024-04-25T01:13:32.642447+0000","last_peered":"2024-04-25T01:13:32.642447+0000","last_clean":"2024-04-25T01:13:32.642447+0000","last_became_active":"2024-04-25T01:13:31.517826+0000","last_became_peered":"2024-04-25T01:13:31.517826+0000","last_unstale":"2024-04-25T01:13:32.642447+0000","last_undegraded":"2024-04-25T01:13:32.642447+0000","last_fullsized":"2024-04-25T01:13:32.642447+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:42.211894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784875+0000","last_change":"2024-04-25T01:13:31.514663+0000","last_active":"2024-04-25T01:13:33.784875+0000","last_peered":"2024-04-25T01:13:33.784875+0000","last_clean":"2024-04-25T01:13:33.784875+0000","last_became_active":"2024-04-25T01:13:31.514246+0000","last_became_peered":"2024-04-25T01:13:31.514246+0000","last_unstale":"2024-04-25T01:13:33.784875+0000","last_undegraded":"2024-04-25T01:13:33.784875+0000","last_fullsized":"2024-04-25T01:13:33.784875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:27.235598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784264+0000","last_change":"2024-04-25T01:13:31.533374+0000","last_active":"2024-04-25T01:13:33.784264+0000","last_peered":"2024-04-25T01:13:33.784264+0000","last_clean":"2024-04-25T01:13:33.784264+0000","last_became_active":"2024-04-25T01:13:31.533145+0000","last_became_peered":"2024-04-25T01:13:31.533145+0000","last_unstale":"2024-04-25T01:13:33.784264+0000","last_undegraded":"2024-04-25T01:13:33.784264+0000","last_fullsized":"2024-04-25T01:13:33.784264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:50.987823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.786295+0000","last_change":"2024-04-25T01:13:31.541682+0000","last_active":"2024-04-25T01:13:33.786295+0000","last_peered":"2024-04-25T01:13:33.786295+0000","last_clean":"2024-04-25T01:13:33.786295+0000","last_became_active":"2024-04-25T01:13:31.541257+0000","last_became_peered":"2024-04-25T01:13:31.541257+0000","last_unstale":"2024-04-25T01:13:33.786295+0000","last_undegraded":"2024-04-25T01:13:33.786295+0000","last_fullsized":"2024-04-25T01:13:33.786295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:08.033895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.788402+0000","last_change":"2024-04-25T01:13:31.539521+0000","last_active":"2024-04-25T01:13:33.788402+0000","last_peered":"2024-04-25T01:13:33.788402+0000","last_clean":"2024-04-25T01:13:33.788402+0000","last_became_active":"2024-04-25T01:13:31.538973+0000","last_became_peered":"2024-04-25T01:13:31.538973+0000","last_unstale":"2024-04-25T01:13:33.788402+0000","last_undegraded":"2024-04-25T01:13:33.788402+0000","last_fullsized":"2024-04-25T01:13:33.788402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:06.425942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.642574+0000","last_change":"2024-04-25T01:13:31.519068+0000","last_active":"2024-04-25T01:13:32.642574+0000","last_peered":"2024-04-25T01:13:32.642574+0000","last_clean":"2024-04-25T01:13:32.642574+0000","last_became_active":"2024-04-25T01:13:31.518705+0000","last_became_peered":"2024-04-25T01:13:31.518705+0000","last_unstale":"2024-04-25T01:13:32.642574+0000","last_undegraded":"2024-04-25T01:13:32.642574+0000","last_fullsized":"2024-04-25T01:13:32.642574+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:11.186149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784569+0000","last_change":"2024-04-25T01:13:31.514572+0000","last_active":"2024-04-25T01:13:33.784569+0000","last_peered":"2024-04-25T01:13:33.784569+0000","last_clean":"2024-04-25T01:13:33.784569+0000","last_became_active":"2024-04-25T01:13:31.514178+0000","last_became_peered":"2024-04-25T01:13:31.514178+0000","last_unstale":"2024-04-25T01:13:33.784569+0000","last_undegraded":"2024-04-25T01:13:33.784569+0000","last_fullsized":"2024-04-25T01:13:33.784569+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:17.856802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.497951+0000","last_change":"2024-04-25T01:13:31.520837+0000","last_active":"2024-04-25T01:13:32.497951+0000","last_peered":"2024-04-25T01:13:32.497951+0000","last_clean":"2024-04-25T01:13:32.497951+0000","last_became_active":"2024-04-25T01:13:31.515601+0000","last_became_peered":"2024-04-25T01:13:31.515601+0000","last_unstale":"2024-04-25T01:13:32.497951+0000","last_undegraded":"2024-04-25T01:13:32.497951+0000","last_fullsized":"2024-04-25T01:13:32.497951+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:37.583156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784063+0000","last_change":"2024-04-25T01:13:31.527420+0000","last_active":"2024-04-25T01:13:33.784063+0000","last_peered":"2024-04-25T01:13:33.784063+0000","last_clean":"2024-04-25T01:13:33.784063+0000","last_became_active":"2024-04-25T01:13:31.527160+0000","last_became_peered":"2024-04-25T01:13:31.527160+0000","last_unstale":"2024-04-25T01:13:33.784063+0000","last_undegraded":"2024-04-25T01:13:33.784063+0000","last_fullsized":"2024-04-25T01:13:33.784063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:17:46.201486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.496914+0000","last_change":"2024-04-25T01:13:31.518356+0000","last_active":"2024-04-25T01:13:32.496914+0000","last_peered":"2024-04-25T01:13:32.496914+0000","last_clean":"2024-04-25T01:13:32.496914+0000","last_became_active":"2024-04-25T01:13:31.517654+0000","last_became_peered":"2024-04-25T01:13:31.517654+0000","last_unstale":"2024-04-25T01:13:32.496914+0000","last_undegraded":"2024-04-25T01:13:32.496914+0000","last_fullsized":"2024-04-25T01:13:32.496914+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:17.254963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.495100+0000","last_change":"2024-04-25T01:13:31.526217+0000","last_active":"2024-04-25T01:13:32.495100+0000","last_peered":"2024-04-25T01:13:32.495100+0000","last_clean":"2024-04-25T01:13:32.495100+0000","last_became_active":"2024-04-25T01:13:31.525801+0000","last_became_peered":"2024-04-25T01:13:31.525801+0000","last_unstale":"2024-04-25T01:13:32.495100+0000","last_undegraded":"2024-04-25T01:13:32.495100+0000","last_fullsized":"2024-04-25T01:13:32.495100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:48:30.244205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.787594+0000","last_change":"2024-04-25T01:13:31.543596+0000","last_active":"2024-04-25T01:13:33.787594+0000","last_peered":"2024-04-25T01:13:33.787594+0000","last_clean":"2024-04-25T01:13:33.787594+0000","last_became_active":"2024-04-25T01:13:31.542149+0000","last_became_peered":"2024-04-25T01:13:31.542149+0000","last_unstale":"2024-04-25T01:13:33.787594+0000","last_undegraded":"2024-04-25T01:13:33.787594+0000","last_fullsized":"2024-04-25T01:13:33.787594+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:05:50.392151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.642431+0000","last_change":"2024-04-25T01:13:31.513575+0000","last_active":"2024-04-25T01:13:32.642431+0000","last_peered":"2024-04-25T01:13:32.642431+0000","last_clean":"2024-04-25T01:13:32.642431+0000","last_became_active":"2024-04-25T01:13:31.513085+0000","last_became_peered":"2024-04-25T01:13:31.513085+0000","last_unstale":"2024-04-25T01:13:32.642431+0000","last_undegraded":"2024-04-25T01:13:32.642431+0000","last_fullsized":"2024-04-25T01:13:32.642431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:36.853033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":41803,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":39,"num_read_kb":48,"num_write":135,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":319488,"data_stored":95231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318289,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":445784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579925,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1589248,"data_stored":1058551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743450,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":446020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005089,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":445940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266725,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32680,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712472,"statfs":{"total":95995035648,"available":95961571328,"internally_reserved":0,"allocated":1536000,"data_stored":1036774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430251,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":929792,"data_stored":445202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":466097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":446680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":448164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117050,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":946176,"data_stored":445486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855414,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":445846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691889,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1556480,"data_stored":1037163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":38193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":9562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":40162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:13:40.526 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:13:40.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.789+0000 7fa515400640 1 -- 172.21.15.31:0/62693521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa510107bc0 msgr2=0x7fa510107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:40.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.789+0000 7fa515400640 1 --2- 172.21.15.31:0/62693521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa510107bc0 0x7fa510107fe0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa500009900 tx=0x7fa50002a410 comp rx=0 tx=0).stop 2024-04-25T01:13:40.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.789+0000 7fa515400640 1 -- 172.21.15.31:0/62693521 shutdown_connections 2024-04-25T01:13:40.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.789+0000 7fa515400640 1 --2- 172.21.15.31:0/62693521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa510109770 0x7fa510110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:40.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.789+0000 7fa515400640 1 --2- 172.21.15.31:0/62693521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa510108db0 0x7fa510109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:40.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.789+0000 7fa515400640 1 --2- 172.21.15.31:0/62693521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa510107bc0 0x7fa510107fe0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:40.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.789+0000 7fa515400640 1 -- 172.21.15.31:0/62693521 >> 172.21.15.31:0/62693521 conn(0x7fa510076b80 msgr2=0x7fa510078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:40.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.789+0000 7fa515400640 1 -- 172.21.15.31:0/62693521 shutdown_connections 2024-04-25T01:13:40.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.789+0000 7fa515400640 1 -- 172.21.15.31:0/62693521 wait complete. 2024-04-25T01:13:40.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.790+0000 7fa515400640 1 Processor -- start 2024-04-25T01:13:40.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.790+0000 7fa515400640 1 -- start start 2024-04-25T01:13:40.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.791+0000 7fa515400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa510107bc0 0x7fa51019df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:40.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.791+0000 7fa515400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa510108db0 0x7fa51019e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:40.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.791+0000 7fa515400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa510109770 0x7fa5101a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:40.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.791+0000 7fa515400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa510113860 con 0x7fa510109770 2024-04-25T01:13:40.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.791+0000 7fa515400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa5101136e0 con 0x7fa510108db0 2024-04-25T01:13:40.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.791+0000 7fa515400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa5101139e0 con 0x7fa510107bc0 2024-04-25T01:13:40.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.791+0000 7fa507400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa510108db0 0x7fa51019e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:40.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.791+0000 7fa514a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa510109770 0x7fa5101a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:40.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.791+0000 7fa507400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa510108db0 0x7fa51019e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:40956/0 (socket says 172.21.15.31:40956) 2024-04-25T01:13:40.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.791+0000 7fa507400640 1 -- 172.21.15.31:0/2913532888 learned_addr learned my addr 172.21.15.31:0/2913532888 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.791+0000 7fa50fe00640 1 --2- 172.21.15.31:0/2913532888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa510107bc0 0x7fa51019df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.791+0000 7fa507400640 1 -- 172.21.15.31:0/2913532888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa510107bc0 msgr2=0x7fa51019df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.791+0000 7fa507400640 1 --2- 172.21.15.31:0/2913532888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa510107bc0 0x7fa51019df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.791+0000 7fa507400640 1 -- 172.21.15.31:0/2913532888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa510109770 msgr2=0x7fa5101a54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.791+0000 7fa507400640 1 --2- 172.21.15.31:0/2913532888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa510109770 0x7fa5101a54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.791+0000 7fa507400640 1 -- 172.21.15.31:0/2913532888 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5101a5b20 con 0x7fa510108db0 2024-04-25T01:13:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.791+0000 7fa514a00640 1 --2- 172.21.15.31:0/2913532888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa510109770 0x7fa5101a54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:13:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.792+0000 7fa507400640 1 --2- 172.21.15.31:0/2913532888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa510108db0 0x7fa51019e490 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa4f800ea60 tx=0x7fa4f800ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.792+0000 7fa50d600640 1 -- 172.21.15.31:0/2913532888 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f800cd20 con 0x7fa510108db0 2024-04-25T01:13:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.792+0000 7fa50d600640 1 -- 172.21.15.31:0/2913532888 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f80035f0 con 0x7fa510108db0 2024-04-25T01:13:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.792+0000 7fa515400640 1 -- 172.21.15.31:0/2913532888 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5101a5e10 con 0x7fa510108db0 2024-04-25T01:13:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.792+0000 7fa50d600640 1 -- 172.21.15.31:0/2913532888 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8010690 con 0x7fa510108db0 2024-04-25T01:13:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.792+0000 7fa515400640 1 -- 172.21.15.31:0/2913532888 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5101a6320 con 0x7fa510108db0 2024-04-25T01:13:40.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.793+0000 7fa515400640 1 -- 172.21.15.31:0/2913532888 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa51010efb0 con 0x7fa510108db0 2024-04-25T01:13:40.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.795+0000 7fa50d600640 1 -- 172.21.15.31:0/2913532888 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa4f80037a0 con 0x7fa510108db0 2024-04-25T01:13:40.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.796+0000 7fa50d600640 1 --2- 172.21.15.31:0/2913532888 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa4f40a2ca0 0x7fa4f40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:40.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.796+0000 7fa50d600640 1 -- 172.21.15.31:0/2913532888 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fa4f80cf370 con 0x7fa510108db0 2024-04-25T01:13:40.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.797+0000 7fa50fe00640 1 --2- 172.21.15.31:0/2913532888 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa4f40a2ca0 0x7fa4f40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:40.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.797+0000 7fa50fe00640 1 --2- 172.21.15.31:0/2913532888 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa4f40a2ca0 0x7fa4f40a5160 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa50002aa00 tx=0x7fa5000315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:40.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:40.801+0000 7fa50d600640 1 -- 172.21.15.31:0/2913532888 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa4f809c220 con 0x7fa510108db0 2024-04-25T01:13:40.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:40 smithi169 ceph-mon[32759]: from='client.14595 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:13:40.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:40 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3471414325' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:13:40.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:40 smithi040 ceph-mon[32518]: from='client.14595 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:13:40.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:40 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3471414325' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:13:40.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:41.001+0000 7fa515400640 1 -- 172.21.15.31:0/2913532888 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa510061b80 con 0x7fa4f40a2ca0 2024-04-25T01:13:41.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:40 smithi031 ceph-mon[28053]: from='client.14595 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:13:41.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:40 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3471414325' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:13:41.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:41.025+0000 7fa50d600640 1 -- 172.21.15.31:0/2913532888 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258773 (secure 0 0 0) 0x7fa510061b80 con 0x7fa4f40a2ca0 2024-04-25T01:13:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:13:41.017 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-04-25T01:13:41.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:41.030+0000 7fa515400640 1 -- 172.21.15.31:0/2913532888 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa4f40a2ca0 msgr2=0x7fa4f40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:41.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:41.030+0000 7fa515400640 1 --2- 172.21.15.31:0/2913532888 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa4f40a2ca0 0x7fa4f40a5160 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa50002aa00 tx=0x7fa5000315b0 comp rx=0 tx=0).stop 2024-04-25T01:13:41.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:41.030+0000 7fa515400640 1 -- 172.21.15.31:0/2913532888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa510108db0 msgr2=0x7fa51019e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:41.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:41.030+0000 7fa515400640 1 --2- 172.21.15.31:0/2913532888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa510108db0 0x7fa51019e490 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa4f800ea60 tx=0x7fa4f800ee60 comp rx=0 tx=0).stop 2024-04-25T01:13:41.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:41.030+0000 7fa515400640 1 -- 172.21.15.31:0/2913532888 shutdown_connections 2024-04-25T01:13:41.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:41.030+0000 7fa515400640 1 --2- 172.21.15.31:0/2913532888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa510109770 0x7fa5101a54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:41.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:41.030+0000 7fa515400640 1 --2- 172.21.15.31:0/2913532888 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa4f40a2ca0 0x7fa4f40a5160 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:41.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:41.030+0000 7fa515400640 1 --2- 172.21.15.31:0/2913532888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa510108db0 0x7fa51019e490 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:41.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:41.030+0000 7fa515400640 1 --2- 172.21.15.31:0/2913532888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa510107bc0 0x7fa51019df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:41.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:41.030+0000 7fa515400640 1 -- 172.21.15.31:0/2913532888 >> 172.21.15.31:0/2913532888 conn(0x7fa510076b80 msgr2=0x7fa51010c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:41.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:41.031+0000 7fa515400640 1 -- 172.21.15.31:0/2913532888 shutdown_connections 2024-04-25T01:13:41.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:41.031+0000 7fa515400640 1 -- 172.21.15.31:0/2913532888 wait complete. 2024-04-25T01:13:41.037 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-04-25T01:13:40.072760+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632288,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":165,"num_read_kb":157,"num_write":317,"num_write_kb":3016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":256,"ondisk_log_size":256,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374556,"kb_used_data":12908,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567268,"statfs":{"total":1151940427776,"available":1151556882432,"internally_reserved":0,"allocated":13217792,"data_stored":7167707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":41920,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":39,"num_read_kb":48,"num_write":136,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.581218"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.642563+0000","last_change":"2024-04-25T01:13:31.536782+0000","last_active":"2024-04-25T01:13:32.642563+0000","last_peered":"2024-04-25T01:13:32.642563+0000","last_clean":"2024-04-25T01:13:32.642563+0000","last_became_active":"2024-04-25T01:13:31.536426+0000","last_became_peered":"2024-04-25T01:13:31.536426+0000","last_unstale":"2024-04-25T01:13:32.642563+0000","last_undegraded":"2024-04-25T01:13:32.642563+0000","last_fullsized":"2024-04-25T01:13:32.642563+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:07.489179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.786932+0000","last_change":"2024-04-25T01:13:31.543521+0000","last_active":"2024-04-25T01:13:33.786932+0000","last_peered":"2024-04-25T01:13:33.786932+0000","last_clean":"2024-04-25T01:13:33.786932+0000","last_became_active":"2024-04-25T01:13:31.542767+0000","last_became_peered":"2024-04-25T01:13:31.542767+0000","last_unstale":"2024-04-25T01:13:33.786932+0000","last_undegraded":"2024-04-25T01:13:33.786932+0000","last_fullsized":"2024-04-25T01:13:33.786932+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:33.287564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.787132+0000","last_change":"2024-04-25T01:13:31.537297+0000","last_active":"2024-04-25T01:13:33.787132+0000","last_peered":"2024-04-25T01:13:33.787132+0000","last_clean":"2024-04-25T01:13:33.787132+0000","last_became_active":"2024-04-25T01:13:31.536930+0000","last_became_peered":"2024-04-25T01:13:31.536930+0000","last_unstale":"2024-04-25T01:13:33.787132+0000","last_undegraded":"2024-04-25T01:13:33.787132+0000","last_fullsized":"2024-04-25T01:13:33.787132+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:58:35.465287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784311+0000","last_change":"2024-04-25T01:13:31.516514+0000","last_active":"2024-04-25T01:13:33.784311+0000","last_peered":"2024-04-25T01:13:33.784311+0000","last_clean":"2024-04-25T01:13:33.784311+0000","last_became_active":"2024-04-25T01:13:31.516192+0000","last_became_peered":"2024-04-25T01:13:31.516192+0000","last_unstale":"2024-04-25T01:13:33.784311+0000","last_undegraded":"2024-04-25T01:13:33.784311+0000","last_fullsized":"2024-04-25T01:13:33.784311+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:27.416626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.495010+0000","last_change":"2024-04-25T01:13:31.538662+0000","last_active":"2024-04-25T01:13:32.495010+0000","last_peered":"2024-04-25T01:13:32.495010+0000","last_clean":"2024-04-25T01:13:32.495010+0000","last_became_active":"2024-04-25T01:13:31.538362+0000","last_became_peered":"2024-04-25T01:13:31.538362+0000","last_unstale":"2024-04-25T01:13:32.495010+0000","last_undegraded":"2024-04-25T01:13:32.495010+0000","last_fullsized":"2024-04-25T01:13:32.495010+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:10.744891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.667900+0000","last_change":"2024-04-25T01:13:31.533051+0000","last_active":"2024-04-25T01:13:32.667900+0000","last_peered":"2024-04-25T01:13:32.667900+0000","last_clean":"2024-04-25T01:13:32.667900+0000","last_became_active":"2024-04-25T01:13:31.532652+0000","last_became_peered":"2024-04-25T01:13:31.532652+0000","last_unstale":"2024-04-25T01:13:32.667900+0000","last_undegraded":"2024-04-25T01:13:32.667900+0000","last_fullsized":"2024-04-25T01:13:32.667900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:41.998434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784902+0000","last_change":"2024-04-25T01:13:31.530362+0000","last_active":"2024-04-25T01:13:33.784902+0000","last_peered":"2024-04-25T01:13:33.784902+0000","last_clean":"2024-04-25T01:13:33.784902+0000","last_became_active":"2024-04-25T01:13:31.529869+0000","last_became_peered":"2024-04-25T01:13:31.529869+0000","last_unstale":"2024-04-25T01:13:33.784902+0000","last_undegraded":"2024-04-25T01:13:33.784902+0000","last_fullsized":"2024-04-25T01:13:33.784902+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:54.499230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.496815+0000","last_change":"2024-04-25T01:13:31.522133+0000","last_active":"2024-04-25T01:13:32.496815+0000","last_peered":"2024-04-25T01:13:32.496815+0000","last_clean":"2024-04-25T01:13:32.496815+0000","last_became_active":"2024-04-25T01:13:31.521552+0000","last_became_peered":"2024-04-25T01:13:31.521552+0000","last_unstale":"2024-04-25T01:13:32.496815+0000","last_undegraded":"2024-04-25T01:13:32.496815+0000","last_fullsized":"2024-04-25T01:13:32.496815+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:06.651854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.495576+0000","last_change":"2024-04-25T01:13:31.518589+0000","last_active":"2024-04-25T01:13:32.495576+0000","last_peered":"2024-04-25T01:13:32.495576+0000","last_clean":"2024-04-25T01:13:32.495576+0000","last_became_active":"2024-04-25T01:13:31.518183+0000","last_became_peered":"2024-04-25T01:13:31.518183+0000","last_unstale":"2024-04-25T01:13:32.495576+0000","last_undegraded":"2024-04-25T01:13:32.495576+0000","last_fullsized":"2024-04-25T01:13:32.495576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:06:17.534019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.668063+0000","last_change":"2024-04-25T01:13:31.522281+0000","last_active":"2024-04-25T01:13:32.668063+0000","last_peered":"2024-04-25T01:13:32.668063+0000","last_clean":"2024-04-25T01:13:32.668063+0000","last_became_active":"2024-04-25T01:13:31.521906+0000","last_became_peered":"2024-04-25T01:13:31.521906+0000","last_unstale":"2024-04-25T01:13:32.668063+0000","last_undegraded":"2024-04-25T01:13:32.668063+0000","last_fullsized":"2024-04-25T01:13:32.668063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:58.761284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.785685+0000","last_change":"2024-04-25T01:13:31.514766+0000","last_active":"2024-04-25T01:13:33.785685+0000","last_peered":"2024-04-25T01:13:33.785685+0000","last_clean":"2024-04-25T01:13:33.785685+0000","last_became_active":"2024-04-25T01:13:31.514391+0000","last_became_peered":"2024-04-25T01:13:31.514391+0000","last_unstale":"2024-04-25T01:13:33.785685+0000","last_undegraded":"2024-04-25T01:13:33.785685+0000","last_fullsized":"2024-04-25T01:13:33.785685+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:07.398446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784305+0000","last_change":"2024-04-25T01:13:31.518976+0000","last_active":"2024-04-25T01:13:33.784305+0000","last_peered":"2024-04-25T01:13:33.784305+0000","last_clean":"2024-04-25T01:13:33.784305+0000","last_became_active":"2024-04-25T01:13:31.518566+0000","last_became_peered":"2024-04-25T01:13:31.518566+0000","last_unstale":"2024-04-25T01:13:33.784305+0000","last_undegraded":"2024-04-25T01:13:33.784305+0000","last_fullsized":"2024-04-25T01:13:33.784305+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:59.477996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.787769+0000","last_change":"2024-04-25T01:13:31.532880+0000","last_active":"2024-04-25T01:13:33.787769+0000","last_peered":"2024-04-25T01:13:33.787769+0000","last_clean":"2024-04-25T01:13:33.787769+0000","last_became_active":"2024-04-25T01:13:31.531470+0000","last_became_peered":"2024-04-25T01:13:31.531470+0000","last_unstale":"2024-04-25T01:13:33.787769+0000","last_undegraded":"2024-04-25T01:13:33.787769+0000","last_fullsized":"2024-04-25T01:13:33.787769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:55.431431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.785850+0000","last_change":"2024-04-25T01:13:31.532472+0000","last_active":"2024-04-25T01:13:33.785850+0000","last_peered":"2024-04-25T01:13:33.785850+0000","last_clean":"2024-04-25T01:13:33.785850+0000","last_became_active":"2024-04-25T01:13:31.530850+0000","last_became_peered":"2024-04-25T01:13:31.530850+0000","last_unstale":"2024-04-25T01:13:33.785850+0000","last_undegraded":"2024-04-25T01:13:33.785850+0000","last_fullsized":"2024-04-25T01:13:33.785850+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:50:26.890858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.785818+0000","last_change":"2024-04-25T01:13:31.539930+0000","last_active":"2024-04-25T01:13:33.785818+0000","last_peered":"2024-04-25T01:13:33.785818+0000","last_clean":"2024-04-25T01:13:33.785818+0000","last_became_active":"2024-04-25T01:13:31.539523+0000","last_became_peered":"2024-04-25T01:13:31.539523+0000","last_unstale":"2024-04-25T01:13:33.785818+0000","last_undegraded":"2024-04-25T01:13:33.785818+0000","last_fullsized":"2024-04-25T01:13:33.785818+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:07.983893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.494913+0000","last_change":"2024-04-25T01:13:31.536437+0000","last_active":"2024-04-25T01:13:32.494913+0000","last_peered":"2024-04-25T01:13:32.494913+0000","last_clean":"2024-04-25T01:13:32.494913+0000","last_became_active":"2024-04-25T01:13:31.536066+0000","last_became_peered":"2024-04-25T01:13:31.536066+0000","last_unstale":"2024-04-25T01:13:32.494913+0000","last_undegraded":"2024-04-25T01:13:32.494913+0000","last_fullsized":"2024-04-25T01:13:32.494913+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:50.986971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.642681+0000","last_change":"2024-04-25T01:13:31.536706+0000","last_active":"2024-04-25T01:13:32.642681+0000","last_peered":"2024-04-25T01:13:32.642681+0000","last_clean":"2024-04-25T01:13:32.642681+0000","last_became_active":"2024-04-25T01:13:31.536318+0000","last_became_peered":"2024-04-25T01:13:31.536318+0000","last_unstale":"2024-04-25T01:13:32.642681+0000","last_undegraded":"2024-04-25T01:13:32.642681+0000","last_fullsized":"2024-04-25T01:13:32.642681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:35.526515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.560216+0000","last_change":"2024-04-25T01:13:31.524462+0000","last_active":"2024-04-25T01:13:32.560216+0000","last_peered":"2024-04-25T01:13:32.560216+0000","last_clean":"2024-04-25T01:13:32.560216+0000","last_became_active":"2024-04-25T01:13:31.524018+0000","last_became_peered":"2024-04-25T01:13:31.524018+0000","last_unstale":"2024-04-25T01:13:32.560216+0000","last_undegraded":"2024-04-25T01:13:32.560216+0000","last_fullsized":"2024-04-25T01:13:32.560216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:43.767053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784645+0000","last_change":"2024-04-25T01:13:31.531553+0000","last_active":"2024-04-25T01:13:33.784645+0000","last_peered":"2024-04-25T01:13:33.784645+0000","last_clean":"2024-04-25T01:13:33.784645+0000","last_became_active":"2024-04-25T01:13:31.531237+0000","last_became_peered":"2024-04-25T01:13:31.531237+0000","last_unstale":"2024-04-25T01:13:33.784645+0000","last_undegraded":"2024-04-25T01:13:33.784645+0000","last_fullsized":"2024-04-25T01:13:33.784645+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:41.839370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:35.560238+0000","last_change":"2024-04-25T01:13:30.516847+0000","last_active":"2024-04-25T01:13:35.560238+0000","last_peered":"2024-04-25T01:13:35.560238+0000","last_clean":"2024-04-25T01:13:35.560238+0000","last_became_active":"2024-04-25T01:13:30.516500+0000","last_became_peered":"2024-04-25T01:13:30.516500+0000","last_unstale":"2024-04-25T01:13:35.560238+0000","last_undegraded":"2024-04-25T01:13:35.560238+0000","last_fullsized":"2024-04-25T01:13:35.560238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:37.703075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.497385+0000","last_change":"2024-04-25T01:13:31.518822+0000","last_active":"2024-04-25T01:13:32.497385+0000","last_peered":"2024-04-25T01:13:32.497385+0000","last_clean":"2024-04-25T01:13:32.497385+0000","last_became_active":"2024-04-25T01:13:31.516726+0000","last_became_peered":"2024-04-25T01:13:31.516726+0000","last_unstale":"2024-04-25T01:13:32.497385+0000","last_undegraded":"2024-04-25T01:13:32.497385+0000","last_fullsized":"2024-04-25T01:13:32.497385+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:02.360104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.497986+0000","last_change":"2024-04-25T01:13:30.531875+0000","last_active":"2024-04-25T01:13:32.497986+0000","last_peered":"2024-04-25T01:13:32.497986+0000","last_clean":"2024-04-25T01:13:32.497986+0000","last_became_active":"2024-04-25T01:13:30.531532+0000","last_became_peered":"2024-04-25T01:13:30.531532+0000","last_unstale":"2024-04-25T01:13:32.497986+0000","last_undegraded":"2024-04-25T01:13:32.497986+0000","last_fullsized":"2024-04-25T01:13:32.497986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:30.820946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.788544+0000","last_change":"2024-04-25T01:13:31.534211+0000","last_active":"2024-04-25T01:13:33.788544+0000","last_peered":"2024-04-25T01:13:33.788544+0000","last_clean":"2024-04-25T01:13:33.788544+0000","last_became_active":"2024-04-25T01:13:31.533828+0000","last_became_peered":"2024-04-25T01:13:31.533828+0000","last_unstale":"2024-04-25T01:13:33.788544+0000","last_undegraded":"2024-04-25T01:13:33.788544+0000","last_fullsized":"2024-04-25T01:13:33.788544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:39.488524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:35.547492+0000","last_change":"2024-04-25T01:13:30.517922+0000","last_active":"2024-04-25T01:13:35.547492+0000","last_peered":"2024-04-25T01:13:35.547492+0000","last_clean":"2024-04-25T01:13:35.547492+0000","last_became_active":"2024-04-25T01:13:30.517571+0000","last_became_peered":"2024-04-25T01:13:30.517571+0000","last_unstale":"2024-04-25T01:13:35.547492+0000","last_undegraded":"2024-04-25T01:13:35.547492+0000","last_fullsized":"2024-04-25T01:13:35.547492+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:16:34.901210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.495220+0000","last_change":"2024-04-25T01:13:31.530460+0000","last_active":"2024-04-25T01:13:32.495220+0000","last_peered":"2024-04-25T01:13:32.495220+0000","last_clean":"2024-04-25T01:13:32.495220+0000","last_became_active":"2024-04-25T01:13:31.530084+0000","last_became_peered":"2024-04-25T01:13:31.530084+0000","last_unstale":"2024-04-25T01:13:32.495220+0000","last_undegraded":"2024-04-25T01:13:32.495220+0000","last_fullsized":"2024-04-25T01:13:32.495220+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:36.810301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.495042+0000","last_change":"2024-04-25T01:13:30.514320+0000","last_active":"2024-04-25T01:13:32.495042+0000","last_peered":"2024-04-25T01:13:32.495042+0000","last_clean":"2024-04-25T01:13:32.495042+0000","last_became_active":"2024-04-25T01:13:30.513752+0000","last_became_peered":"2024-04-25T01:13:30.513752+0000","last_unstale":"2024-04-25T01:13:32.495042+0000","last_undegraded":"2024-04-25T01:13:32.495042+0000","last_fullsized":"2024-04-25T01:13:32.495042+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:19.814454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784661+0000","last_change":"2024-04-25T01:13:31.514516+0000","last_active":"2024-04-25T01:13:33.784661+0000","last_peered":"2024-04-25T01:13:33.784661+0000","last_clean":"2024-04-25T01:13:33.784661+0000","last_became_active":"2024-04-25T01:13:31.514208+0000","last_became_peered":"2024-04-25T01:13:31.514208+0000","last_unstale":"2024-04-25T01:13:33.784661+0000","last_undegraded":"2024-04-25T01:13:33.784661+0000","last_fullsized":"2024-04-25T01:13:33.784661+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:10:11.204770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:37.538570+0000","last_change":"2024-04-25T01:13:30.519701+0000","last_active":"2024-04-25T01:13:37.538570+0000","last_peered":"2024-04-25T01:13:37.538570+0000","last_clean":"2024-04-25T01:13:37.538570+0000","last_became_active":"2024-04-25T01:13:30.519315+0000","last_became_peered":"2024-04-25T01:13:30.519315+0000","last_unstale":"2024-04-25T01:13:37.538570+0000","last_undegraded":"2024-04-25T01:13:37.538570+0000","last_fullsized":"2024-04-25T01:13:37.538570+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:34.613473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.495007+0000","last_change":"2024-04-25T01:13:31.528376+0000","last_active":"2024-04-25T01:13:32.495007+0000","last_peered":"2024-04-25T01:13:32.495007+0000","last_clean":"2024-04-25T01:13:32.495007+0000","last_became_active":"2024-04-25T01:13:31.527622+0000","last_became_peered":"2024-04-25T01:13:31.527622+0000","last_unstale":"2024-04-25T01:13:32.495007+0000","last_undegraded":"2024-04-25T01:13:32.495007+0000","last_fullsized":"2024-04-25T01:13:32.495007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:17.901898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:35.558911+0000","last_change":"2024-04-25T01:13:30.514798+0000","last_active":"2024-04-25T01:13:35.558911+0000","last_peered":"2024-04-25T01:13:35.558911+0000","last_clean":"2024-04-25T01:13:35.558911+0000","last_became_active":"2024-04-25T01:13:30.514422+0000","last_became_peered":"2024-04-25T01:13:30.514422+0000","last_unstale":"2024-04-25T01:13:35.558911+0000","last_undegraded":"2024-04-25T01:13:35.558911+0000","last_fullsized":"2024-04-25T01:13:35.558911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:27.584578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784646+0000","last_change":"2024-04-25T01:13:31.510479+0000","last_active":"2024-04-25T01:13:33.784646+0000","last_peered":"2024-04-25T01:13:33.784646+0000","last_clean":"2024-04-25T01:13:33.784646+0000","last_became_active":"2024-04-25T01:13:31.509166+0000","last_became_peered":"2024-04-25T01:13:31.509166+0000","last_unstale":"2024-04-25T01:13:33.784646+0000","last_undegraded":"2024-04-25T01:13:33.784646+0000","last_fullsized":"2024-04-25T01:13:33.784646+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:42.604266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:35.562022+0000","last_change":"2024-04-25T01:13:30.520435+0000","last_active":"2024-04-25T01:13:35.562022+0000","last_peered":"2024-04-25T01:13:35.562022+0000","last_clean":"2024-04-25T01:13:35.562022+0000","last_became_active":"2024-04-25T01:13:30.520042+0000","last_became_peered":"2024-04-25T01:13:30.520042+0000","last_unstale":"2024-04-25T01:13:35.562022+0000","last_undegraded":"2024-04-25T01:13:35.562022+0000","last_fullsized":"2024-04-25T01:13:35.562022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:37.818149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.497331+0000","last_change":"2024-04-25T01:13:31.524799+0000","last_active":"2024-04-25T01:13:32.497331+0000","last_peered":"2024-04-25T01:13:32.497331+0000","last_clean":"2024-04-25T01:13:32.497331+0000","last_became_active":"2024-04-25T01:13:31.524454+0000","last_became_peered":"2024-04-25T01:13:31.524454+0000","last_unstale":"2024-04-25T01:13:32.497331+0000","last_undegraded":"2024-04-25T01:13:32.497331+0000","last_fullsized":"2024-04-25T01:13:32.497331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:15.930922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:34.351397+0000","last_change":"2024-04-25T01:13:30.514104+0000","last_active":"2024-04-25T01:13:34.351397+0000","last_peered":"2024-04-25T01:13:34.351397+0000","last_clean":"2024-04-25T01:13:34.351397+0000","last_became_active":"2024-04-25T01:13:30.513742+0000","last_became_peered":"2024-04-25T01:13:30.513742+0000","last_unstale":"2024-04-25T01:13:34.351397+0000","last_undegraded":"2024-04-25T01:13:34.351397+0000","last_fullsized":"2024-04-25T01:13:34.351397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:42.189997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.559329+0000","last_change":"2024-04-25T01:13:31.528223+0000","last_active":"2024-04-25T01:13:32.559329+0000","last_peered":"2024-04-25T01:13:32.559329+0000","last_clean":"2024-04-25T01:13:32.559329+0000","last_became_active":"2024-04-25T01:13:31.526906+0000","last_became_peered":"2024-04-25T01:13:31.526906+0000","last_unstale":"2024-04-25T01:13:32.559329+0000","last_undegraded":"2024-04-25T01:13:32.559329+0000","last_fullsized":"2024-04-25T01:13:32.559329+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:12:16.276974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:37.537727+0000","last_change":"2024-04-25T01:13:30.519174+0000","last_active":"2024-04-25T01:13:37.537727+0000","last_peered":"2024-04-25T01:13:37.537727+0000","last_clean":"2024-04-25T01:13:37.537727+0000","last_became_active":"2024-04-25T01:13:30.518792+0000","last_became_peered":"2024-04-25T01:13:30.518792+0000","last_unstale":"2024-04-25T01:13:37.537727+0000","last_undegraded":"2024-04-25T01:13:37.537727+0000","last_fullsized":"2024-04-25T01:13:37.537727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:27.746435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.495454+0000","last_change":"2024-04-25T01:13:31.521482+0000","last_active":"2024-04-25T01:13:32.495454+0000","last_peered":"2024-04-25T01:13:32.495454+0000","last_clean":"2024-04-25T01:13:32.495454+0000","last_became_active":"2024-04-25T01:13:31.521114+0000","last_became_peered":"2024-04-25T01:13:31.521114+0000","last_unstale":"2024-04-25T01:13:32.495454+0000","last_undegraded":"2024-04-25T01:13:32.495454+0000","last_fullsized":"2024-04-25T01:13:32.495454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:48.748691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:34.591831+0000","last_change":"2024-04-25T01:13:30.518427+0000","last_active":"2024-04-25T01:13:34.591831+0000","last_peered":"2024-04-25T01:13:34.591831+0000","last_clean":"2024-04-25T01:13:34.591831+0000","last_became_active":"2024-04-25T01:13:30.518005+0000","last_became_peered":"2024-04-25T01:13:30.518005+0000","last_unstale":"2024-04-25T01:13:34.591831+0000","last_undegraded":"2024-04-25T01:13:34.591831+0000","last_fullsized":"2024-04-25T01:13:34.591831+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:29:21.087411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.788126+0000","last_change":"2024-04-25T01:13:31.532598+0000","last_active":"2024-04-25T01:13:33.788126+0000","last_peered":"2024-04-25T01:13:33.788126+0000","last_clean":"2024-04-25T01:13:33.788126+0000","last_became_active":"2024-04-25T01:13:31.530468+0000","last_became_peered":"2024-04-25T01:13:31.530468+0000","last_unstale":"2024-04-25T01:13:33.788126+0000","last_undegraded":"2024-04-25T01:13:33.788126+0000","last_fullsized":"2024-04-25T01:13:33.788126+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:28.254924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:34.593481+0000","last_change":"2024-04-25T01:13:30.520507+0000","last_active":"2024-04-25T01:13:34.593481+0000","last_peered":"2024-04-25T01:13:34.593481+0000","last_clean":"2024-04-25T01:13:34.593481+0000","last_became_active":"2024-04-25T01:13:30.520165+0000","last_became_peered":"2024-04-25T01:13:30.520165+0000","last_unstale":"2024-04-25T01:13:34.593481+0000","last_undegraded":"2024-04-25T01:13:34.593481+0000","last_fullsized":"2024-04-25T01:13:34.593481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:26.173090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.495294+0000","last_change":"2024-04-25T01:13:31.517021+0000","last_active":"2024-04-25T01:13:32.495294+0000","last_peered":"2024-04-25T01:13:32.495294+0000","last_clean":"2024-04-25T01:13:32.495294+0000","last_became_active":"2024-04-25T01:13:31.516676+0000","last_became_peered":"2024-04-25T01:13:31.516676+0000","last_unstale":"2024-04-25T01:13:32.495294+0000","last_undegraded":"2024-04-25T01:13:32.495294+0000","last_fullsized":"2024-04-25T01:13:32.495294+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:25.314544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'19","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:38.577943+0000","last_change":"2024-04-25T01:13:30.511963+0000","last_active":"2024-04-25T01:13:38.577943+0000","last_peered":"2024-04-25T01:13:38.577943+0000","last_clean":"2024-04-25T01:13:38.577943+0000","last_became_active":"2024-04-25T01:13:30.511142+0000","last_became_peered":"2024-04-25T01:13:30.511142+0000","last_unstale":"2024-04-25T01:13:38.577943+0000","last_undegraded":"2024-04-25T01:13:38.577943+0000","last_fullsized":"2024-04-25T01:13:38.577943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:53.545598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38193,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":12,"num_read_kb":26,"num_write":20,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.495404+0000","last_change":"2024-04-25T01:13:31.516236+0000","last_active":"2024-04-25T01:13:32.495404+0000","last_peered":"2024-04-25T01:13:32.495404+0000","last_clean":"2024-04-25T01:13:32.495404+0000","last_became_active":"2024-04-25T01:13:31.515871+0000","last_became_peered":"2024-04-25T01:13:31.515871+0000","last_unstale":"2024-04-25T01:13:32.495404+0000","last_undegraded":"2024-04-25T01:13:32.495404+0000","last_fullsized":"2024-04-25T01:13:32.495404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:24.464246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:35.560766+0000","last_change":"2024-04-25T01:13:30.513070+0000","last_active":"2024-04-25T01:13:35.560766+0000","last_peered":"2024-04-25T01:13:35.560766+0000","last_clean":"2024-04-25T01:13:35.560766+0000","last_became_active":"2024-04-25T01:13:30.512701+0000","last_became_peered":"2024-04-25T01:13:30.512701+0000","last_unstale":"2024-04-25T01:13:35.560766+0000","last_undegraded":"2024-04-25T01:13:35.560766+0000","last_fullsized":"2024-04-25T01:13:35.560766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:33.482272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.787225+0000","last_change":"2024-04-25T01:13:31.538313+0000","last_active":"2024-04-25T01:13:33.787225+0000","last_peered":"2024-04-25T01:13:33.787225+0000","last_clean":"2024-04-25T01:13:33.787225+0000","last_became_active":"2024-04-25T01:13:31.537907+0000","last_became_peered":"2024-04-25T01:13:31.537907+0000","last_unstale":"2024-04-25T01:13:33.787225+0000","last_undegraded":"2024-04-25T01:13:33.787225+0000","last_fullsized":"2024-04-25T01:13:33.787225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:19.484311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:34.596792+0000","last_change":"2024-04-25T01:13:30.520170+0000","last_active":"2024-04-25T01:13:34.596792+0000","last_peered":"2024-04-25T01:13:34.596792+0000","last_clean":"2024-04-25T01:13:34.596792+0000","last_became_active":"2024-04-25T01:13:30.519636+0000","last_became_peered":"2024-04-25T01:13:30.519636+0000","last_unstale":"2024-04-25T01:13:34.596792+0000","last_undegraded":"2024-04-25T01:13:34.596792+0000","last_fullsized":"2024-04-25T01:13:34.596792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:00:32.670127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.785623+0000","last_change":"2024-04-25T01:13:31.537630+0000","last_active":"2024-04-25T01:13:33.785623+0000","last_peered":"2024-04-25T01:13:33.785623+0000","last_clean":"2024-04-25T01:13:33.785623+0000","last_became_active":"2024-04-25T01:13:31.537164+0000","last_became_peered":"2024-04-25T01:13:31.537164+0000","last_unstale":"2024-04-25T01:13:33.785623+0000","last_undegraded":"2024-04-25T01:13:33.785623+0000","last_fullsized":"2024-04-25T01:13:33.785623+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:04.091194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:36.555909+0000","last_change":"2024-04-25T01:13:30.514322+0000","last_active":"2024-04-25T01:13:36.555909+0000","last_peered":"2024-04-25T01:13:36.555909+0000","last_clean":"2024-04-25T01:13:36.555909+0000","last_became_active":"2024-04-25T01:13:30.513990+0000","last_became_peered":"2024-04-25T01:13:30.513990+0000","last_unstale":"2024-04-25T01:13:36.555909+0000","last_undegraded":"2024-04-25T01:13:36.555909+0000","last_fullsized":"2024-04-25T01:13:36.555909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:59.995489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784873+0000","last_change":"2024-04-25T01:13:31.519730+0000","last_active":"2024-04-25T01:13:33.784873+0000","last_peered":"2024-04-25T01:13:33.784873+0000","last_clean":"2024-04-25T01:13:33.784873+0000","last_became_active":"2024-04-25T01:13:31.519363+0000","last_became_peered":"2024-04-25T01:13:31.519363+0000","last_unstale":"2024-04-25T01:13:33.784873+0000","last_undegraded":"2024-04-25T01:13:33.784873+0000","last_fullsized":"2024-04-25T01:13:33.784873+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:26.400754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:35.560473+0000","last_change":"2024-04-25T01:13:30.517220+0000","last_active":"2024-04-25T01:13:35.560473+0000","last_peered":"2024-04-25T01:13:35.560473+0000","last_clean":"2024-04-25T01:13:35.560473+0000","last_became_active":"2024-04-25T01:13:30.516825+0000","last_became_peered":"2024-04-25T01:13:30.516825+0000","last_unstale":"2024-04-25T01:13:35.560473+0000","last_undegraded":"2024-04-25T01:13:35.560473+0000","last_fullsized":"2024-04-25T01:13:35.560473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:04.452894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.783919+0000","last_change":"2024-04-25T01:13:31.517424+0000","last_active":"2024-04-25T01:13:33.783919+0000","last_peered":"2024-04-25T01:13:33.783919+0000","last_clean":"2024-04-25T01:13:33.783919+0000","last_became_active":"2024-04-25T01:13:31.517017+0000","last_became_peered":"2024-04-25T01:13:31.517017+0000","last_unstale":"2024-04-25T01:13:33.783919+0000","last_undegraded":"2024-04-25T01:13:33.783919+0000","last_fullsized":"2024-04-25T01:13:33.783919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:16.516450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:35.532404+0000","last_change":"2024-04-25T01:13:30.515249+0000","last_active":"2024-04-25T01:13:35.532404+0000","last_peered":"2024-04-25T01:13:35.532404+0000","last_clean":"2024-04-25T01:13:35.532404+0000","last_became_active":"2024-04-25T01:13:30.514819+0000","last_became_peered":"2024-04-25T01:13:30.514819+0000","last_unstale":"2024-04-25T01:13:35.532404+0000","last_undegraded":"2024-04-25T01:13:35.532404+0000","last_fullsized":"2024-04-25T01:13:35.532404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:52.503564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.642502+0000","last_change":"2024-04-25T01:13:31.519143+0000","last_active":"2024-04-25T01:13:32.642502+0000","last_peered":"2024-04-25T01:13:32.642502+0000","last_clean":"2024-04-25T01:13:32.642502+0000","last_became_active":"2024-04-25T01:13:31.518755+0000","last_became_peered":"2024-04-25T01:13:31.518755+0000","last_unstale":"2024-04-25T01:13:32.642502+0000","last_undegraded":"2024-04-25T01:13:32.642502+0000","last_fullsized":"2024-04-25T01:13:32.642502+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:02.870725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.559759+0000","last_change":"2024-04-25T01:13:30.522489+0000","last_active":"2024-04-25T01:13:32.559759+0000","last_peered":"2024-04-25T01:13:32.559759+0000","last_clean":"2024-04-25T01:13:32.559759+0000","last_became_active":"2024-04-25T01:13:30.522128+0000","last_became_peered":"2024-04-25T01:13:30.522128+0000","last_unstale":"2024-04-25T01:13:32.559759+0000","last_undegraded":"2024-04-25T01:13:32.559759+0000","last_fullsized":"2024-04-25T01:13:32.559759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:12.875627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.872438+0000","last_change":"2024-04-25T01:13:31.537536+0000","last_active":"2024-04-25T01:13:33.872438+0000","last_peered":"2024-04-25T01:13:33.872438+0000","last_clean":"2024-04-25T01:13:33.872438+0000","last_became_active":"2024-04-25T01:13:31.537169+0000","last_became_peered":"2024-04-25T01:13:31.537169+0000","last_unstale":"2024-04-25T01:13:33.872438+0000","last_undegraded":"2024-04-25T01:13:33.872438+0000","last_fullsized":"2024-04-25T01:13:33.872438+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:55.147719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784991+0000","last_change":"2024-04-25T01:13:30.523870+0000","last_active":"2024-04-25T01:13:33.784991+0000","last_peered":"2024-04-25T01:13:33.784991+0000","last_clean":"2024-04-25T01:13:33.784991+0000","last_became_active":"2024-04-25T01:13:30.523504+0000","last_became_peered":"2024-04-25T01:13:30.523504+0000","last_unstale":"2024-04-25T01:13:33.784991+0000","last_undegraded":"2024-04-25T01:13:33.784991+0000","last_fullsized":"2024-04-25T01:13:33.784991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:42.498898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.667722+0000","last_change":"2024-04-25T01:13:31.539691+0000","last_active":"2024-04-25T01:13:32.667722+0000","last_peered":"2024-04-25T01:13:32.667722+0000","last_clean":"2024-04-25T01:13:32.667722+0000","last_became_active":"2024-04-25T01:13:31.539287+0000","last_became_peered":"2024-04-25T01:13:31.539287+0000","last_unstale":"2024-04-25T01:13:32.667722+0000","last_undegraded":"2024-04-25T01:13:32.667722+0000","last_fullsized":"2024-04-25T01:13:32.667722+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:47.985167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784050+0000","last_change":"2024-04-25T01:13:31.513438+0000","last_active":"2024-04-25T01:13:33.784050+0000","last_peered":"2024-04-25T01:13:33.784050+0000","last_clean":"2024-04-25T01:13:33.784050+0000","last_became_active":"2024-04-25T01:13:31.513087+0000","last_became_peered":"2024-04-25T01:13:31.513087+0000","last_unstale":"2024-04-25T01:13:33.784050+0000","last_undegraded":"2024-04-25T01:13:33.784050+0000","last_fullsized":"2024-04-25T01:13:33.784050+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:12.999714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:35.561740+0000","last_change":"2024-04-25T01:13:30.527528+0000","last_active":"2024-04-25T01:13:35.561740+0000","last_peered":"2024-04-25T01:13:35.561740+0000","last_clean":"2024-04-25T01:13:35.561740+0000","last_became_active":"2024-04-25T01:13:30.526040+0000","last_became_peered":"2024-04-25T01:13:30.526040+0000","last_unstale":"2024-04-25T01:13:35.561740+0000","last_undegraded":"2024-04-25T01:13:35.561740+0000","last_fullsized":"2024-04-25T01:13:35.561740+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:31.943727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.497478+0000","last_change":"2024-04-25T01:13:31.523311+0000","last_active":"2024-04-25T01:13:32.497478+0000","last_peered":"2024-04-25T01:13:32.497478+0000","last_clean":"2024-04-25T01:13:32.497478+0000","last_became_active":"2024-04-25T01:13:31.518099+0000","last_became_peered":"2024-04-25T01:13:31.518099+0000","last_unstale":"2024-04-25T01:13:32.497478+0000","last_undegraded":"2024-04-25T01:13:32.497478+0000","last_fullsized":"2024-04-25T01:13:32.497478+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:02.947202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:35.553491+0000","last_change":"2024-04-25T01:13:30.512804+0000","last_active":"2024-04-25T01:13:35.553491+0000","last_peered":"2024-04-25T01:13:35.553491+0000","last_clean":"2024-04-25T01:13:35.553491+0000","last_became_active":"2024-04-25T01:13:30.512406+0000","last_became_peered":"2024-04-25T01:13:30.512406+0000","last_unstale":"2024-04-25T01:13:35.553491+0000","last_undegraded":"2024-04-25T01:13:35.553491+0000","last_fullsized":"2024-04-25T01:13:35.553491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:00.042244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":340,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784803+0000","last_change":"2024-04-25T01:12:07.906198+0000","last_active":"2024-04-25T01:13:33.784803+0000","last_peered":"2024-04-25T01:13:33.784803+0000","last_clean":"2024-04-25T01:13:33.784803+0000","last_became_active":"2024-04-25T01:12:07.905724+0000","last_became_peered":"2024-04-25T01:12:07.905724+0000","last_unstale":"2024-04-25T01:13:33.784803+0000","last_undegraded":"2024-04-25T01:13:33.784803+0000","last_fullsized":"2024-04-25T01:13:33.784803+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:18.712681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:18.712681+0000","last_clean_scrub_stamp":"2024-04-25T01:08:18.712681+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:13.899974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.785543+0000","last_change":"2024-04-25T01:13:31.534664+0000","last_active":"2024-04-25T01:13:33.785543+0000","last_peered":"2024-04-25T01:13:33.785543+0000","last_clean":"2024-04-25T01:13:33.785543+0000","last_became_active":"2024-04-25T01:13:31.530082+0000","last_became_peered":"2024-04-25T01:13:31.530082+0000","last_unstale":"2024-04-25T01:13:33.785543+0000","last_undegraded":"2024-04-25T01:13:33.785543+0000","last_fullsized":"2024-04-25T01:13:33.785543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:08:32.717997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.785037+0000","last_change":"2024-04-25T01:13:30.514295+0000","last_active":"2024-04-25T01:13:33.785037+0000","last_peered":"2024-04-25T01:13:33.785037+0000","last_clean":"2024-04-25T01:13:33.785037+0000","last_became_active":"2024-04-25T01:13:30.513844+0000","last_became_peered":"2024-04-25T01:13:30.513844+0000","last_unstale":"2024-04-25T01:13:33.785037+0000","last_undegraded":"2024-04-25T01:13:33.785037+0000","last_fullsized":"2024-04-25T01:13:33.785037+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:36.652771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:34.597140+0000","last_change":"2024-04-25T01:13:30.523088+0000","last_active":"2024-04-25T01:13:34.597140+0000","last_peered":"2024-04-25T01:13:34.597140+0000","last_clean":"2024-04-25T01:13:34.597140+0000","last_became_active":"2024-04-25T01:13:30.522642+0000","last_became_peered":"2024-04-25T01:13:30.522642+0000","last_unstale":"2024-04-25T01:13:34.597140+0000","last_undegraded":"2024-04-25T01:13:34.597140+0000","last_fullsized":"2024-04-25T01:13:34.597140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:55.501983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.787889+0000","last_change":"2024-04-25T01:13:31.531703+0000","last_active":"2024-04-25T01:13:33.787889+0000","last_peered":"2024-04-25T01:13:33.787889+0000","last_clean":"2024-04-25T01:13:33.787889+0000","last_became_active":"2024-04-25T01:13:31.531160+0000","last_became_peered":"2024-04-25T01:13:31.531160+0000","last_unstale":"2024-04-25T01:13:33.787889+0000","last_undegraded":"2024-04-25T01:13:33.787889+0000","last_fullsized":"2024-04-25T01:13:33.787889+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:21.993998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:35.562148+0000","last_change":"2024-04-25T01:13:30.517908+0000","last_active":"2024-04-25T01:13:35.562148+0000","last_peered":"2024-04-25T01:13:35.562148+0000","last_clean":"2024-04-25T01:13:35.562148+0000","last_became_active":"2024-04-25T01:13:30.517559+0000","last_became_peered":"2024-04-25T01:13:30.517559+0000","last_unstale":"2024-04-25T01:13:35.562148+0000","last_undegraded":"2024-04-25T01:13:35.562148+0000","last_fullsized":"2024-04-25T01:13:35.562148+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:44.357328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.497950+0000","last_change":"2024-04-25T01:13:31.515760+0000","last_active":"2024-04-25T01:13:32.497950+0000","last_peered":"2024-04-25T01:13:32.497950+0000","last_clean":"2024-04-25T01:13:32.497950+0000","last_became_active":"2024-04-25T01:13:31.514957+0000","last_became_peered":"2024-04-25T01:13:31.514957+0000","last_unstale":"2024-04-25T01:13:32.497950+0000","last_undegraded":"2024-04-25T01:13:32.497950+0000","last_fullsized":"2024-04-25T01:13:32.497950+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:46.169171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:34.592541+0000","last_change":"2024-04-25T01:13:30.523933+0000","last_active":"2024-04-25T01:13:34.592541+0000","last_peered":"2024-04-25T01:13:34.592541+0000","last_clean":"2024-04-25T01:13:34.592541+0000","last_became_active":"2024-04-25T01:13:30.523497+0000","last_became_peered":"2024-04-25T01:13:30.523497+0000","last_unstale":"2024-04-25T01:13:34.592541+0000","last_undegraded":"2024-04-25T01:13:34.592541+0000","last_fullsized":"2024-04-25T01:13:34.592541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:17:59.336812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.787296+0000","last_change":"2024-04-25T01:13:31.540188+0000","last_active":"2024-04-25T01:13:33.787296+0000","last_peered":"2024-04-25T01:13:33.787296+0000","last_clean":"2024-04-25T01:13:33.787296+0000","last_became_active":"2024-04-25T01:13:31.537121+0000","last_became_peered":"2024-04-25T01:13:31.537121+0000","last_unstale":"2024-04-25T01:13:33.787296+0000","last_undegraded":"2024-04-25T01:13:33.787296+0000","last_fullsized":"2024-04-25T01:13:33.787296+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:57.162639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:34.597355+0000","last_change":"2024-04-25T01:13:30.516975+0000","last_active":"2024-04-25T01:13:34.597355+0000","last_peered":"2024-04-25T01:13:34.597355+0000","last_clean":"2024-04-25T01:13:34.597355+0000","last_became_active":"2024-04-25T01:13:30.516391+0000","last_became_peered":"2024-04-25T01:13:30.516391+0000","last_unstale":"2024-04-25T01:13:34.597355+0000","last_undegraded":"2024-04-25T01:13:34.597355+0000","last_fullsized":"2024-04-25T01:13:34.597355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:57:53.486835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.494819+0000","last_change":"2024-04-25T01:13:31.514491+0000","last_active":"2024-04-25T01:13:32.494819+0000","last_peered":"2024-04-25T01:13:32.494819+0000","last_clean":"2024-04-25T01:13:32.494819+0000","last_became_active":"2024-04-25T01:13:31.514156+0000","last_became_peered":"2024-04-25T01:13:31.514156+0000","last_unstale":"2024-04-25T01:13:32.494819+0000","last_undegraded":"2024-04-25T01:13:32.494819+0000","last_fullsized":"2024-04-25T01:13:32.494819+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:18.588291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:34.598150+0000","last_change":"2024-04-25T01:13:30.521258+0000","last_active":"2024-04-25T01:13:34.598150+0000","last_peered":"2024-04-25T01:13:34.598150+0000","last_clean":"2024-04-25T01:13:34.598150+0000","last_became_active":"2024-04-25T01:13:30.518359+0000","last_became_peered":"2024-04-25T01:13:30.518359+0000","last_unstale":"2024-04-25T01:13:34.598150+0000","last_undegraded":"2024-04-25T01:13:34.598150+0000","last_fullsized":"2024-04-25T01:13:34.598150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:35.790798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.642277+0000","last_change":"2024-04-25T01:13:31.536755+0000","last_active":"2024-04-25T01:13:32.642277+0000","last_peered":"2024-04-25T01:13:32.642277+0000","last_clean":"2024-04-25T01:13:32.642277+0000","last_became_active":"2024-04-25T01:13:31.536303+0000","last_became_peered":"2024-04-25T01:13:31.536303+0000","last_unstale":"2024-04-25T01:13:32.642277+0000","last_undegraded":"2024-04-25T01:13:32.642277+0000","last_fullsized":"2024-04-25T01:13:32.642277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:44.793993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784439+0000","last_change":"2024-04-25T01:13:30.513635+0000","last_active":"2024-04-25T01:13:33.784439+0000","last_peered":"2024-04-25T01:13:33.784439+0000","last_clean":"2024-04-25T01:13:33.784439+0000","last_became_active":"2024-04-25T01:13:30.511902+0000","last_became_peered":"2024-04-25T01:13:30.511902+0000","last_unstale":"2024-04-25T01:13:33.784439+0000","last_undegraded":"2024-04-25T01:13:33.784439+0000","last_fullsized":"2024-04-25T01:13:33.784439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:52.506547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.497421+0000","last_change":"2024-04-25T01:13:31.543234+0000","last_active":"2024-04-25T01:13:32.497421+0000","last_peered":"2024-04-25T01:13:32.497421+0000","last_clean":"2024-04-25T01:13:32.497421+0000","last_became_active":"2024-04-25T01:13:31.542419+0000","last_became_peered":"2024-04-25T01:13:31.542419+0000","last_unstale":"2024-04-25T01:13:32.497421+0000","last_undegraded":"2024-04-25T01:13:32.497421+0000","last_fullsized":"2024-04-25T01:13:32.497421+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:57.226505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:34.592128+0000","last_change":"2024-04-25T01:13:30.516913+0000","last_active":"2024-04-25T01:13:34.592128+0000","last_peered":"2024-04-25T01:13:34.592128+0000","last_clean":"2024-04-25T01:13:34.592128+0000","last_became_active":"2024-04-25T01:13:30.516478+0000","last_became_peered":"2024-04-25T01:13:30.516478+0000","last_unstale":"2024-04-25T01:13:34.592128+0000","last_undegraded":"2024-04-25T01:13:34.592128+0000","last_fullsized":"2024-04-25T01:13:34.592128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:25.502072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.785184+0000","last_change":"2024-04-25T01:13:31.532194+0000","last_active":"2024-04-25T01:13:33.785184+0000","last_peered":"2024-04-25T01:13:33.785184+0000","last_clean":"2024-04-25T01:13:33.785184+0000","last_became_active":"2024-04-25T01:13:31.531798+0000","last_became_peered":"2024-04-25T01:13:31.531798+0000","last_unstale":"2024-04-25T01:13:33.785184+0000","last_undegraded":"2024-04-25T01:13:33.785184+0000","last_fullsized":"2024-04-25T01:13:33.785184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:36.872827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:35.561022+0000","last_change":"2024-04-25T01:13:30.522565+0000","last_active":"2024-04-25T01:13:35.561022+0000","last_peered":"2024-04-25T01:13:35.561022+0000","last_clean":"2024-04-25T01:13:35.561022+0000","last_became_active":"2024-04-25T01:13:30.522207+0000","last_became_peered":"2024-04-25T01:13:30.522207+0000","last_unstale":"2024-04-25T01:13:35.561022+0000","last_undegraded":"2024-04-25T01:13:35.561022+0000","last_fullsized":"2024-04-25T01:13:35.561022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:48.263665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.785683+0000","last_change":"2024-04-25T01:13:31.514330+0000","last_active":"2024-04-25T01:13:33.785683+0000","last_peered":"2024-04-25T01:13:33.785683+0000","last_clean":"2024-04-25T01:13:33.785683+0000","last_became_active":"2024-04-25T01:13:31.513948+0000","last_became_peered":"2024-04-25T01:13:31.513948+0000","last_unstale":"2024-04-25T01:13:33.785683+0000","last_undegraded":"2024-04-25T01:13:33.785683+0000","last_fullsized":"2024-04-25T01:13:33.785683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:57.544298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:34.351495+0000","last_change":"2024-04-25T01:13:30.515951+0000","last_active":"2024-04-25T01:13:34.351495+0000","last_peered":"2024-04-25T01:13:34.351495+0000","last_clean":"2024-04-25T01:13:34.351495+0000","last_became_active":"2024-04-25T01:13:30.515388+0000","last_became_peered":"2024-04-25T01:13:30.515388+0000","last_unstale":"2024-04-25T01:13:34.351495+0000","last_undegraded":"2024-04-25T01:13:34.351495+0000","last_fullsized":"2024-04-25T01:13:34.351495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:11.041466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.785615+0000","last_change":"2024-04-25T01:13:31.503386+0000","last_active":"2024-04-25T01:13:33.785615+0000","last_peered":"2024-04-25T01:13:33.785615+0000","last_clean":"2024-04-25T01:13:33.785615+0000","last_became_active":"2024-04-25T01:13:31.502966+0000","last_became_peered":"2024-04-25T01:13:31.502966+0000","last_unstale":"2024-04-25T01:13:33.785615+0000","last_undegraded":"2024-04-25T01:13:33.785615+0000","last_fullsized":"2024-04-25T01:13:33.785615+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:07.520811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.789001+0000","last_change":"2024-04-25T01:13:30.528033+0000","last_active":"2024-04-25T01:13:33.789001+0000","last_peered":"2024-04-25T01:13:33.789001+0000","last_clean":"2024-04-25T01:13:33.789001+0000","last_became_active":"2024-04-25T01:13:30.527609+0000","last_became_peered":"2024-04-25T01:13:30.527609+0000","last_unstale":"2024-04-25T01:13:33.789001+0000","last_undegraded":"2024-04-25T01:13:33.789001+0000","last_fullsized":"2024-04-25T01:13:33.789001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:08.182355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.785675+0000","last_change":"2024-04-25T01:13:31.516569+0000","last_active":"2024-04-25T01:13:33.785675+0000","last_peered":"2024-04-25T01:13:33.785675+0000","last_clean":"2024-04-25T01:13:33.785675+0000","last_became_active":"2024-04-25T01:13:31.515746+0000","last_became_peered":"2024-04-25T01:13:31.515746+0000","last_unstale":"2024-04-25T01:13:33.785675+0000","last_undegraded":"2024-04-25T01:13:33.785675+0000","last_fullsized":"2024-04-25T01:13:33.785675+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:27.860296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.642447+0000","last_change":"2024-04-25T01:13:31.518161+0000","last_active":"2024-04-25T01:13:32.642447+0000","last_peered":"2024-04-25T01:13:32.642447+0000","last_clean":"2024-04-25T01:13:32.642447+0000","last_became_active":"2024-04-25T01:13:31.517826+0000","last_became_peered":"2024-04-25T01:13:31.517826+0000","last_unstale":"2024-04-25T01:13:32.642447+0000","last_undegraded":"2024-04-25T01:13:32.642447+0000","last_fullsized":"2024-04-25T01:13:32.642447+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:42.211894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784875+0000","last_change":"2024-04-25T01:13:31.514663+0000","last_active":"2024-04-25T01:13:33.784875+0000","last_peered":"2024-04-25T01:13:33.784875+0000","last_clean":"2024-04-25T01:13:33.784875+0000","last_became_active":"2024-04-25T01:13:31.514246+0000","last_became_peered":"2024-04-25T01:13:31.514246+0000","last_unstale":"2024-04-25T01:13:33.784875+0000","last_undegraded":"2024-04-25T01:13:33.784875+0000","last_fullsized":"2024-04-25T01:13:33.784875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:27.235598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784264+0000","last_change":"2024-04-25T01:13:31.533374+0000","last_active":"2024-04-25T01:13:33.784264+0000","last_peered":"2024-04-25T01:13:33.784264+0000","last_clean":"2024-04-25T01:13:33.784264+0000","last_became_active":"2024-04-25T01:13:31.533145+0000","last_became_peered":"2024-04-25T01:13:31.533145+0000","last_unstale":"2024-04-25T01:13:33.784264+0000","last_undegraded":"2024-04-25T01:13:33.784264+0000","last_fullsized":"2024-04-25T01:13:33.784264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:50.987823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.786295+0000","last_change":"2024-04-25T01:13:31.541682+0000","last_active":"2024-04-25T01:13:33.786295+0000","last_peered":"2024-04-25T01:13:33.786295+0000","last_clean":"2024-04-25T01:13:33.786295+0000","last_became_active":"2024-04-25T01:13:31.541257+0000","last_became_peered":"2024-04-25T01:13:31.541257+0000","last_unstale":"2024-04-25T01:13:33.786295+0000","last_undegraded":"2024-04-25T01:13:33.786295+0000","last_fullsized":"2024-04-25T01:13:33.786295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:08.033895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.788402+0000","last_change":"2024-04-25T01:13:31.539521+0000","last_active":"2024-04-25T01:13:33.788402+0000","last_peered":"2024-04-25T01:13:33.788402+0000","last_clean":"2024-04-25T01:13:33.788402+0000","last_became_active":"2024-04-25T01:13:31.538973+0000","last_became_peered":"2024-04-25T01:13:31.538973+0000","last_unstale":"2024-04-25T01:13:33.788402+0000","last_undegraded":"2024-04-25T01:13:33.788402+0000","last_fullsized":"2024-04-25T01:13:33.788402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:06.425942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.642574+0000","last_change":"2024-04-25T01:13:31.519068+0000","last_active":"2024-04-25T01:13:32.642574+0000","last_peered":"2024-04-25T01:13:32.642574+0000","last_clean":"2024-04-25T01:13:32.642574+0000","last_became_active":"2024-04-25T01:13:31.518705+0000","last_became_peered":"2024-04-25T01:13:31.518705+0000","last_unstale":"2024-04-25T01:13:32.642574+0000","last_undegraded":"2024-04-25T01:13:32.642574+0000","last_fullsized":"2024-04-25T01:13:32.642574+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:11.186149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784569+0000","last_change":"2024-04-25T01:13:31.514572+0000","last_active":"2024-04-25T01:13:33.784569+0000","last_peered":"2024-04-25T01:13:33.784569+0000","last_clean":"2024-04-25T01:13:33.784569+0000","last_became_active":"2024-04-25T01:13:31.514178+0000","last_became_peered":"2024-04-25T01:13:31.514178+0000","last_unstale":"2024-04-25T01:13:33.784569+0000","last_undegraded":"2024-04-25T01:13:33.784569+0000","last_fullsized":"2024-04-25T01:13:33.784569+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:17.856802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.497951+0000","last_change":"2024-04-25T01:13:31.520837+0000","last_active":"2024-04-25T01:13:32.497951+0000","last_peered":"2024-04-25T01:13:32.497951+0000","last_clean":"2024-04-25T01:13:32.497951+0000","last_became_active":"2024-04-25T01:13:31.515601+0000","last_became_peered":"2024-04-25T01:13:31.515601+0000","last_unstale":"2024-04-25T01:13:32.497951+0000","last_undegraded":"2024-04-25T01:13:32.497951+0000","last_fullsized":"2024-04-25T01:13:32.497951+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:37.583156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.784063+0000","last_change":"2024-04-25T01:13:31.527420+0000","last_active":"2024-04-25T01:13:33.784063+0000","last_peered":"2024-04-25T01:13:33.784063+0000","last_clean":"2024-04-25T01:13:33.784063+0000","last_became_active":"2024-04-25T01:13:31.527160+0000","last_became_peered":"2024-04-25T01:13:31.527160+0000","last_unstale":"2024-04-25T01:13:33.784063+0000","last_undegraded":"2024-04-25T01:13:33.784063+0000","last_fullsized":"2024-04-25T01:13:33.784063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:17:46.201486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.496914+0000","last_change":"2024-04-25T01:13:31.518356+0000","last_active":"2024-04-25T01:13:32.496914+0000","last_peered":"2024-04-25T01:13:32.496914+0000","last_clean":"2024-04-25T01:13:32.496914+0000","last_became_active":"2024-04-25T01:13:31.517654+0000","last_became_peered":"2024-04-25T01:13:31.517654+0000","last_unstale":"2024-04-25T01:13:32.496914+0000","last_undegraded":"2024-04-25T01:13:32.496914+0000","last_fullsized":"2024-04-25T01:13:32.496914+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:17.254963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.495100+0000","last_change":"2024-04-25T01:13:31.526217+0000","last_active":"2024-04-25T01:13:32.495100+0000","last_peered":"2024-04-25T01:13:32.495100+0000","last_clean":"2024-04-25T01:13:32.495100+0000","last_became_active":"2024-04-25T01:13:31.525801+0000","last_became_peered":"2024-04-25T01:13:31.525801+0000","last_unstale":"2024-04-25T01:13:32.495100+0000","last_undegraded":"2024-04-25T01:13:32.495100+0000","last_fullsized":"2024-04-25T01:13:32.495100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:48:30.244205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:33.787594+0000","last_change":"2024-04-25T01:13:31.543596+0000","last_active":"2024-04-25T01:13:33.787594+0000","last_peered":"2024-04-25T01:13:33.787594+0000","last_clean":"2024-04-25T01:13:33.787594+0000","last_became_active":"2024-04-25T01:13:31.542149+0000","last_became_peered":"2024-04-25T01:13:31.542149+0000","last_unstale":"2024-04-25T01:13:33.787594+0000","last_undegraded":"2024-04-25T01:13:33.787594+0000","last_fullsized":"2024-04-25T01:13:33.787594+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:05:50.392151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T01:13:32.642431+0000","last_change":"2024-04-25T01:13:31.513575+0000","last_active":"2024-04-25T01:13:32.642431+0000","last_peered":"2024-04-25T01:13:32.642431+0000","last_clean":"2024-04-25T01:13:32.642431+0000","last_became_active":"2024-04-25T01:13:31.513085+0000","last_became_peered":"2024-04-25T01:13:31.513085+0000","last_unstale":"2024-04-25T01:13:32.642431+0000","last_undegraded":"2024-04-25T01:13:32.642431+0000","last_fullsized":"2024-04-25T01:13:32.642431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:36.853033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":41803,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":39,"num_read_kb":48,"num_write":135,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":319488,"data_stored":95231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318289,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":445784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579925,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1589248,"data_stored":1058551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743450,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":446020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005089,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":445940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266725,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32680,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712472,"statfs":{"total":95995035648,"available":95961571328,"internally_reserved":0,"allocated":1536000,"data_stored":1036774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430251,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":929792,"data_stored":445202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":466097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":446680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":448164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117050,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":946176,"data_stored":445486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855414,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":445846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691889,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1556480,"data_stored":1037163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":38193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":9562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":40162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:13:41.041 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T01:13:41.042 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T01:13:41.042 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T01:13:41.548 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:13:41.859 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:41 smithi031 ceph-mon[28053]: pgmap v321: 97 pgs: 97 active+clean; 617 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9.1 KiB/s wr, 20 op/s 2024-04-25T01:13:41.860 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:41 smithi031 ceph-mon[28053]: from='client.24742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:13:41.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:41 smithi169 ceph-mon[32759]: pgmap v321: 97 pgs: 97 active+clean; 617 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9.1 KiB/s wr, 20 op/s 2024-04-25T01:13:41.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:41 smithi169 ceph-mon[32759]: from='client.24742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:13:41.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:41 smithi040 ceph-mon[32518]: pgmap v321: 97 pgs: 97 active+clean; 617 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9.1 KiB/s wr, 20 op/s 2024-04-25T01:13:41.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:41 smithi040 ceph-mon[32518]: from='client.24742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:13:42.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:42 smithi169 ceph-mon[32759]: from='client.24740 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:13:42.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:42 smithi040 ceph-mon[32518]: from='client.24740 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:13:43.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:42 smithi031 ceph-mon[28053]: from='client.24740 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:13:43.655 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:43 smithi031 ceph-mon[28053]: pgmap v322: 97 pgs: 97 active+clean; 618 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 8.0 KiB/s wr, 18 op/s 2024-04-25T01:13:43.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:43 smithi169 ceph-mon[32759]: pgmap v322: 97 pgs: 97 active+clean; 618 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 8.0 KiB/s wr, 18 op/s 2024-04-25T01:13:43.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:43 smithi040 ceph-mon[32518]: pgmap v322: 97 pgs: 97 active+clean; 618 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 8.0 KiB/s wr, 18 op/s 2024-04-25T01:13:43.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.970+0000 7fda10a00700 1 -- 172.21.15.31:0/4204438967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c105510 msgr2=0x7fda0c105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:43.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.970+0000 7fda10a00700 1 --2- 172.21.15.31:0/4204438967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c105510 0x7fda0c105970 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009920 tx=0x7fd9f8009c30 comp rx=0 tx=0).stop 2024-04-25T01:13:43.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.971+0000 7fda10a00700 1 -- 172.21.15.31:0/4204438967 shutdown_connections 2024-04-25T01:13:43.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.971+0000 7fda10a00700 1 --2- 172.21.15.31:0/4204438967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda0c104310 0x7fda0c104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:43.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.971+0000 7fda10a00700 1 --2- 172.21.15.31:0/4204438967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda0c105eb0 0x7fda0c10c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:43.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.971+0000 7fda10a00700 1 --2- 172.21.15.31:0/4204438967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c105510 0x7fda0c105970 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:43.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.971+0000 7fda10a00700 1 -- 172.21.15.31:0/4204438967 >> 172.21.15.31:0/4204438967 conn(0x7fda0c0ff8b0 msgr2=0x7fda0c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:43.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.971+0000 7fda10a00700 1 -- 172.21.15.31:0/4204438967 shutdown_connections 2024-04-25T01:13:43.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.971+0000 7fda10a00700 1 -- 172.21.15.31:0/4204438967 wait complete. 2024-04-25T01:13:43.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.972+0000 7fda10a00700 1 Processor -- start 2024-04-25T01:13:43.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.973+0000 7fda10a00700 1 -- start start 2024-04-25T01:13:43.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.973+0000 7fda10a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda0c104310 0x7fda0c19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:43.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.973+0000 7fda10a00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c105510 0x7fda0c19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:43.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.973+0000 7fda10a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda0c105eb0 0x7fda0c1a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:43.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.973+0000 7fda10a00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c10ff30 con 0x7fda0c104310 2024-04-25T01:13:43.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.973+0000 7fda10a00700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c10fdb0 con 0x7fda0c105510 2024-04-25T01:13:43.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.973+0000 7fda10a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c1100b0 con 0x7fda0c105eb0 2024-04-25T01:13:43.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.973+0000 7fda0aa00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c105510 0x7fda0c19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:43.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.973+0000 7fda0b400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda0c104310 0x7fda0c19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:43.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.973+0000 7fda0aa00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c105510 0x7fda0c19aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:40974/0 (socket says 172.21.15.31:40974) 2024-04-25T01:13:43.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.973+0000 7fda0aa00700 1 -- 172.21.15.31:0/2619291221 learned_addr learned my addr 172.21.15.31:0/2619291221 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.974+0000 7fda0be00700 1 --2- 172.21.15.31:0/2619291221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda0c105eb0 0x7fda0c1a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.974+0000 7fda0b400700 1 -- 172.21.15.31:0/2619291221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda0c105eb0 msgr2=0x7fda0c1a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.974+0000 7fda0b400700 1 --2- 172.21.15.31:0/2619291221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda0c105eb0 0x7fda0c1a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.974+0000 7fda0b400700 1 -- 172.21.15.31:0/2619291221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c105510 msgr2=0x7fda0c19aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.974+0000 7fda0b400700 1 --2- 172.21.15.31:0/2619291221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c105510 0x7fda0c19aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.974+0000 7fda0b400700 1 -- 172.21.15.31:0/2619291221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0c1a2170 con 0x7fda0c104310 2024-04-25T01:13:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.974+0000 7fda0b400700 1 --2- 172.21.15.31:0/2619291221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda0c104310 0x7fda0c19a510 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00e9c0 tx=0x7fd9fc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.975+0000 7fda03e00700 1 -- 172.21.15.31:0/2619291221 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc004220 con 0x7fda0c104310 2024-04-25T01:13:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.975+0000 7fda10a00700 1 -- 172.21.15.31:0/2619291221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda0c1a2460 con 0x7fda0c104310 2024-04-25T01:13:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.975+0000 7fda03e00700 1 -- 172.21.15.31:0/2619291221 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9fc0043c0 con 0x7fda0c104310 2024-04-25T01:13:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.975+0000 7fda03e00700 1 -- 172.21.15.31:0/2619291221 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc003be0 con 0x7fda0c104310 2024-04-25T01:13:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.975+0000 7fda10a00700 1 -- 172.21.15.31:0/2619291221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda0c1a29f0 con 0x7fda0c104310 2024-04-25T01:13:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.976+0000 7fda10a00700 1 -- 172.21.15.31:0/2619291221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda0c063890 con 0x7fda0c104310 2024-04-25T01:13:43.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.978+0000 7fda03e00700 1 -- 172.21.15.31:0/2619291221 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd9fc00c7c0 con 0x7fda0c104310 2024-04-25T01:13:43.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.980+0000 7fda03e00700 1 --2- 172.21.15.31:0/2619291221 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd9f00a2de0 0x7fd9f00a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:43.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.980+0000 7fda03e00700 1 -- 172.21.15.31:0/2619291221 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fd9fc0cf150 con 0x7fda0c104310 2024-04-25T01:13:43.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.980+0000 7fda0aa00700 1 --2- 172.21.15.31:0/2619291221 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd9f00a2de0 0x7fd9f00a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:43.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.981+0000 7fda0aa00700 1 --2- 172.21.15.31:0/2619291221 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd9f00a2de0 0x7fd9f00a52a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009680 tx=0x7fd9f8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:43.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:43.984+0000 7fda03e00700 1 -- 172.21.15.31:0/2619291221 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd9fc09c000 con 0x7fda0c104310 2024-04-25T01:13:44.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:44.308+0000 7fda10a00700 1 -- 172.21.15.31:0/2619291221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fda0c02dbd0 con 0x7fda0c104310 2024-04-25T01:13:44.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:44.311+0000 7fda03e00700 1 -- 172.21.15.31:0/2619291221 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fd9fc09b900 con 0x7fda0c104310 2024-04-25T01:13:44.300 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-04-25T01:13:44.300 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBIrilmnsqAEhAAUHX2DBzPz2bB73opp541TQ== 2024-04-25T01:13:44.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:44.315+0000 7fda10a00700 1 -- 172.21.15.31:0/2619291221 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd9f00a2de0 msgr2=0x7fd9f00a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:44.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:44.315+0000 7fda10a00700 1 --2- 172.21.15.31:0/2619291221 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd9f00a2de0 0x7fd9f00a52a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009680 tx=0x7fd9f8019040 comp rx=0 tx=0).stop 2024-04-25T01:13:44.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:44.316+0000 7fda10a00700 1 -- 172.21.15.31:0/2619291221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda0c104310 msgr2=0x7fda0c19a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:44.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:44.316+0000 7fda10a00700 1 --2- 172.21.15.31:0/2619291221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda0c104310 0x7fda0c19a510 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00e9c0 tx=0x7fd9fc00ecd0 comp rx=0 tx=0).stop 2024-04-25T01:13:44.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:44.316+0000 7fda10a00700 1 -- 172.21.15.31:0/2619291221 shutdown_connections 2024-04-25T01:13:44.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:44.316+0000 7fda10a00700 1 --2- 172.21.15.31:0/2619291221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda0c104310 0x7fda0c19a510 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:44.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:44.316+0000 7fda10a00700 1 --2- 172.21.15.31:0/2619291221 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd9f00a2de0 0x7fd9f00a52a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:44.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:44.316+0000 7fda10a00700 1 --2- 172.21.15.31:0/2619291221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda0c105eb0 0x7fda0c1a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:44.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:44.316+0000 7fda10a00700 1 --2- 172.21.15.31:0/2619291221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c105510 0x7fda0c19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:44.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:44.316+0000 7fda10a00700 1 -- 172.21.15.31:0/2619291221 >> 172.21.15.31:0/2619291221 conn(0x7fda0c0ff8b0 msgr2=0x7fda0c1087b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:44.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:44.316+0000 7fda10a00700 1 -- 172.21.15.31:0/2619291221 shutdown_connections 2024-04-25T01:13:44.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:44.316+0000 7fda10a00700 1 -- 172.21.15.31:0/2619291221 wait complete. 2024-04-25T01:13:44.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:44 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2619291221' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T01:13:44.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:44 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2619291221' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T01:13:44.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:44 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2619291221' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T01:13:44.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:44 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2619291221' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T01:13:45.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:44 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2619291221' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T01:13:45.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:44 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2619291221' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T01:13:45.214 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T01:13:45.214 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T01:13:45.214 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T01:13:45.349 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T01:13:45.349 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T01:13:45.349 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T01:13:45.628 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:45 smithi031 ceph-mon[28053]: pgmap v323: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.0 KiB/s wr, 17 op/s 2024-04-25T01:13:45.827 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:13:45.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:45 smithi169 ceph-mon[32759]: pgmap v323: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.0 KiB/s wr, 17 op/s 2024-04-25T01:13:45.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:45 smithi040 ceph-mon[32518]: pgmap v323: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.0 KiB/s wr, 17 op/s 2024-04-25T01:13:47.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:47 smithi169 ceph-mon[32759]: pgmap v324: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.4 KiB/s wr, 16 op/s 2024-04-25T01:13:47.947 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:47 smithi040 ceph-mon[32518]: pgmap v324: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.4 KiB/s wr, 16 op/s 2024-04-25T01:13:47.958 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:47 smithi031 ceph-mon[28053]: pgmap v324: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.4 KiB/s wr, 16 op/s 2024-04-25T01:13:48.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.145+0000 7ff612000700 1 -- 172.21.15.31:0/3679468015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff60c102e90 msgr2=0x7ff60c106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:48.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.145+0000 7ff612000700 1 --2- 172.21.15.31:0/3679468015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff60c102e90 0x7ff60c106e90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009990 tx=0x7ff5f8009ca0 comp rx=0 tx=0).stop 2024-04-25T01:13:48.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.145+0000 7ff612000700 1 -- 172.21.15.31:0/3679468015 shutdown_connections 2024-04-25T01:13:48.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.145+0000 7ff612000700 1 --2- 172.21.15.31:0/3679468015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff60c107570 0x7ff60c10f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:48.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.145+0000 7ff612000700 1 --2- 172.21.15.31:0/3679468015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c1024e0 0x7ff60c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:48.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.145+0000 7ff612000700 1 --2- 172.21.15.31:0/3679468015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff60c102e90 0x7ff60c106e90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:48.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.145+0000 7ff612000700 1 -- 172.21.15.31:0/3679468015 >> 172.21.15.31:0/3679468015 conn(0x7ff60c079390 msgr2=0x7ff60c0797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:48.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.145+0000 7ff612000700 1 -- 172.21.15.31:0/3679468015 shutdown_connections 2024-04-25T01:13:48.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.145+0000 7ff612000700 1 -- 172.21.15.31:0/3679468015 wait complete. 2024-04-25T01:13:48.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.147+0000 7ff612000700 1 Processor -- start 2024-04-25T01:13:48.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.147+0000 7ff612000700 1 -- start start 2024-04-25T01:13:48.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.148+0000 7ff612000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff60c1024e0 0x7ff60c074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:48.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.148+0000 7ff612000700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c102e90 0x7ff60c074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:48.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.148+0000 7ff612000700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff60c107570 0x7ff60c074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:48.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.148+0000 7ff612000700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c112510 con 0x7ff60c107570 2024-04-25T01:13:48.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.148+0000 7ff612000700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c112390 con 0x7ff60c102e90 2024-04-25T01:13:48.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.148+0000 7ff612000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c112690 con 0x7ff60c1024e0 2024-04-25T01:13:48.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.148+0000 7ff60be00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff60c107570 0x7ff60c074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:48.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.148+0000 7ff60be00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff60c107570 0x7ff60c074fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41842/0 (socket says 172.21.15.31:41842) 2024-04-25T01:13:48.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.148+0000 7ff60b400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff60c1024e0 0x7ff60c074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:48.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.148+0000 7ff60be00700 1 -- 172.21.15.31:0/1532397918 learned_addr learned my addr 172.21.15.31:0/1532397918 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:48.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.148+0000 7ff60aa00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c102e90 0x7ff60c074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:48.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.148+0000 7ff60be00700 1 -- 172.21.15.31:0/1532397918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff60c1024e0 msgr2=0x7ff60c074530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:48.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.148+0000 7ff60be00700 1 --2- 172.21.15.31:0/1532397918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff60c1024e0 0x7ff60c074530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:48.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.148+0000 7ff60be00700 1 -- 172.21.15.31:0/1532397918 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c102e90 msgr2=0x7ff60c074a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:48.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.149+0000 7ff60be00700 1 --2- 172.21.15.31:0/1532397918 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c102e90 0x7ff60c074a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:48.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.149+0000 7ff60be00700 1 -- 172.21.15.31:0/1532397918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60c1a6f40 con 0x7ff60c107570 2024-04-25T01:13:48.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.149+0000 7ff60aa00700 1 --2- 172.21.15.31:0/1532397918 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c102e90 0x7ff60c074a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:13:48.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.149+0000 7ff60be00700 1 --2- 172.21.15.31:0/1532397918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff60c107570 0x7ff60c074fb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff60400ef30 tx=0x7ff60400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:48.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.149+0000 7ff603e00700 1 -- 172.21.15.31:0/1532397918 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6040092e0 con 0x7ff60c107570 2024-04-25T01:13:48.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.149+0000 7ff612000700 1 -- 172.21.15.31:0/1532397918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff60c1a71d0 con 0x7ff60c107570 2024-04-25T01:13:48.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.149+0000 7ff603e00700 1 -- 172.21.15.31:0/1532397918 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff604004bb0 con 0x7ff60c107570 2024-04-25T01:13:48.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.149+0000 7ff603e00700 1 -- 172.21.15.31:0/1532397918 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff604004280 con 0x7ff60c107570 2024-04-25T01:13:48.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.149+0000 7ff612000700 1 -- 172.21.15.31:0/1532397918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff60c1a76e0 con 0x7ff60c107570 2024-04-25T01:13:48.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.150+0000 7ff612000700 1 -- 172.21.15.31:0/1532397918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff60c103970 con 0x7ff60c107570 2024-04-25T01:13:48.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.152+0000 7ff603e00700 1 -- 172.21.15.31:0/1532397918 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff60401b030 con 0x7ff60c107570 2024-04-25T01:13:48.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.154+0000 7ff603e00700 1 --2- 172.21.15.31:0/1532397918 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff5f00a2cb0 0x7ff5f00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:48.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.154+0000 7ff603e00700 1 -- 172.21.15.31:0/1532397918 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7ff6040cf510 con 0x7ff60c107570 2024-04-25T01:13:48.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.154+0000 7ff60b400700 1 --2- 172.21.15.31:0/1532397918 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff5f00a2cb0 0x7ff5f00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:48.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.155+0000 7ff60b400700 1 --2- 172.21.15.31:0/1532397918 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff5f00a2cb0 0x7ff5f00a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc009c60 tx=0x7ff5fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:48.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.157+0000 7ff603e00700 1 -- 172.21.15.31:0/1532397918 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff604022020 con 0x7ff60c107570 2024-04-25T01:13:48.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.450+0000 7ff612000700 1 -- 172.21.15.31:0/1532397918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7ff60c02db30 con 0x7ff60c107570 2024-04-25T01:13:48.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.466+0000 7ff603e00700 1 -- 172.21.15.31:0/1532397918 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263199 (secure 0 0 0) 0x7ff604010040 con 0x7ff60c107570 2024-04-25T01:13:48.457 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:13:48.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.473+0000 7ff612000700 1 -- 172.21.15.31:0/1532397918 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff5f00a2cb0 msgr2=0x7ff5f00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:48.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.473+0000 7ff612000700 1 --2- 172.21.15.31:0/1532397918 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff5f00a2cb0 0x7ff5f00a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc009c60 tx=0x7ff5fc009260 comp rx=0 tx=0).stop 2024-04-25T01:13:48.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.473+0000 7ff612000700 1 -- 172.21.15.31:0/1532397918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff60c107570 msgr2=0x7ff60c074fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:48.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.473+0000 7ff612000700 1 --2- 172.21.15.31:0/1532397918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff60c107570 0x7ff60c074fb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff60400ef30 tx=0x7ff60400c3a0 comp rx=0 tx=0).stop 2024-04-25T01:13:48.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.473+0000 7ff612000700 1 -- 172.21.15.31:0/1532397918 shutdown_connections 2024-04-25T01:13:48.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.473+0000 7ff612000700 1 --2- 172.21.15.31:0/1532397918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff60c107570 0x7ff60c074fb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:48.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.473+0000 7ff612000700 1 --2- 172.21.15.31:0/1532397918 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff5f00a2cb0 0x7ff5f00a5170 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:48.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.473+0000 7ff612000700 1 --2- 172.21.15.31:0/1532397918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff60c1024e0 0x7ff60c074530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:48.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.473+0000 7ff612000700 1 --2- 172.21.15.31:0/1532397918 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c102e90 0x7ff60c074a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:48.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.473+0000 7ff612000700 1 -- 172.21.15.31:0/1532397918 >> 172.21.15.31:0/1532397918 conn(0x7ff60c079390 msgr2=0x7ff60c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:48.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.474+0000 7ff612000700 1 -- 172.21.15.31:0/1532397918 shutdown_connections 2024-04-25T01:13:48.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:48.474+0000 7ff612000700 1 -- 172.21.15.31:0/1532397918 wait complete. 2024-04-25T01:13:48.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:48 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1532397918' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T01:13:48.944 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:48 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1532397918' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T01:13:48.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:48 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1532397918' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T01:13:49.310 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":2874594057},{"type":"v1","addr":"172.21.15.31:6801","nonce":2874594057}]},"active_addr":"172.21.15.31:6801/2874594057","active_change":"2024-04-25T01:05:31.932435+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.31:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":638237247}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":187704274}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":3643837105}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":2811376504}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":1152637704}]}]} 2024-04-25T01:13:49.318 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T01:13:49.318 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T01:13:49.318 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T01:13:49.826 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:13:49.944 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:49 smithi169 ceph-mon[32759]: pgmap v325: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.1 KiB/s wr, 14 op/s 2024-04-25T01:13:49.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:49 smithi040 ceph-mon[32518]: pgmap v325: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.1 KiB/s wr, 14 op/s 2024-04-25T01:13:49.978 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:49 smithi031 ceph-mon[28053]: pgmap v325: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.1 KiB/s wr, 14 op/s 2024-04-25T01:13:51.782 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:51 smithi031 ceph-mon[28053]: pgmap v326: 97 pgs: 97 active+clean; 633 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-04-25T01:13:51.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:51 smithi169 ceph-mon[32759]: pgmap v326: 97 pgs: 97 active+clean; 633 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-04-25T01:13:51.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:51 smithi040 ceph-mon[32518]: pgmap v326: 97 pgs: 97 active+clean; 633 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-04-25T01:13:52.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.072+0000 7f3f47e00700 1 -- 172.21.15.31:0/1726726988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f401082f0 msgr2=0x7f3f401086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:52.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.072+0000 7f3f47e00700 1 --2- 172.21.15.31:0/1726726988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f401082f0 0x7f3f401086d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009920 tx=0x7f3f34009c30 comp rx=0 tx=0).stop 2024-04-25T01:13:52.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.072+0000 7f3f47e00700 1 -- 172.21.15.31:0/1726726988 shutdown_connections 2024-04-25T01:13:52.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.072+0000 7f3f47e00700 1 --2- 172.21.15.31:0/1726726988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f401082f0 0x7f3f401086d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:52.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.072+0000 7f3f47e00700 1 --2- 172.21.15.31:0/1726726988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f40102ca0 0x7f3f4010f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:52.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.072+0000 7f3f47e00700 1 --2- 172.21.15.31:0/1726726988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f401022e0 0x7f3f40102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:52.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.072+0000 7f3f47e00700 1 -- 172.21.15.31:0/1726726988 >> 172.21.15.31:0/1726726988 conn(0x7f3f40079390 msgr2=0x7f3f400797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:52.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.073+0000 7f3f47e00700 1 -- 172.21.15.31:0/1726726988 shutdown_connections 2024-04-25T01:13:52.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.073+0000 7f3f47e00700 1 -- 172.21.15.31:0/1726726988 wait complete. 2024-04-25T01:13:52.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.074+0000 7f3f47e00700 1 Processor -- start 2024-04-25T01:13:52.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.074+0000 7f3f47e00700 1 -- start start 2024-04-25T01:13:52.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.075+0000 7f3f47e00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f401022e0 0x7f3f401a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:52.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.075+0000 7f3f47e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f40102ca0 0x7f3f401a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:52.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.075+0000 7f3f47e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f401082f0 0x7f3f4019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:52.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.075+0000 7f3f47e00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3f401125f0 con 0x7f3f40102ca0 2024-04-25T01:13:52.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.075+0000 7f3f47e00700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3f40112470 con 0x7f3f401022e0 2024-04-25T01:13:52.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.075+0000 7f3f47e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3f40112770 con 0x7f3f401082f0 2024-04-25T01:13:52.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.075+0000 7f3f44a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f40102ca0 0x7f3f401a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:52.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.075+0000 7f3f45400700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f401022e0 0x7f3f401a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:52.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.075+0000 7f3f44a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f40102ca0 0x7f3f401a0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41862/0 (socket says 172.21.15.31:41862) 2024-04-25T01:13:52.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.075+0000 7f3f44a00700 1 -- 172.21.15.31:0/1667562802 learned_addr learned my addr 172.21.15.31:0/1667562802 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:52.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.075+0000 7f3f45e00700 1 --2- 172.21.15.31:0/1667562802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f401082f0 0x7f3f4019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:52.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.075+0000 7f3f45400700 1 -- 172.21.15.31:0/1667562802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f401082f0 msgr2=0x7f3f4019aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:52.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.075+0000 7f3f45400700 1 --2- 172.21.15.31:0/1667562802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f401082f0 0x7f3f4019aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:52.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.075+0000 7f3f45400700 1 -- 172.21.15.31:0/1667562802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f40102ca0 msgr2=0x7f3f401a0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:52.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.075+0000 7f3f45400700 1 --2- 172.21.15.31:0/1667562802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f40102ca0 0x7f3f401a0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:52.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.075+0000 7f3f45400700 1 -- 172.21.15.31:0/1667562802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4019b240 con 0x7f3f401022e0 2024-04-25T01:13:52.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.075+0000 7f3f44a00700 1 --2- 172.21.15.31:0/1667562802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f40102ca0 0x7f3f401a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:13:52.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.076+0000 7f3f45400700 1 --2- 172.21.15.31:0/1667562802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f401022e0 0x7f3f401a0920 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400a010 tx=0x7f3f3400fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:52.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.076+0000 7f3f3e000700 1 -- 172.21.15.31:0/1667562802 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f340050c0 con 0x7f3f401022e0 2024-04-25T01:13:52.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.076+0000 7f3f3e000700 1 -- 172.21.15.31:0/1667562802 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f3400b560 con 0x7f3f401022e0 2024-04-25T01:13:52.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.076+0000 7f3f3e000700 1 -- 172.21.15.31:0/1667562802 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f34003a00 con 0x7f3f401022e0 2024-04-25T01:13:52.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.076+0000 7f3f47e00700 1 -- 172.21.15.31:0/1667562802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f4019b4d0 con 0x7f3f401022e0 2024-04-25T01:13:52.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.076+0000 7f3f47e00700 1 -- 172.21.15.31:0/1667562802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f4019b900 con 0x7f3f401022e0 2024-04-25T01:13:52.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.077+0000 7f3f47e00700 1 -- 172.21.15.31:0/1667562802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f40068f20 con 0x7f3f401022e0 2024-04-25T01:13:52.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.080+0000 7f3f3e000700 1 -- 172.21.15.31:0/1667562802 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3f34029060 con 0x7f3f401022e0 2024-04-25T01:13:52.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.081+0000 7f3f3e000700 1 --2- 172.21.15.31:0/1667562802 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f3f280a2cb0 0x7f3f280a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:52.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.081+0000 7f3f3e000700 1 -- 172.21.15.31:0/1667562802 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f3f340cf830 con 0x7f3f401022e0 2024-04-25T01:13:52.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.082+0000 7f3f44a00700 1 --2- 172.21.15.31:0/1667562802 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f3f280a2cb0 0x7f3f280a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:52.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.082+0000 7f3f44a00700 1 --2- 172.21.15.31:0/1667562802 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f3f280a2cb0 0x7f3f280a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3f40079d70 tx=0x7f3f2c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:52.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.086+0000 7f3f3e000700 1 -- 172.21.15.31:0/1667562802 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3f34013030 con 0x7f3f401022e0 2024-04-25T01:13:52.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.322+0000 7f3f47e00700 1 -- 172.21.15.31:0/1667562802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f401a7920 con 0x7f3f401022e0 2024-04-25T01:13:52.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.324+0000 7f3f3e000700 1 -- 172.21.15.31:0/1667562802 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22172 (secure 0 0 0) 0x7f3f3409c770 con 0x7f3f401022e0 2024-04-25T01:13:52.311 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:13:52.311 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"95849b94-029f-11ef-bc93-c7b262605968","created":"2024-04-25T01:04:23.310856+0000","modified":"2024-04-25T01:13:32.489971+0000","last_up_change":"2024-04-25T01:12:29.393907+0000","last_in_change":"2024-04-25T01:12:08.978148+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:18.662391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:13:29.258771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:13:30.480996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa08e4c-fe6b-4116-bdf5-1adf5b465d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6803","nonce":788174091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6805","nonce":788174091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6809","nonce":788174091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6807","nonce":788174091}]},"public_addr":"172.21.15.31:6803/788174091","cluster_addr":"172.21.15.31:6805/788174091","heartbeat_back_addr":"172.21.15.31:6809/788174091","heartbeat_front_addr":"172.21.15.31:6807/788174091","state":["exists","up"]},{"osd":1,"uuid":"85339fe7-0dbc-4485-9334-2b039993c1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6801","nonce":3196394509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6803","nonce":3196394509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6807","nonce":3196394509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6805","nonce":3196394509}]},"public_addr":"172.21.15.40:6801/3196394509","cluster_addr":"172.21.15.40:6803/3196394509","heartbeat_back_addr":"172.21.15.40:6807/3196394509","heartbeat_front_addr":"172.21.15.40:6805/3196394509","state":["exists","up"]},{"osd":2,"uuid":"ab62ecb2-8aae-4987-9e2e-589e0f2fee5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6801","nonce":1955211197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6803","nonce":1955211197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6807","nonce":1955211197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6805","nonce":1955211197}]},"public_addr":"172.21.15.169:6801/1955211197","cluster_addr":"172.21.15.169:6803/1955211197","heartbeat_back_addr":"172.21.15.169:6807/1955211197","heartbeat_front_addr":"172.21.15.169:6805/1955211197","state":["exists","up"]},{"osd":3,"uuid":"8b5a7881-3401-495c-8ca4-c90685919d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6811","nonce":1228688643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6813","nonce":1228688643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6817","nonce":1228688643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6815","nonce":1228688643}]},"public_addr":"172.21.15.31:6811/1228688643","cluster_addr":"172.21.15.31:6813/1228688643","heartbeat_back_addr":"172.21.15.31:6817/1228688643","heartbeat_front_addr":"172.21.15.31:6815/1228688643","state":["exists","up"]},{"osd":4,"uuid":"56deb61d-a334-4cf2-8ac7-fbb6e6947092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6809","nonce":1913779091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6811","nonce":1913779091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6815","nonce":1913779091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6813","nonce":1913779091}]},"public_addr":"172.21.15.40:6809/1913779091","cluster_addr":"172.21.15.40:6811/1913779091","heartbeat_back_addr":"172.21.15.40:6815/1913779091","heartbeat_front_addr":"172.21.15.40:6813/1913779091","state":["exists","up"]},{"osd":5,"uuid":"8da55693-f9c7-49a7-99e0-165078f8001b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6809","nonce":2549650781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6811","nonce":2549650781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6815","nonce":2549650781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6813","nonce":2549650781}]},"public_addr":"172.21.15.169:6809/2549650781","cluster_addr":"172.21.15.169:6811/2549650781","heartbeat_back_addr":"172.21.15.169:6815/2549650781","heartbeat_front_addr":"172.21.15.169:6813/2549650781","state":["exists","up"]},{"osd":6,"uuid":"fc8350d2-1ad6-43cb-89f4-d336afc99ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6819","nonce":611507624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6821","nonce":611507624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6825","nonce":611507624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6823","nonce":611507624}]},"public_addr":"172.21.15.31:6819/611507624","cluster_addr":"172.21.15.31:6821/611507624","heartbeat_back_addr":"172.21.15.31:6825/611507624","heartbeat_front_addr":"172.21.15.31:6823/611507624","state":["exists","up"]},{"osd":7,"uuid":"3121b58b-8c94-40b5-9d48-f56552d7c39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6817","nonce":3776085849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6819","nonce":3776085849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6823","nonce":3776085849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6821","nonce":3776085849}]},"public_addr":"172.21.15.40:6817/3776085849","cluster_addr":"172.21.15.40:6819/3776085849","heartbeat_back_addr":"172.21.15.40:6823/3776085849","heartbeat_front_addr":"172.21.15.40:6821/3776085849","state":["exists","up"]},{"osd":8,"uuid":"f60407df-91f5-477a-9702-ddaef24faf82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6817","nonce":2736546122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6819","nonce":2736546122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6823","nonce":2736546122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6821","nonce":2736546122}]},"public_addr":"172.21.15.169:6817/2736546122","cluster_addr":"172.21.15.169:6819/2736546122","heartbeat_back_addr":"172.21.15.169:6823/2736546122","heartbeat_front_addr":"172.21.15.169:6821/2736546122","state":["exists","up"]},{"osd":9,"uuid":"7506a407-1960-49bc-9e38-7f2a8bf996ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6827","nonce":1187207880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6829","nonce":1187207880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6833","nonce":1187207880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6831","nonce":1187207880}]},"public_addr":"172.21.15.31:6827/1187207880","cluster_addr":"172.21.15.31:6829/1187207880","heartbeat_back_addr":"172.21.15.31:6833/1187207880","heartbeat_front_addr":"172.21.15.31:6831/1187207880","state":["exists","up"]},{"osd":10,"uuid":"831c48e2-b0d0-451f-959b-228e927542cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6825","nonce":3050148001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6827","nonce":3050148001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6831","nonce":3050148001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6829","nonce":3050148001}]},"public_addr":"172.21.15.40:6825/3050148001","cluster_addr":"172.21.15.40:6827/3050148001","heartbeat_back_addr":"172.21.15.40:6831/3050148001","heartbeat_front_addr":"172.21.15.40:6829/3050148001","state":["exists","up"]},{"osd":11,"uuid":"144fe0aa-f757-4b3c-90ad-99775e17d57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6825","nonce":262401944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6827","nonce":262401944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6831","nonce":262401944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6829","nonce":262401944}]},"public_addr":"172.21.15.169:6825/262401944","cluster_addr":"172.21.15.169:6827/262401944","heartbeat_back_addr":"172.21.15.169:6831/262401944","heartbeat_front_addr":"172.21.15.169:6829/262401944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:25.640995+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:51.036909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.635633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:48.467912+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:14.454971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:37.609315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:12.441807+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:38.239215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:02.230758+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:35.762553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:02.051405+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:26.569593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6801/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4004014066":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/1930101646":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/2638309002":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2238638505":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2902841144":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4190696323":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/480558521":"2024-04-26T01:04:57.149272+0000","172.21.15.31:6801/2638309002":"2024-04-26T01:04:57.149272+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:13:52.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.328+0000 7f3f47e00700 1 -- 172.21.15.31:0/1667562802 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f3f280a2cb0 msgr2=0x7f3f280a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:52.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.328+0000 7f3f47e00700 1 --2- 172.21.15.31:0/1667562802 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f3f280a2cb0 0x7f3f280a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3f40079d70 tx=0x7f3f2c009260 comp rx=0 tx=0).stop 2024-04-25T01:13:52.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.328+0000 7f3f47e00700 1 -- 172.21.15.31:0/1667562802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f401022e0 msgr2=0x7f3f401a0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:52.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.328+0000 7f3f47e00700 1 --2- 172.21.15.31:0/1667562802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f401022e0 0x7f3f401a0920 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400a010 tx=0x7f3f3400fa70 comp rx=0 tx=0).stop 2024-04-25T01:13:52.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.328+0000 7f3f47e00700 1 -- 172.21.15.31:0/1667562802 shutdown_connections 2024-04-25T01:13:52.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.328+0000 7f3f47e00700 1 --2- 172.21.15.31:0/1667562802 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f3f280a2cb0 0x7f3f280a5170 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:52.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.328+0000 7f3f47e00700 1 --2- 172.21.15.31:0/1667562802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f401082f0 0x7f3f4019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:52.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.328+0000 7f3f47e00700 1 --2- 172.21.15.31:0/1667562802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f401022e0 0x7f3f401a0920 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:52.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.328+0000 7f3f47e00700 1 --2- 172.21.15.31:0/1667562802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f40102ca0 0x7f3f401a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:52.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.328+0000 7f3f47e00700 1 -- 172.21.15.31:0/1667562802 >> 172.21.15.31:0/1667562802 conn(0x7f3f40079390 msgr2=0x7f3f400744d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:52.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.328+0000 7f3f47e00700 1 -- 172.21.15.31:0/1667562802 shutdown_connections 2024-04-25T01:13:52.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:52.328+0000 7f3f47e00700 1 -- 172.21.15.31:0/1667562802 wait complete. 2024-04-25T01:13:52.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:52 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1667562802' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:13:52.947 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:52 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1667562802' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:13:53.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:52 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1667562802' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:13:53.205 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T01:13:53.206 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T01:13:53.681 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:13:53.944 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:53 smithi169 ceph-mon[32759]: pgmap v327: 97 pgs: 97 active+clean; 633 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-04-25T01:13:53.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:53 smithi040 ceph-mon[32518]: pgmap v327: 97 pgs: 97 active+clean; 633 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-04-25T01:13:54.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:53 smithi031 ceph-mon[28053]: pgmap v327: 97 pgs: 97 active+clean; 633 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-04-25T01:13:55.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:55 smithi031 ceph-mon[28053]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-25T01:13:55.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:55 smithi169 ceph-mon[32759]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-25T01:13:55.949 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:55 smithi040 ceph-mon[32518]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-25T01:13:56.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.042+0000 7f147da00700 1 -- 172.21.15.31:0/251477783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1478103c90 msgr2=0x7f147810c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:56.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.042+0000 7f147da00700 1 --2- 172.21.15.31:0/251477783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1478103c90 0x7f147810c180 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f14600098f0 tx=0x7f1460009c00 comp rx=0 tx=0).stop 2024-04-25T01:13:56.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.042+0000 7f147da00700 1 -- 172.21.15.31:0/251477783 shutdown_connections 2024-04-25T01:13:56.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.042+0000 7f147da00700 1 --2- 172.21.15.31:0/251477783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f147810c7a0 0x7f147810ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:56.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.042+0000 7f147da00700 1 --2- 172.21.15.31:0/251477783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1478103370 0x7f1478103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:56.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.043+0000 7f147da00700 1 --2- 172.21.15.31:0/251477783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1478103c90 0x7f147810c180 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:56.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.043+0000 7f147da00700 1 -- 172.21.15.31:0/251477783 >> 172.21.15.31:0/251477783 conn(0x7f14780fcbc0 msgr2=0x7f14780fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:56.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.043+0000 7f147da00700 1 -- 172.21.15.31:0/251477783 shutdown_connections 2024-04-25T01:13:56.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.044+0000 7f147da00700 1 -- 172.21.15.31:0/251477783 wait complete. 2024-04-25T01:13:56.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.045+0000 7f147da00700 1 Processor -- start 2024-04-25T01:13:56.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.045+0000 7f147da00700 1 -- start start 2024-04-25T01:13:56.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.046+0000 7f147da00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1478103370 0x7f1478100fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:56.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.046+0000 7f147da00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1478103c90 0x7f1478101510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:56.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.046+0000 7f147da00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f147810c7a0 0x7f1478101a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:56.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.046+0000 7f147da00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f14781124c0 con 0x7f147810c7a0 2024-04-25T01:13:56.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.046+0000 7f147da00700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1478112340 con 0x7f1478103c90 2024-04-25T01:13:56.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.046+0000 7f147da00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1478112640 con 0x7f1478103370 2024-04-25T01:13:56.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.046+0000 7f147d000700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f147810c7a0 0x7f1478101a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:56.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.046+0000 7f147d000700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f147810c7a0 0x7f1478101a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41880/0 (socket says 172.21.15.31:41880) 2024-04-25T01:13:56.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.046+0000 7f147d000700 1 -- 172.21.15.31:0/1439099959 learned_addr learned my addr 172.21.15.31:0/1439099959 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:13:56.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.046+0000 7f146fe00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1478103c90 0x7f1478101510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:56.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.046+0000 7f1477e00700 1 --2- 172.21.15.31:0/1439099959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1478103370 0x7f1478100fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:56.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.047+0000 7f146fe00700 1 -- 172.21.15.31:0/1439099959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1478103370 msgr2=0x7f1478100fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:56.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.047+0000 7f146fe00700 1 --2- 172.21.15.31:0/1439099959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1478103370 0x7f1478100fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:56.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.047+0000 7f146fe00700 1 -- 172.21.15.31:0/1439099959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f147810c7a0 msgr2=0x7f1478101a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:56.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.047+0000 7f146fe00700 1 --2- 172.21.15.31:0/1439099959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f147810c7a0 0x7f1478101a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:56.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.047+0000 7f146fe00700 1 -- 172.21.15.31:0/1439099959 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14781a6f70 con 0x7f1478103c90 2024-04-25T01:13:56.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.047+0000 7f147d000700 1 --2- 172.21.15.31:0/1439099959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f147810c7a0 0x7f1478101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:13:56.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.047+0000 7f146fe00700 1 --2- 172.21.15.31:0/1439099959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1478103c90 0x7f1478101510 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1460000c00 tx=0x7f146000f9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:56.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.047+0000 7f1475600700 1 -- 172.21.15.31:0/1439099959 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460005100 con 0x7f1478103c90 2024-04-25T01:13:56.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.047+0000 7f1475600700 1 -- 172.21.15.31:0/1439099959 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f146000b560 con 0x7f1478103c90 2024-04-25T01:13:56.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.047+0000 7f147da00700 1 -- 172.21.15.31:0/1439099959 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14781987d0 con 0x7f1478103c90 2024-04-25T01:13:56.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.048+0000 7f1475600700 1 -- 172.21.15.31:0/1439099959 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460003a00 con 0x7f1478103c90 2024-04-25T01:13:56.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.048+0000 7f147da00700 1 -- 172.21.15.31:0/1439099959 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1478198c80 con 0x7f1478103c90 2024-04-25T01:13:56.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.049+0000 7f147da00700 1 -- 172.21.15.31:0/1439099959 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1478068f20 con 0x7f1478103c90 2024-04-25T01:13:56.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.051+0000 7f1475600700 1 -- 172.21.15.31:0/1439099959 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f146000b710 con 0x7f1478103c90 2024-04-25T01:13:56.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.052+0000 7f1475600700 1 --2- 172.21.15.31:0/1439099959 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f145c0a2cb0 0x7f145c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:13:56.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.052+0000 7f1475600700 1 -- 172.21.15.31:0/1439099959 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f14600cfaa0 con 0x7f1478103c90 2024-04-25T01:13:56.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.053+0000 7f1477e00700 1 --2- 172.21.15.31:0/1439099959 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f145c0a2cb0 0x7f145c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:13:56.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.053+0000 7f1477e00700 1 --2- 172.21.15.31:0/1439099959 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f145c0a2cb0 0x7f145c0a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1468009750 tx=0x7f1468016040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:13:56.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.057+0000 7f1475600700 1 -- 172.21.15.31:0/1439099959 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1460013030 con 0x7f1478103c90 2024-04-25T01:13:56.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.293+0000 7f147da00700 1 -- 172.21.15.31:0/1439099959 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f147802db30 con 0x7f1478103c90 2024-04-25T01:13:56.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.295+0000 7f1475600700 1 -- 172.21.15.31:0/1439099959 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22172 (secure 0 0 0) 0x7f146009c9e0 con 0x7f1478103c90 2024-04-25T01:13:56.281 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:13:56.281 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"95849b94-029f-11ef-bc93-c7b262605968","created":"2024-04-25T01:04:23.310856+0000","modified":"2024-04-25T01:13:32.489971+0000","last_up_change":"2024-04-25T01:12:29.393907+0000","last_in_change":"2024-04-25T01:12:08.978148+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:18.662391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:13:29.258771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:13:30.480996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa08e4c-fe6b-4116-bdf5-1adf5b465d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6803","nonce":788174091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6805","nonce":788174091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6809","nonce":788174091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6807","nonce":788174091}]},"public_addr":"172.21.15.31:6803/788174091","cluster_addr":"172.21.15.31:6805/788174091","heartbeat_back_addr":"172.21.15.31:6809/788174091","heartbeat_front_addr":"172.21.15.31:6807/788174091","state":["exists","up"]},{"osd":1,"uuid":"85339fe7-0dbc-4485-9334-2b039993c1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6801","nonce":3196394509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6803","nonce":3196394509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6807","nonce":3196394509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6805","nonce":3196394509}]},"public_addr":"172.21.15.40:6801/3196394509","cluster_addr":"172.21.15.40:6803/3196394509","heartbeat_back_addr":"172.21.15.40:6807/3196394509","heartbeat_front_addr":"172.21.15.40:6805/3196394509","state":["exists","up"]},{"osd":2,"uuid":"ab62ecb2-8aae-4987-9e2e-589e0f2fee5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6801","nonce":1955211197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6803","nonce":1955211197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6807","nonce":1955211197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6805","nonce":1955211197}]},"public_addr":"172.21.15.169:6801/1955211197","cluster_addr":"172.21.15.169:6803/1955211197","heartbeat_back_addr":"172.21.15.169:6807/1955211197","heartbeat_front_addr":"172.21.15.169:6805/1955211197","state":["exists","up"]},{"osd":3,"uuid":"8b5a7881-3401-495c-8ca4-c90685919d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6811","nonce":1228688643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6813","nonce":1228688643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6817","nonce":1228688643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6815","nonce":1228688643}]},"public_addr":"172.21.15.31:6811/1228688643","cluster_addr":"172.21.15.31:6813/1228688643","heartbeat_back_addr":"172.21.15.31:6817/1228688643","heartbeat_front_addr":"172.21.15.31:6815/1228688643","state":["exists","up"]},{"osd":4,"uuid":"56deb61d-a334-4cf2-8ac7-fbb6e6947092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6809","nonce":1913779091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6811","nonce":1913779091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6815","nonce":1913779091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6813","nonce":1913779091}]},"public_addr":"172.21.15.40:6809/1913779091","cluster_addr":"172.21.15.40:6811/1913779091","heartbeat_back_addr":"172.21.15.40:6815/1913779091","heartbeat_front_addr":"172.21.15.40:6813/1913779091","state":["exists","up"]},{"osd":5,"uuid":"8da55693-f9c7-49a7-99e0-165078f8001b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6809","nonce":2549650781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6811","nonce":2549650781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6815","nonce":2549650781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6813","nonce":2549650781}]},"public_addr":"172.21.15.169:6809/2549650781","cluster_addr":"172.21.15.169:6811/2549650781","heartbeat_back_addr":"172.21.15.169:6815/2549650781","heartbeat_front_addr":"172.21.15.169:6813/2549650781","state":["exists","up"]},{"osd":6,"uuid":"fc8350d2-1ad6-43cb-89f4-d336afc99ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6819","nonce":611507624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6821","nonce":611507624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6825","nonce":611507624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6823","nonce":611507624}]},"public_addr":"172.21.15.31:6819/611507624","cluster_addr":"172.21.15.31:6821/611507624","heartbeat_back_addr":"172.21.15.31:6825/611507624","heartbeat_front_addr":"172.21.15.31:6823/611507624","state":["exists","up"]},{"osd":7,"uuid":"3121b58b-8c94-40b5-9d48-f56552d7c39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6817","nonce":3776085849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6819","nonce":3776085849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6823","nonce":3776085849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6821","nonce":3776085849}]},"public_addr":"172.21.15.40:6817/3776085849","cluster_addr":"172.21.15.40:6819/3776085849","heartbeat_back_addr":"172.21.15.40:6823/3776085849","heartbeat_front_addr":"172.21.15.40:6821/3776085849","state":["exists","up"]},{"osd":8,"uuid":"f60407df-91f5-477a-9702-ddaef24faf82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6817","nonce":2736546122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6819","nonce":2736546122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6823","nonce":2736546122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6821","nonce":2736546122}]},"public_addr":"172.21.15.169:6817/2736546122","cluster_addr":"172.21.15.169:6819/2736546122","heartbeat_back_addr":"172.21.15.169:6823/2736546122","heartbeat_front_addr":"172.21.15.169:6821/2736546122","state":["exists","up"]},{"osd":9,"uuid":"7506a407-1960-49bc-9e38-7f2a8bf996ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6827","nonce":1187207880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6829","nonce":1187207880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6833","nonce":1187207880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6831","nonce":1187207880}]},"public_addr":"172.21.15.31:6827/1187207880","cluster_addr":"172.21.15.31:6829/1187207880","heartbeat_back_addr":"172.21.15.31:6833/1187207880","heartbeat_front_addr":"172.21.15.31:6831/1187207880","state":["exists","up"]},{"osd":10,"uuid":"831c48e2-b0d0-451f-959b-228e927542cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6825","nonce":3050148001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6827","nonce":3050148001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6831","nonce":3050148001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6829","nonce":3050148001}]},"public_addr":"172.21.15.40:6825/3050148001","cluster_addr":"172.21.15.40:6827/3050148001","heartbeat_back_addr":"172.21.15.40:6831/3050148001","heartbeat_front_addr":"172.21.15.40:6829/3050148001","state":["exists","up"]},{"osd":11,"uuid":"144fe0aa-f757-4b3c-90ad-99775e17d57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6825","nonce":262401944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6827","nonce":262401944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6831","nonce":262401944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6829","nonce":262401944}]},"public_addr":"172.21.15.169:6825/262401944","cluster_addr":"172.21.15.169:6827/262401944","heartbeat_back_addr":"172.21.15.169:6831/262401944","heartbeat_front_addr":"172.21.15.169:6829/262401944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:25.640995+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:51.036909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.635633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:48.467912+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:14.454971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:37.609315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:12.441807+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:38.239215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:02.230758+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:35.762553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:02.051405+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:26.569593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6801/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4004014066":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/1930101646":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/2638309002":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2238638505":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2902841144":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4190696323":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/480558521":"2024-04-26T01:04:57.149272+0000","172.21.15.31:6801/2638309002":"2024-04-26T01:04:57.149272+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:13:56.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.298+0000 7f147da00700 1 -- 172.21.15.31:0/1439099959 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f145c0a2cb0 msgr2=0x7f145c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:56.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.298+0000 7f147da00700 1 --2- 172.21.15.31:0/1439099959 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f145c0a2cb0 0x7f145c0a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1468009750 tx=0x7f1468016040 comp rx=0 tx=0).stop 2024-04-25T01:13:56.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.298+0000 7f147da00700 1 -- 172.21.15.31:0/1439099959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1478103c90 msgr2=0x7f1478101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:13:56.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.298+0000 7f147da00700 1 --2- 172.21.15.31:0/1439099959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1478103c90 0x7f1478101510 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1460000c00 tx=0x7f146000f9c0 comp rx=0 tx=0).stop 2024-04-25T01:13:56.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.298+0000 7f147da00700 1 -- 172.21.15.31:0/1439099959 shutdown_connections 2024-04-25T01:13:56.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.298+0000 7f147da00700 1 --2- 172.21.15.31:0/1439099959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f147810c7a0 0x7f1478101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:56.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.298+0000 7f147da00700 1 --2- 172.21.15.31:0/1439099959 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f145c0a2cb0 0x7f145c0a5170 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:56.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.298+0000 7f147da00700 1 --2- 172.21.15.31:0/1439099959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1478103370 0x7f1478100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:56.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.298+0000 7f147da00700 1 --2- 172.21.15.31:0/1439099959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1478103c90 0x7f1478101510 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:13:56.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.298+0000 7f147da00700 1 -- 172.21.15.31:0/1439099959 >> 172.21.15.31:0/1439099959 conn(0x7f14780fcbc0 msgr2=0x7f14780fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:13:56.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.298+0000 7f147da00700 1 -- 172.21.15.31:0/1439099959 shutdown_connections 2024-04-25T01:13:56.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:13:56.298+0000 7f147da00700 1 -- 172.21.15.31:0/1439099959 wait complete. 2024-04-25T01:13:56.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:56 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1439099959' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:13:56.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:56 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1439099959' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:13:56.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:56 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1439099959' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:13:57.065 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T01:13:57.065 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T01:13:57.066 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T01:13:57.066 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T01:13:57.067 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T01:13:57.067 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T01:13:57.067 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T01:13:57.068 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T01:13:57.068 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T01:13:57.069 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T01:13:57.069 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T01:13:57.070 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T01:13:57.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:57 smithi169 ceph-mon[32759]: pgmap v329: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-25T01:13:57.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:57 smithi040 ceph-mon[32518]: pgmap v329: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-25T01:13:58.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:57 smithi031 ceph-mon[28053]: pgmap v329: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-25T01:13:58.153 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:13:58.293 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:13:58.481 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:13:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:13:59.643 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:13:59.655 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:13:59.655 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:13:59.656 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:13:59.656 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:13:59.666 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:13:59.666 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:13:59.668 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:13:59.749 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:13:59 smithi031 ceph-mon[28053]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-25T01:13:59.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:13:59 smithi169 ceph-mon[32759]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-25T01:13:59.947 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:13:59 smithi040 ceph-mon[32518]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-25T01:14:01.944 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:01 smithi169 ceph-mon[32759]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T01:14:01.947 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:01 smithi040 ceph-mon[32518]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T01:14:02.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:01 smithi031 ceph-mon[28053]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T01:14:03.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:03 smithi040 ceph-mon[32518]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T01:14:04.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:03 smithi031 ceph-mon[28053]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T01:14:04.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:03 smithi169 ceph-mon[32759]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T01:14:05.808 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:05 smithi031 ceph-mon[28053]: pgmap v333: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T01:14:06.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:05 smithi169 ceph-mon[32759]: pgmap v333: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T01:14:06.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:05 smithi040 ceph-mon[32518]: pgmap v333: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T01:14:07.948 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:07 smithi040 ceph-mon[32518]: pgmap v334: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T01:14:08.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:07 smithi031 ceph-mon[28053]: pgmap v334: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T01:14:08.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:07 smithi169 ceph-mon[32759]: pgmap v334: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T01:14:10.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:09 smithi031 ceph-mon[28053]: pgmap v335: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T01:14:10.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:09 smithi169 ceph-mon[32759]: pgmap v335: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T01:14:10.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:09 smithi040 ceph-mon[32518]: pgmap v335: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T01:14:11.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.232+0000 7f84fbe00700 1 -- 172.21.15.31:0/1261107129 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84fc109680 msgr2=0x7f84fc111250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:11.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.232+0000 7f84fbe00700 1 --2- 172.21.15.31:0/1261107129 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84fc109680 0x7f84fc111250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009fc0 tx=0x7f84f400b330 comp rx=0 tx=0).stop 2024-04-25T01:14:11.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.232+0000 7f84fbe00700 1 -- 172.21.15.31:0/1261107129 shutdown_connections 2024-04-25T01:14:11.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.232+0000 7f84fbe00700 1 --2- 172.21.15.31:0/1261107129 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84fc109680 0x7f84fc111250 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:11.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.232+0000 7f84fbe00700 1 --2- 172.21.15.31:0/1261107129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84fc1045a0 0x7f84fc104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:11.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.232+0000 7f84fbe00700 1 --2- 172.21.15.31:0/1261107129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84fc104f50 0x7f84fc108fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:11.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.232+0000 7f84fbe00700 1 -- 172.21.15.31:0/1261107129 >> 172.21.15.31:0/1261107129 conn(0x7f84fc0ffe10 msgr2=0x7f84fc102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:11.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.232+0000 7f84fbe00700 1 -- 172.21.15.31:0/1261107129 shutdown_connections 2024-04-25T01:14:11.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.232+0000 7f84fbe00700 1 -- 172.21.15.31:0/1261107129 wait complete. 2024-04-25T01:14:11.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.233+0000 7f84fbe00700 1 Processor -- start 2024-04-25T01:14:11.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.233+0000 7f84fbe00700 1 -- start start 2024-04-25T01:14:11.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.234+0000 7f84fbe00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84fc1045a0 0x7f84fc1a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.234+0000 7f84fbe00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84fc104f50 0x7f84fc1a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.234+0000 7f84fbe00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84fc109680 0x7f84fc19a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.234+0000 7f84fbe00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f84fc1146e0 con 0x7f84fc1045a0 2024-04-25T01:14:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.234+0000 7f84fbe00700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f84fc114560 con 0x7f84fc104f50 2024-04-25T01:14:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.234+0000 7f84fbe00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f84fc114860 con 0x7f84fc109680 2024-04-25T01:14:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.234+0000 7f84faa00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84fc1045a0 0x7f84fc1a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.234+0000 7f84faa00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84fc1045a0 0x7f84fc1a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35450/0 (socket says 172.21.15.31:35450) 2024-04-25T01:14:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.234+0000 7f84faa00700 1 -- 172.21.15.31:0/4240088599 learned_addr learned my addr 172.21.15.31:0/4240088599 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:14:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.234+0000 7f84fb400700 1 --2- 172.21.15.31:0/4240088599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84fc109680 0x7f84fc19a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.234+0000 7f84fa000700 1 --2- 172.21.15.31:0/4240088599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84fc104f50 0x7f84fc1a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.235+0000 7f84fb400700 1 -- 172.21.15.31:0/4240088599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84fc104f50 msgr2=0x7f84fc1a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.235+0000 7f84fb400700 1 --2- 172.21.15.31:0/4240088599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84fc104f50 0x7f84fc1a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:11.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.235+0000 7f84fb400700 1 -- 172.21.15.31:0/4240088599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84fc1045a0 msgr2=0x7f84fc1a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:11.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.235+0000 7f84fb400700 1 --2- 172.21.15.31:0/4240088599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84fc1045a0 0x7f84fc1a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:11.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.235+0000 7f84fb400700 1 -- 172.21.15.31:0/4240088599 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84fc19b060 con 0x7f84fc109680 2024-04-25T01:14:11.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.235+0000 7f84faa00700 1 --2- 172.21.15.31:0/4240088599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84fc1045a0 0x7f84fc1a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:14:11.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.235+0000 7f84fb400700 1 --2- 172.21.15.31:0/4240088599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84fc109680 0x7f84fc19a900 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f84f4015000 tx=0x7f84f400cfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:11.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.235+0000 7f84f3400700 1 -- 172.21.15.31:0/4240088599 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4021070 con 0x7f84fc109680 2024-04-25T01:14:11.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.235+0000 7f84f3400700 1 -- 172.21.15.31:0/4240088599 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84f4007ac0 con 0x7f84fc109680 2024-04-25T01:14:11.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.235+0000 7f84fbe00700 1 -- 172.21.15.31:0/4240088599 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84fc19b2f0 con 0x7f84fc109680 2024-04-25T01:14:11.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.235+0000 7f84f3400700 1 -- 172.21.15.31:0/4240088599 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4005040 con 0x7f84fc109680 2024-04-25T01:14:11.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.235+0000 7f84fbe00700 1 -- 172.21.15.31:0/4240088599 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84fc1a7610 con 0x7f84fc109680 2024-04-25T01:14:11.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.236+0000 7f84fbe00700 1 -- 172.21.15.31:0/4240088599 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f84fc0629c0 con 0x7f84fc109680 2024-04-25T01:14:11.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.238+0000 7f84f3400700 1 -- 172.21.15.31:0/4240088599 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f84f4024070 con 0x7f84fc109680 2024-04-25T01:14:11.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.240+0000 7f84f3400700 1 --2- 172.21.15.31:0/4240088599 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f84e00a2cb0 0x7f84e00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:11.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.240+0000 7f84f3400700 1 -- 172.21.15.31:0/4240088599 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f84f400f020 con 0x7f84fc109680 2024-04-25T01:14:11.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.240+0000 7f84faa00700 1 --2- 172.21.15.31:0/4240088599 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f84e00a2cb0 0x7f84e00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:11.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.240+0000 7f84f3400700 1 --2- 172.21.15.31:0/4240088599 >> [v2:172.21.15.40:6824/3050148001,v1:172.21.15.40:6825/3050148001] conn(0x7f84e00ac490 0x7f84e00ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:11.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.240+0000 7f84f3400700 1 -- 172.21.15.31:0/4240088599 --> [v2:172.21.15.40:6824/3050148001,v1:172.21.15.40:6825/3050148001] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f84f400aea0 con 0x7f84e00ac490 2024-04-25T01:14:11.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.240+0000 7f84f3400700 1 -- 172.21.15.31:0/4240088599 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f84f40d0a20 con 0x7f84fc109680 2024-04-25T01:14:11.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.241+0000 7f84fa000700 1 --2- 172.21.15.31:0/4240088599 >> [v2:172.21.15.40:6824/3050148001,v1:172.21.15.40:6825/3050148001] conn(0x7f84e00ac490 0x7f84e00ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:11.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.241+0000 7f84faa00700 1 --2- 172.21.15.31:0/4240088599 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f84e00a2cb0 0x7f84e00a5170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f84fc19ba40 tx=0x7f84ec009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:11.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.241+0000 7f84fa000700 1 --2- 172.21.15.31:0/4240088599 >> [v2:172.21.15.40:6824/3050148001,v1:172.21.15.40:6825/3050148001] conn(0x7f84e00ac490 0x7f84e00ae8b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:11.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.242+0000 7f84f3400700 1 -- 172.21.15.31:0/4240088599 <== osd.10 v2:172.21.15.40:6824/3050148001 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f84f400aea0 con 0x7f84e00ac490 2024-04-25T01:14:11.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.261+0000 7f84fbe00700 1 -- 172.21.15.31:0/4240088599 --> [v2:172.21.15.40:6824/3050148001,v1:172.21.15.40:6825/3050148001] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f84fc062e40 con 0x7f84e00ac490 2024-04-25T01:14:11.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.261+0000 7f84f3400700 1 -- 172.21.15.31:0/4240088599 <== osd.10 v2:172.21.15.40:6824/3050148001 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f84fc062e40 con 0x7f84e00ac490 2024-04-25T01:14:11.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.262+0000 7f84fbe00700 1 -- 172.21.15.31:0/4240088599 >> [v2:172.21.15.40:6824/3050148001,v1:172.21.15.40:6825/3050148001] conn(0x7f84e00ac490 msgr2=0x7f84e00ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:11.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.262+0000 7f84fbe00700 1 --2- 172.21.15.31:0/4240088599 >> [v2:172.21.15.40:6824/3050148001,v1:172.21.15.40:6825/3050148001] conn(0x7f84e00ac490 0x7f84e00ae8b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:11.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.262+0000 7f84fbe00700 1 -- 172.21.15.31:0/4240088599 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f84e00a2cb0 msgr2=0x7f84e00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:11.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.262+0000 7f84fbe00700 1 --2- 172.21.15.31:0/4240088599 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f84e00a2cb0 0x7f84e00a5170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f84fc19ba40 tx=0x7f84ec009310 comp rx=0 tx=0).stop 2024-04-25T01:14:11.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.262+0000 7f84fbe00700 1 -- 172.21.15.31:0/4240088599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84fc109680 msgr2=0x7f84fc19a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:11.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.262+0000 7f84fbe00700 1 --2- 172.21.15.31:0/4240088599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84fc109680 0x7f84fc19a900 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f84f4015000 tx=0x7f84f400cfa0 comp rx=0 tx=0).stop 2024-04-25T01:14:11.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.263+0000 7f84fb400700 1 -- 172.21.15.31:0/4240088599 reap_dead start 2024-04-25T01:14:11.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.263+0000 7f84fbe00700 1 -- 172.21.15.31:0/4240088599 shutdown_connections 2024-04-25T01:14:11.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.263+0000 7f84fbe00700 1 -- 172.21.15.31:0/4240088599 >> 172.21.15.31:0/4240088599 conn(0x7f84fc0ffe10 msgr2=0x7f84fc100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:11.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.263+0000 7f84fbe00700 1 -- 172.21.15.31:0/4240088599 shutdown_connections 2024-04-25T01:14:11.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.263+0000 7f84fbe00700 1 -- 172.21.15.31:0/4240088599 wait complete. 2024-04-25T01:14:11.948 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:11 smithi031 ceph-mon[28053]: pgmap v336: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T01:14:12.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:11 smithi169 ceph-mon[32759]: pgmap v336: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T01:14:12.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:11 smithi040 ceph-mon[32518]: pgmap v336: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T01:14:12.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.961+0000 7fa62de00700 1 -- 172.21.15.31:0/304649733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6281024e0 msgr2=0x7fa6281028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:12.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.961+0000 7fa62de00700 1 --2- 172.21.15.31:0/304649733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6281024e0 0x7fa6281028c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa618009990 tx=0x7fa618009ca0 comp rx=0 tx=0).stop 2024-04-25T01:14:12.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.961+0000 7fa62de00700 1 -- 172.21.15.31:0/304649733 shutdown_connections 2024-04-25T01:14:12.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.961+0000 7fa62de00700 1 --2- 172.21.15.31:0/304649733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6281024e0 0x7fa6281028c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:12.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.961+0000 7fa62de00700 1 --2- 172.21.15.31:0/304649733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa628107570 0x7fa62810f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:12.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.961+0000 7fa62de00700 1 --2- 172.21.15.31:0/304649733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa628102e90 0x7fa628106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:12.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.961+0000 7fa62de00700 1 -- 172.21.15.31:0/304649733 >> 172.21.15.31:0/304649733 conn(0x7fa628079390 msgr2=0x7fa6280797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:12.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.961+0000 7fa62de00700 1 -- 172.21.15.31:0/304649733 shutdown_connections 2024-04-25T01:14:12.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.961+0000 7fa62de00700 1 -- 172.21.15.31:0/304649733 wait complete. 2024-04-25T01:14:12.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.962+0000 7fa62de00700 1 Processor -- start 2024-04-25T01:14:12.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.962+0000 7fa62de00700 1 -- start start 2024-04-25T01:14:12.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.963+0000 7fa62de00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6281024e0 0x7fa6281a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:12.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.963+0000 7fa62de00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa628102e90 0x7fa6281a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:12.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.963+0000 7fa62de00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa628107570 0x7fa62819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:12.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.963+0000 7fa62de00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa6281125b0 con 0x7fa6281024e0 2024-04-25T01:14:12.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.963+0000 7fa62de00700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa628112430 con 0x7fa628102e90 2024-04-25T01:14:12.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.963+0000 7fa62de00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa628112730 con 0x7fa628107570 2024-04-25T01:14:12.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.963+0000 7fa626a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6281024e0 0x7fa6281a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:12.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.963+0000 7fa626a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6281024e0 0x7fa6281a08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35476/0 (socket says 172.21.15.31:35476) 2024-04-25T01:14:12.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.963+0000 7fa626a00700 1 -- 172.21.15.31:0/175280485 learned_addr learned my addr 172.21.15.31:0/175280485 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:14:12.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.963+0000 7fa626000700 1 --2- 172.21.15.31:0/175280485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa628102e90 0x7fa6281a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:12.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.963+0000 7fa627400700 1 --2- 172.21.15.31:0/175280485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa628107570 0x7fa62819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:12.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.963+0000 7fa626a00700 1 -- 172.21.15.31:0/175280485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa628107570 msgr2=0x7fa62819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:12.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.963+0000 7fa626a00700 1 --2- 172.21.15.31:0/175280485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa628107570 0x7fa62819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:12.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.963+0000 7fa626a00700 1 -- 172.21.15.31:0/175280485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa628102e90 msgr2=0x7fa6281a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:12.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.963+0000 7fa626a00700 1 --2- 172.21.15.31:0/175280485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa628102e90 0x7fa6281a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:12.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.963+0000 7fa626a00700 1 -- 172.21.15.31:0/175280485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62819b1f0 con 0x7fa6281024e0 2024-04-25T01:14:12.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.964+0000 7fa626a00700 1 --2- 172.21.15.31:0/175280485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6281024e0 0x7fa6281a08d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa618000c00 tx=0x7fa61800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:12.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.964+0000 7fa62ca00700 1 -- 172.21.15.31:0/175280485 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa618027070 con 0x7fa6281024e0 2024-04-25T01:14:12.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.964+0000 7fa62ca00700 1 -- 172.21.15.31:0/175280485 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6180056d0 con 0x7fa6281024e0 2024-04-25T01:14:12.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.964+0000 7fa62de00700 1 -- 172.21.15.31:0/175280485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa62819b480 con 0x7fa6281024e0 2024-04-25T01:14:12.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.964+0000 7fa62ca00700 1 -- 172.21.15.31:0/175280485 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa618022e10 con 0x7fa6281024e0 2024-04-25T01:14:12.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.964+0000 7fa62de00700 1 -- 172.21.15.31:0/175280485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6281a77a0 con 0x7fa6281024e0 2024-04-25T01:14:12.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.965+0000 7fa62de00700 1 -- 172.21.15.31:0/175280485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa62810e710 con 0x7fa6281024e0 2024-04-25T01:14:12.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.967+0000 7fa62ca00700 1 -- 172.21.15.31:0/175280485 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa61802a070 con 0x7fa6281024e0 2024-04-25T01:14:12.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.969+0000 7fa62ca00700 1 --2- 172.21.15.31:0/175280485 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa60c0a2cb0 0x7fa60c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:12.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.969+0000 7fa62ca00700 1 -- 172.21.15.31:0/175280485 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fa6180cfd80 con 0x7fa6281024e0 2024-04-25T01:14:12.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.969+0000 7fa626000700 1 --2- 172.21.15.31:0/175280485 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa60c0a2cb0 0x7fa60c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:12.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.969+0000 7fa62ca00700 1 --2- 172.21.15.31:0/175280485 >> [v2:172.21.15.169:6816/2736546122,v1:172.21.15.169:6817/2736546122] conn(0x7fa60c0ac490 0x7fa60c0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:12.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.969+0000 7fa62ca00700 1 -- 172.21.15.31:0/175280485 --> [v2:172.21.15.169:6816/2736546122,v1:172.21.15.169:6817/2736546122] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa61801ae50 con 0x7fa60c0ac490 2024-04-25T01:14:12.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.969+0000 7fa62ca00700 1 -- 172.21.15.31:0/175280485 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fa6180d4030 con 0x7fa6281024e0 2024-04-25T01:14:12.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.970+0000 7fa626000700 1 --2- 172.21.15.31:0/175280485 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa60c0a2cb0 0x7fa60c0a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa614009680 tx=0x7fa614009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:12.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.970+0000 7fa627400700 1 --2- 172.21.15.31:0/175280485 >> [v2:172.21.15.169:6816/2736546122,v1:172.21.15.169:6817/2736546122] conn(0x7fa60c0ac490 0x7fa60c0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:12.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.971+0000 7fa627400700 1 --2- 172.21.15.31:0/175280485 >> [v2:172.21.15.169:6816/2736546122,v1:172.21.15.169:6817/2736546122] conn(0x7fa60c0ac490 0x7fa60c0ae8b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:12.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.972+0000 7fa62ca00700 1 -- 172.21.15.31:0/175280485 <== osd.8 v2:172.21.15.169:6816/2736546122 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa61801ae50 con 0x7fa60c0ac490 2024-04-25T01:14:12.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.991+0000 7fa62de00700 1 -- 172.21.15.31:0/175280485 --> [v2:172.21.15.169:6816/2736546122,v1:172.21.15.169:6817/2736546122] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa628103970 con 0x7fa60c0ac490 2024-04-25T01:14:12.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.992+0000 7fa62ca00700 1 -- 172.21.15.31:0/175280485 <== osd.8 v2:172.21.15.169:6816/2736546122 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa628103970 con 0x7fa60c0ac490 2024-04-25T01:14:12.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.992+0000 7fa62de00700 1 -- 172.21.15.31:0/175280485 >> [v2:172.21.15.169:6816/2736546122,v1:172.21.15.169:6817/2736546122] conn(0x7fa60c0ac490 msgr2=0x7fa60c0ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:12.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.992+0000 7fa62de00700 1 --2- 172.21.15.31:0/175280485 >> [v2:172.21.15.169:6816/2736546122,v1:172.21.15.169:6817/2736546122] conn(0x7fa60c0ac490 0x7fa60c0ae8b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:12.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.992+0000 7fa62de00700 1 -- 172.21.15.31:0/175280485 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa60c0a2cb0 msgr2=0x7fa60c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:12.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.992+0000 7fa62de00700 1 --2- 172.21.15.31:0/175280485 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa60c0a2cb0 0x7fa60c0a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa614009680 tx=0x7fa614009260 comp rx=0 tx=0).stop 2024-04-25T01:14:12.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.992+0000 7fa62de00700 1 -- 172.21.15.31:0/175280485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6281024e0 msgr2=0x7fa6281a08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:12.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.992+0000 7fa62de00700 1 --2- 172.21.15.31:0/175280485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6281024e0 0x7fa6281a08d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa618000c00 tx=0x7fa61800bab0 comp rx=0 tx=0).stop 2024-04-25T01:14:12.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.993+0000 7fa627400700 1 -- 172.21.15.31:0/175280485 reap_dead start 2024-04-25T01:14:12.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.993+0000 7fa62de00700 1 -- 172.21.15.31:0/175280485 shutdown_connections 2024-04-25T01:14:12.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.993+0000 7fa62de00700 1 -- 172.21.15.31:0/175280485 >> 172.21.15.31:0/175280485 conn(0x7fa628079390 msgr2=0x7fa628101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:12.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.993+0000 7fa62de00700 1 -- 172.21.15.31:0/175280485 shutdown_connections 2024-04-25T01:14:12.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:11.993+0000 7fa62de00700 1 -- 172.21.15.31:0/175280485 wait complete. 2024-04-25T01:14:13.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.325+0000 7fd08c800700 1 -- 172.21.15.31:0/3483193159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd084107570 msgr2=0x7fd08410f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:13.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.325+0000 7fd08c800700 1 --2- 172.21.15.31:0/3483193159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd084107570 0x7fd08410f140 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd08000b0a0 tx=0x7fd08000b3b0 comp rx=0 tx=0).stop 2024-04-25T01:14:13.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.325+0000 7fd08c800700 1 -- 172.21.15.31:0/3483193159 shutdown_connections 2024-04-25T01:14:13.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.325+0000 7fd08c800700 1 --2- 172.21.15.31:0/3483193159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd084107570 0x7fd08410f140 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:13.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.325+0000 7fd08c800700 1 --2- 172.21.15.31:0/3483193159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0841024e0 0x7fd0841028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:13.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.325+0000 7fd08c800700 1 --2- 172.21.15.31:0/3483193159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd084102e90 0x7fd084106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:13.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.325+0000 7fd08c800700 1 -- 172.21.15.31:0/3483193159 >> 172.21.15.31:0/3483193159 conn(0x7fd084079390 msgr2=0x7fd0840797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:13.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.325+0000 7fd08c800700 1 -- 172.21.15.31:0/3483193159 shutdown_connections 2024-04-25T01:14:13.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.325+0000 7fd08c800700 1 -- 172.21.15.31:0/3483193159 wait complete. 2024-04-25T01:14:13.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.326+0000 7fd08c800700 1 Processor -- start 2024-04-25T01:14:13.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.326+0000 7fd08c800700 1 -- start start 2024-04-25T01:14:13.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.327+0000 7fd08c800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0841024e0 0x7fd0841a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:13.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.327+0000 7fd08c800700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd084102e90 0x7fd0841a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:13.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.327+0000 7fd08c800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd084107570 0x7fd08419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:13.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.327+0000 7fd08c800700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd0841125a0 con 0x7fd0841024e0 2024-04-25T01:14:13.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.327+0000 7fd08c800700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd084112420 con 0x7fd084102e90 2024-04-25T01:14:13.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.327+0000 7fd08c800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd084112720 con 0x7fd084107570 2024-04-25T01:14:13.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.327+0000 7fd089e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0841024e0 0x7fd0841a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:13.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.327+0000 7fd08a800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd084107570 0x7fd08419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:13.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.327+0000 7fd08a800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd084107570 0x7fd08419aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:47934/0 (socket says 172.21.15.31:47934) 2024-04-25T01:14:13.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.327+0000 7fd08a800700 1 -- 172.21.15.31:0/4095068009 learned_addr learned my addr 172.21.15.31:0/4095068009 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:14:13.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.327+0000 7fd089e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0841024e0 0x7fd0841a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35492/0 (socket says 172.21.15.31:35492) 2024-04-25T01:14:13.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.327+0000 7fd089400700 1 --2- 172.21.15.31:0/4095068009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd084102e90 0x7fd0841a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:13.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.328+0000 7fd08a800700 1 -- 172.21.15.31:0/4095068009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd084102e90 msgr2=0x7fd0841a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:13.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.328+0000 7fd08a800700 1 --2- 172.21.15.31:0/4095068009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd084102e90 0x7fd0841a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:13.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.328+0000 7fd08a800700 1 -- 172.21.15.31:0/4095068009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0841024e0 msgr2=0x7fd0841a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:13.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.328+0000 7fd08a800700 1 --2- 172.21.15.31:0/4095068009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0841024e0 0x7fd0841a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:13.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.328+0000 7fd08a800700 1 -- 172.21.15.31:0/4095068009 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08419b160 con 0x7fd084107570 2024-04-25T01:14:13.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.328+0000 7fd08a800700 1 --2- 172.21.15.31:0/4095068009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd084107570 0x7fd08419aa00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd08000b070 tx=0x7fd080004c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:13.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.328+0000 7fd07ea00700 1 -- 172.21.15.31:0/4095068009 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080021070 con 0x7fd084107570 2024-04-25T01:14:13.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.328+0000 7fd07ea00700 1 -- 172.21.15.31:0/4095068009 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd080003c00 con 0x7fd084107570 2024-04-25T01:14:13.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.328+0000 7fd08c800700 1 -- 172.21.15.31:0/4095068009 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd08419b3f0 con 0x7fd084107570 2024-04-25T01:14:13.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.328+0000 7fd07ea00700 1 -- 172.21.15.31:0/4095068009 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080004740 con 0x7fd084107570 2024-04-25T01:14:13.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.329+0000 7fd08c800700 1 -- 172.21.15.31:0/4095068009 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0841a7710 con 0x7fd084107570 2024-04-25T01:14:13.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.329+0000 7fd08c800700 1 -- 172.21.15.31:0/4095068009 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd08410e780 con 0x7fd084107570 2024-04-25T01:14:13.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.332+0000 7fd07ea00700 1 -- 172.21.15.31:0/4095068009 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd080007980 con 0x7fd084107570 2024-04-25T01:14:13.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.333+0000 7fd07ea00700 1 --2- 172.21.15.31:0/4095068009 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd06c0a2d30 0x7fd06c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:13.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.333+0000 7fd07ea00700 1 -- 172.21.15.31:0/4095068009 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fd08000f020 con 0x7fd084107570 2024-04-25T01:14:13.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.333+0000 7fd089e00700 1 --2- 172.21.15.31:0/4095068009 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd06c0a2d30 0x7fd06c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:13.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.334+0000 7fd07ea00700 1 --2- 172.21.15.31:0/4095068009 >> [v2:172.21.15.40:6808/1913779091,v1:172.21.15.40:6809/1913779091] conn(0x7fd06c0ac510 0x7fd06c0ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:13.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.334+0000 7fd07ea00700 1 -- 172.21.15.31:0/4095068009 --> [v2:172.21.15.40:6808/1913779091,v1:172.21.15.40:6809/1913779091] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd06c0aefe0 con 0x7fd06c0ac510 2024-04-25T01:14:13.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.334+0000 7fd07ea00700 1 -- 172.21.15.31:0/4095068009 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fd0800d0940 con 0x7fd084107570 2024-04-25T01:14:13.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.334+0000 7fd089400700 1 --2- 172.21.15.31:0/4095068009 >> [v2:172.21.15.40:6808/1913779091,v1:172.21.15.40:6809/1913779091] conn(0x7fd06c0ac510 0x7fd06c0ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:13.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.334+0000 7fd089e00700 1 --2- 172.21.15.31:0/4095068009 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd06c0a2d30 0x7fd06c0a51f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd08419bb40 tx=0x7fd078009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:13.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.335+0000 7fd089400700 1 --2- 172.21.15.31:0/4095068009 >> [v2:172.21.15.40:6808/1913779091,v1:172.21.15.40:6809/1913779091] conn(0x7fd06c0ac510 0x7fd06c0ae930 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:13.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.336+0000 7fd07ea00700 1 -- 172.21.15.31:0/4095068009 <== osd.4 v2:172.21.15.40:6808/1913779091 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd06c0aefe0 con 0x7fd06c0ac510 2024-04-25T01:14:13.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.354+0000 7fd08c800700 1 -- 172.21.15.31:0/4095068009 --> [v2:172.21.15.40:6808/1913779091,v1:172.21.15.40:6809/1913779091] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd084103970 con 0x7fd06c0ac510 2024-04-25T01:14:13.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.355+0000 7fd07ea00700 1 -- 172.21.15.31:0/4095068009 <== osd.4 v2:172.21.15.40:6808/1913779091 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd084103970 con 0x7fd06c0ac510 2024-04-25T01:14:13.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.355+0000 7fd08c800700 1 -- 172.21.15.31:0/4095068009 >> [v2:172.21.15.40:6808/1913779091,v1:172.21.15.40:6809/1913779091] conn(0x7fd06c0ac510 msgr2=0x7fd06c0ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:13.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.355+0000 7fd08c800700 1 --2- 172.21.15.31:0/4095068009 >> [v2:172.21.15.40:6808/1913779091,v1:172.21.15.40:6809/1913779091] conn(0x7fd06c0ac510 0x7fd06c0ae930 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:13.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.356+0000 7fd08c800700 1 -- 172.21.15.31:0/4095068009 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd06c0a2d30 msgr2=0x7fd06c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:13.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.356+0000 7fd08c800700 1 --2- 172.21.15.31:0/4095068009 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd06c0a2d30 0x7fd06c0a51f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd08419bb40 tx=0x7fd078009260 comp rx=0 tx=0).stop 2024-04-25T01:14:13.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.356+0000 7fd08c800700 1 -- 172.21.15.31:0/4095068009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd084107570 msgr2=0x7fd08419aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:13.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.356+0000 7fd08c800700 1 --2- 172.21.15.31:0/4095068009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd084107570 0x7fd08419aa00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd08000b070 tx=0x7fd080004c60 comp rx=0 tx=0).stop 2024-04-25T01:14:13.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.356+0000 7fd08a800700 1 -- 172.21.15.31:0/4095068009 reap_dead start 2024-04-25T01:14:13.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.356+0000 7fd08c800700 1 -- 172.21.15.31:0/4095068009 shutdown_connections 2024-04-25T01:14:13.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.356+0000 7fd08c800700 1 -- 172.21.15.31:0/4095068009 >> 172.21.15.31:0/4095068009 conn(0x7fd084079390 msgr2=0x7fd084101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:13.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.356+0000 7fd08c800700 1 -- 172.21.15.31:0/4095068009 shutdown_connections 2024-04-25T01:14:13.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:12.356+0000 7fd08c800700 1 -- 172.21.15.31:0/4095068009 wait complete. 2024-04-25T01:14:13.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.724+0000 7fad2be00700 1 -- 172.21.15.31:0/3101022666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad2c104f50 msgr2=0x7fad2c108fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:13.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.724+0000 7fad2be00700 1 --2- 172.21.15.31:0/3101022666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad2c104f50 0x7fad2c108fa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fad140099c0 tx=0x7fad14009cd0 comp rx=0 tx=0).stop 2024-04-25T01:14:13.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.724+0000 7fad2be00700 1 -- 172.21.15.31:0/3101022666 shutdown_connections 2024-04-25T01:14:13.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.724+0000 7fad2be00700 1 --2- 172.21.15.31:0/3101022666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad2c109680 0x7fad2c111250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:13.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.724+0000 7fad2be00700 1 --2- 172.21.15.31:0/3101022666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad2c1045a0 0x7fad2c104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:13.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.724+0000 7fad2be00700 1 --2- 172.21.15.31:0/3101022666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad2c104f50 0x7fad2c108fa0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:13.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.724+0000 7fad2be00700 1 -- 172.21.15.31:0/3101022666 >> 172.21.15.31:0/3101022666 conn(0x7fad2c0ffe70 msgr2=0x7fad2c102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:13.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.724+0000 7fad2be00700 1 -- 172.21.15.31:0/3101022666 shutdown_connections 2024-04-25T01:14:13.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.724+0000 7fad2be00700 1 -- 172.21.15.31:0/3101022666 wait complete. 2024-04-25T01:14:13.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.725+0000 7fad2be00700 1 Processor -- start 2024-04-25T01:14:13.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.725+0000 7fad2be00700 1 -- start start 2024-04-25T01:14:13.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.726+0000 7fad2be00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad2c1045a0 0x7fad2c1a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:13.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.726+0000 7fad2be00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad2c104f50 0x7fad2c1a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:13.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.726+0000 7fad2be00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad2c109680 0x7fad2c19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:13.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.726+0000 7fad2be00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fad2c1146b0 con 0x7fad2c109680 2024-04-25T01:14:13.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.726+0000 7fad2be00700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fad2c114530 con 0x7fad2c1045a0 2024-04-25T01:14:13.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.726+0000 7fad2be00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fad2c114830 con 0x7fad2c104f50 2024-04-25T01:14:13.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.726+0000 7fad2b400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad2c109680 0x7fad2c19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:13.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.726+0000 7fad2b400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad2c109680 0x7fad2c19a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35500/0 (socket says 172.21.15.31:35500) 2024-04-25T01:14:13.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.726+0000 7fad2b400700 1 -- 172.21.15.31:0/1405763459 learned_addr learned my addr 172.21.15.31:0/1405763459 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:14:13.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.726+0000 7fad22000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad2c104f50 0x7fad2c1a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:13.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.726+0000 7fad2aa00700 1 --2- 172.21.15.31:0/1405763459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad2c1045a0 0x7fad2c1a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:13.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.727+0000 7fad22000700 1 -- 172.21.15.31:0/1405763459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad2c1045a0 msgr2=0x7fad2c1a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:13.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.727+0000 7fad22000700 1 --2- 172.21.15.31:0/1405763459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad2c1045a0 0x7fad2c1a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:13.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.727+0000 7fad22000700 1 -- 172.21.15.31:0/1405763459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad2c109680 msgr2=0x7fad2c19a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:13.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.727+0000 7fad22000700 1 --2- 172.21.15.31:0/1405763459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad2c109680 0x7fad2c19a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:13.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.727+0000 7fad22000700 1 -- 172.21.15.31:0/1405763459 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2c19b090 con 0x7fad2c104f50 2024-04-25T01:14:13.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.727+0000 7fad22000700 1 --2- 172.21.15.31:0/1405763459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad2c104f50 0x7fad2c1a0cb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fad140098e0 tx=0x7fad14003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:13.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.727+0000 7fad23e00700 1 -- 172.21.15.31:0/1405763459 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad14003680 con 0x7fad2c104f50 2024-04-25T01:14:13.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.727+0000 7fad23e00700 1 -- 172.21.15.31:0/1405763459 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad140055a0 con 0x7fad2c104f50 2024-04-25T01:14:13.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.727+0000 7fad23e00700 1 -- 172.21.15.31:0/1405763459 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1400fc60 con 0x7fad2c104f50 2024-04-25T01:14:13.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.727+0000 7fad2be00700 1 -- 172.21.15.31:0/1405763459 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad2c19b320 con 0x7fad2c104f50 2024-04-25T01:14:13.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.728+0000 7fad2be00700 1 -- 172.21.15.31:0/1405763459 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad2c1a7640 con 0x7fad2c104f50 2024-04-25T01:14:13.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.728+0000 7fad2be00700 1 -- 172.21.15.31:0/1405763459 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fad2c0629c0 con 0x7fad2c104f50 2024-04-25T01:14:13.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.731+0000 7fad23e00700 1 -- 172.21.15.31:0/1405763459 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fad1400f460 con 0x7fad2c104f50 2024-04-25T01:14:13.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.732+0000 7fad23e00700 1 --2- 172.21.15.31:0/1405763459 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fad100a2e50 0x7fad100a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:13.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.732+0000 7fad23e00700 1 -- 172.21.15.31:0/1405763459 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fad140d01d0 con 0x7fad2c104f50 2024-04-25T01:14:13.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.732+0000 7fad2aa00700 1 --2- 172.21.15.31:0/1405763459 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fad100a2e50 0x7fad100a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:13.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.733+0000 7fad23e00700 1 --2- 172.21.15.31:0/1405763459 >> [v2:172.21.15.31:6826/1187207880,v1:172.21.15.31:6827/1187207880] conn(0x7fad100ac630 0x7fad100aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:13.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.733+0000 7fad23e00700 1 -- 172.21.15.31:0/1405763459 --> [v2:172.21.15.31:6826/1187207880,v1:172.21.15.31:6827/1187207880] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fad14018e50 con 0x7fad100ac630 2024-04-25T01:14:13.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.733+0000 7fad23e00700 1 -- 172.21.15.31:0/1405763459 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fad140d0580 con 0x7fad2c104f50 2024-04-25T01:14:13.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.733+0000 7fad2b400700 1 --2- 172.21.15.31:0/1405763459 >> [v2:172.21.15.31:6826/1187207880,v1:172.21.15.31:6827/1187207880] conn(0x7fad100ac630 0x7fad100aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:13.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.733+0000 7fad2aa00700 1 --2- 172.21.15.31:0/1405763459 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fad100a2e50 0x7fad100a5310 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fad2c19ba70 tx=0x7fad1c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:13.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.734+0000 7fad2b400700 1 --2- 172.21.15.31:0/1405763459 >> [v2:172.21.15.31:6826/1187207880,v1:172.21.15.31:6827/1187207880] conn(0x7fad100ac630 0x7fad100aea50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:13.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.735+0000 7fad23e00700 1 -- 172.21.15.31:0/1405763459 <== osd.9 v2:172.21.15.31:6826/1187207880 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fad14018e50 con 0x7fad100ac630 2024-04-25T01:14:13.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.753+0000 7fad2be00700 1 -- 172.21.15.31:0/1405763459 --> [v2:172.21.15.31:6826/1187207880,v1:172.21.15.31:6827/1187207880] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fad2c062e40 con 0x7fad100ac630 2024-04-25T01:14:13.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.754+0000 7fad23e00700 1 -- 172.21.15.31:0/1405763459 <== osd.9 v2:172.21.15.31:6826/1187207880 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fad2c062e40 con 0x7fad100ac630 2024-04-25T01:14:13.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.754+0000 7fad2be00700 1 -- 172.21.15.31:0/1405763459 >> [v2:172.21.15.31:6826/1187207880,v1:172.21.15.31:6827/1187207880] conn(0x7fad100ac630 msgr2=0x7fad100aea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:13.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.754+0000 7fad2be00700 1 --2- 172.21.15.31:0/1405763459 >> [v2:172.21.15.31:6826/1187207880,v1:172.21.15.31:6827/1187207880] conn(0x7fad100ac630 0x7fad100aea50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:13.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.754+0000 7fad2be00700 1 -- 172.21.15.31:0/1405763459 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fad100a2e50 msgr2=0x7fad100a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:13.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.754+0000 7fad2be00700 1 --2- 172.21.15.31:0/1405763459 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fad100a2e50 0x7fad100a5310 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fad2c19ba70 tx=0x7fad1c00b040 comp rx=0 tx=0).stop 2024-04-25T01:14:13.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.754+0000 7fad2be00700 1 -- 172.21.15.31:0/1405763459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad2c104f50 msgr2=0x7fad2c1a0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:13.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.754+0000 7fad2be00700 1 --2- 172.21.15.31:0/1405763459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad2c104f50 0x7fad2c1a0cb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fad140098e0 tx=0x7fad14003a90 comp rx=0 tx=0).stop 2024-04-25T01:14:13.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.754+0000 7fad2b400700 1 -- 172.21.15.31:0/1405763459 reap_dead start 2024-04-25T01:14:13.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.754+0000 7fad2be00700 1 -- 172.21.15.31:0/1405763459 shutdown_connections 2024-04-25T01:14:13.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.754+0000 7fad2be00700 1 -- 172.21.15.31:0/1405763459 >> 172.21.15.31:0/1405763459 conn(0x7fad2c0ffe70 msgr2=0x7fad2c100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:13.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.754+0000 7fad2be00700 1 -- 172.21.15.31:0/1405763459 shutdown_connections 2024-04-25T01:14:13.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:13.755+0000 7fad2be00700 1 -- 172.21.15.31:0/1405763459 wait complete. 2024-04-25T01:14:14.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:13 smithi031 ceph-mon[28053]: pgmap v337: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T01:14:14.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:13 smithi169 ceph-mon[32759]: pgmap v337: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T01:14:14.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:13 smithi040 ceph-mon[32518]: pgmap v337: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T01:14:14.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.790+0000 7f6f01000700 1 -- 172.21.15.31:0/42841063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6efc069f40 msgr2=0x7f6efc06a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:14.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.790+0000 7f6f01000700 1 --2- 172.21.15.31:0/42841063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6efc069f40 0x7f6efc06a3a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009990 tx=0x7f6ee8009ca0 comp rx=0 tx=0).stop 2024-04-25T01:14:14.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.790+0000 7f6f01000700 1 -- 172.21.15.31:0/42841063 shutdown_connections 2024-04-25T01:14:14.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.790+0000 7f6f01000700 1 --2- 172.21.15.31:0/42841063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6efc06a8e0 0x7f6efc10f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:14.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.790+0000 7f6f01000700 1 --2- 172.21.15.31:0/42841063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6efc1082b0 0x7f6efc108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:14.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.790+0000 7f6f01000700 1 --2- 172.21.15.31:0/42841063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6efc069f40 0x7f6efc06a3a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:14.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.790+0000 7f6f01000700 1 -- 172.21.15.31:0/42841063 >> 172.21.15.31:0/42841063 conn(0x7f6efc0780e0 msgr2=0x7f6efc0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:14.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.790+0000 7f6f01000700 1 -- 172.21.15.31:0/42841063 shutdown_connections 2024-04-25T01:14:14.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.790+0000 7f6f01000700 1 -- 172.21.15.31:0/42841063 wait complete. 2024-04-25T01:14:14.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.792+0000 7f6f01000700 1 Processor -- start 2024-04-25T01:14:14.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.792+0000 7f6f01000700 1 -- start start 2024-04-25T01:14:14.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.792+0000 7f6f01000700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6efc069f40 0x7f6efc101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:14.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.792+0000 7f6f01000700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6efc06a8e0 0x7f6efc101740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:14.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.792+0000 7f6f01000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6efc1082b0 0x7f6efc101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:14.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.792+0000 7f6f01000700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc112560 con 0x7f6efc06a8e0 2024-04-25T01:14:14.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.792+0000 7f6f01000700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1123e0 con 0x7f6efc069f40 2024-04-25T01:14:14.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.792+0000 7f6f01000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1126e0 con 0x7f6efc1082b0 2024-04-25T01:14:14.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.792+0000 7f6ef9400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6efc06a8e0 0x7f6efc101740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:14.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.793+0000 7f6efa800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6efc1082b0 0x7f6efc101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:14.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.793+0000 7f6ef9400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6efc06a8e0 0x7f6efc101740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35528/0 (socket says 172.21.15.31:35528) 2024-04-25T01:14:14.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.793+0000 7f6ef9400700 1 -- 172.21.15.31:0/1257796795 learned_addr learned my addr 172.21.15.31:0/1257796795 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:14:14.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.793+0000 7f6ef9e00700 1 --2- 172.21.15.31:0/1257796795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6efc069f40 0x7f6efc101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:14.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.793+0000 7f6efa800700 1 -- 172.21.15.31:0/1257796795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6efc069f40 msgr2=0x7f6efc101200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:14.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.793+0000 7f6efa800700 1 --2- 172.21.15.31:0/1257796795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6efc069f40 0x7f6efc101200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:14.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.793+0000 7f6efa800700 1 -- 172.21.15.31:0/1257796795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6efc06a8e0 msgr2=0x7f6efc101740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:14.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.793+0000 7f6efa800700 1 --2- 172.21.15.31:0/1257796795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6efc06a8e0 0x7f6efc101740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:14.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.793+0000 7f6efa800700 1 -- 172.21.15.31:0/1257796795 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6efc1a6fa0 con 0x7f6efc1082b0 2024-04-25T01:14:14.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.793+0000 7f6efa800700 1 --2- 172.21.15.31:0/1257796795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6efc1082b0 0x7f6efc101c80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400efc0 tx=0x7f6ef400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:14.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.793+0000 7f6ef2a00700 1 -- 172.21.15.31:0/1257796795 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef40092e0 con 0x7f6efc1082b0 2024-04-25T01:14:14.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.794+0000 7f6ef2a00700 1 -- 172.21.15.31:0/1257796795 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ef4004bb0 con 0x7f6efc1082b0 2024-04-25T01:14:14.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.794+0000 7f6f01000700 1 -- 172.21.15.31:0/1257796795 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6efc1a7230 con 0x7f6efc1082b0 2024-04-25T01:14:14.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.794+0000 7f6ef2a00700 1 -- 172.21.15.31:0/1257796795 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4004280 con 0x7f6efc1082b0 2024-04-25T01:14:14.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.794+0000 7f6f01000700 1 -- 172.21.15.31:0/1257796795 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6efc1a7720 con 0x7f6efc1082b0 2024-04-25T01:14:14.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.795+0000 7f6f01000700 1 -- 172.21.15.31:0/1257796795 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6efc062940 con 0x7f6efc1082b0 2024-04-25T01:14:14.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.797+0000 7f6ef2a00700 1 -- 172.21.15.31:0/1257796795 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6ef401b030 con 0x7f6efc1082b0 2024-04-25T01:14:14.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.798+0000 7f6ef2a00700 1 --2- 172.21.15.31:0/1257796795 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6ee00a2cb0 0x7f6ee00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:14.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.798+0000 7f6ef2a00700 1 -- 172.21.15.31:0/1257796795 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f6ef40cec90 con 0x7f6efc1082b0 2024-04-25T01:14:14.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.798+0000 7f6ef9e00700 1 --2- 172.21.15.31:0/1257796795 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6ee00a2cb0 0x7f6ee00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:14.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.798+0000 7f6ef2a00700 1 --2- 172.21.15.31:0/1257796795 >> [v2:172.21.15.169:6824/262401944,v1:172.21.15.169:6825/262401944] conn(0x7f6ee00ac490 0x7f6ee00ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:14.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.798+0000 7f6ef2a00700 1 -- 172.21.15.31:0/1257796795 --> [v2:172.21.15.169:6824/262401944,v1:172.21.15.169:6825/262401944] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6ef400be50 con 0x7f6ee00ac490 2024-04-25T01:14:14.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.798+0000 7f6ef2a00700 1 -- 172.21.15.31:0/1257796795 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f6ef40d42a0 con 0x7f6efc1082b0 2024-04-25T01:14:14.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.799+0000 7f6ef9400700 1 --2- 172.21.15.31:0/1257796795 >> [v2:172.21.15.169:6824/262401944,v1:172.21.15.169:6825/262401944] conn(0x7f6ee00ac490 0x7f6ee00ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:14.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.799+0000 7f6ef9e00700 1 --2- 172.21.15.31:0/1257796795 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6ee00a2cb0 0x7f6ee00a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6eec009be0 tx=0x7f6eec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:14.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.799+0000 7f6ef9400700 1 --2- 172.21.15.31:0/1257796795 >> [v2:172.21.15.169:6824/262401944,v1:172.21.15.169:6825/262401944] conn(0x7f6ee00ac490 0x7f6ee00ae8b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:14.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.800+0000 7f6ef2a00700 1 -- 172.21.15.31:0/1257796795 <== osd.11 v2:172.21.15.169:6824/262401944 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6ef400be50 con 0x7f6ee00ac490 2024-04-25T01:14:14.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.819+0000 7f6f01000700 1 -- 172.21.15.31:0/1257796795 --> [v2:172.21.15.169:6824/262401944,v1:172.21.15.169:6825/262401944] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6efc062dc0 con 0x7f6ee00ac490 2024-04-25T01:14:14.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.819+0000 7f6ef2a00700 1 -- 172.21.15.31:0/1257796795 <== osd.11 v2:172.21.15.169:6824/262401944 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6efc062dc0 con 0x7f6ee00ac490 2024-04-25T01:14:14.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.820+0000 7f6f01000700 1 -- 172.21.15.31:0/1257796795 >> [v2:172.21.15.169:6824/262401944,v1:172.21.15.169:6825/262401944] conn(0x7f6ee00ac490 msgr2=0x7f6ee00ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:14.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.820+0000 7f6f01000700 1 --2- 172.21.15.31:0/1257796795 >> [v2:172.21.15.169:6824/262401944,v1:172.21.15.169:6825/262401944] conn(0x7f6ee00ac490 0x7f6ee00ae8b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:14.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.820+0000 7f6f01000700 1 -- 172.21.15.31:0/1257796795 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6ee00a2cb0 msgr2=0x7f6ee00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:14.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.820+0000 7f6f01000700 1 --2- 172.21.15.31:0/1257796795 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6ee00a2cb0 0x7f6ee00a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6eec009be0 tx=0x7f6eec009260 comp rx=0 tx=0).stop 2024-04-25T01:14:14.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.820+0000 7f6f01000700 1 -- 172.21.15.31:0/1257796795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6efc1082b0 msgr2=0x7f6efc101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:14.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.820+0000 7f6f01000700 1 --2- 172.21.15.31:0/1257796795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6efc1082b0 0x7f6efc101c80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400efc0 tx=0x7f6ef400c3a0 comp rx=0 tx=0).stop 2024-04-25T01:14:14.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.820+0000 7f6efa800700 1 -- 172.21.15.31:0/1257796795 reap_dead start 2024-04-25T01:14:14.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.820+0000 7f6f01000700 1 -- 172.21.15.31:0/1257796795 shutdown_connections 2024-04-25T01:14:14.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.820+0000 7f6f01000700 1 -- 172.21.15.31:0/1257796795 >> 172.21.15.31:0/1257796795 conn(0x7f6efc0780e0 msgr2=0x7f6efc0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:14.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.820+0000 7f6f01000700 1 -- 172.21.15.31:0/1257796795 shutdown_connections 2024-04-25T01:14:14.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.821+0000 7f6f01000700 1 -- 172.21.15.31:0/1257796795 wait complete. 2024-04-25T01:14:15.438 INFO:teuthology.orchestra.run.smithi031.stdout:150323855422 2024-04-25T01:14:15.439 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T01:14:15.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.918+0000 7f5ab6e00700 1 -- 172.21.15.31:0/1005468721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab0103c40 msgr2=0x7f5ab010c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:15.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.918+0000 7f5ab6e00700 1 --2- 172.21.15.31:0/1005468721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab0103c40 0x7f5ab010c130 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009990 tx=0x7f5a9c009ca0 comp rx=0 tx=0).stop 2024-04-25T01:14:15.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.918+0000 7f5ab6e00700 1 -- 172.21.15.31:0/1005468721 shutdown_connections 2024-04-25T01:14:15.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.918+0000 7f5ab6e00700 1 --2- 172.21.15.31:0/1005468721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ab0103320 0x7f5ab0103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:15.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.918+0000 7f5ab6e00700 1 --2- 172.21.15.31:0/1005468721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ab010c750 0x7f5ab010ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:15.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.918+0000 7f5ab6e00700 1 --2- 172.21.15.31:0/1005468721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab0103c40 0x7f5ab010c130 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:15.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.918+0000 7f5ab6e00700 1 -- 172.21.15.31:0/1005468721 >> 172.21.15.31:0/1005468721 conn(0x7f5ab00fcb50 msgr2=0x7f5ab00fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:15.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.918+0000 7f5ab6e00700 1 -- 172.21.15.31:0/1005468721 shutdown_connections 2024-04-25T01:14:15.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.918+0000 7f5ab6e00700 1 -- 172.21.15.31:0/1005468721 wait complete. 2024-04-25T01:14:15.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.919+0000 7f5ab6e00700 1 Processor -- start 2024-04-25T01:14:15.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.919+0000 7f5ab6e00700 1 -- start start 2024-04-25T01:14:15.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.920+0000 7f5ab6e00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab0103320 0x7f5ab019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:15.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.920+0000 7f5ab6e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ab0103c40 0x7f5ab019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:15.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.920+0000 7f5ab6e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ab010c750 0x7f5ab0198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:15.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.920+0000 7f5ab6e00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab01124f0 con 0x7f5ab0103c40 2024-04-25T01:14:15.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.920+0000 7f5ab6e00700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab0112370 con 0x7f5ab0103320 2024-04-25T01:14:15.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.920+0000 7f5ab6e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab0112670 con 0x7f5ab010c750 2024-04-25T01:14:15.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.920+0000 7f5ab5000700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ab0103c40 0x7f5ab019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:15.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.920+0000 7f5ab5000700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ab0103c40 0x7f5ab019eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35548/0 (socket says 172.21.15.31:35548) 2024-04-25T01:14:15.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.920+0000 7f5ab5000700 1 -- 172.21.15.31:0/1190034806 learned_addr learned my addr 172.21.15.31:0/1190034806 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:14:15.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.920+0000 7f5ab6400700 1 --2- 172.21.15.31:0/1190034806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ab010c750 0x7f5ab0198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:15.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.920+0000 7f5ab5a00700 1 --2- 172.21.15.31:0/1190034806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab0103320 0x7f5ab019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:15.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.921+0000 7f5ab5000700 1 -- 172.21.15.31:0/1190034806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ab010c750 msgr2=0x7f5ab0198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:15.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.921+0000 7f5ab5000700 1 --2- 172.21.15.31:0/1190034806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ab010c750 0x7f5ab0198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:15.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.921+0000 7f5ab5000700 1 -- 172.21.15.31:0/1190034806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab0103320 msgr2=0x7f5ab019e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:15.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.921+0000 7f5ab5000700 1 --2- 172.21.15.31:0/1190034806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab0103320 0x7f5ab019e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.921+0000 7f5ab5000700 1 -- 172.21.15.31:0/1190034806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab0198ff0 con 0x7f5ab0103c40 2024-04-25T01:14:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.921+0000 7f5ab6400700 1 --2- 172.21.15.31:0/1190034806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ab010c750 0x7f5ab0198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:14:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.921+0000 7f5ab5000700 1 --2- 172.21.15.31:0/1190034806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ab0103c40 0x7f5ab019eb80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00b2e0 tx=0x7f5a9c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.921+0000 7f5aae000700 1 -- 172.21.15.31:0/1190034806 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c004800 con 0x7f5ab0103c40 2024-04-25T01:14:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.921+0000 7f5aae000700 1 -- 172.21.15.31:0/1190034806 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a9c005030 con 0x7f5ab0103c40 2024-04-25T01:14:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.921+0000 7f5ab6e00700 1 -- 172.21.15.31:0/1190034806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab0199280 con 0x7f5ab0103c40 2024-04-25T01:14:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.921+0000 7f5aae000700 1 -- 172.21.15.31:0/1190034806 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c003a40 con 0x7f5ab0103c40 2024-04-25T01:14:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.922+0000 7f5ab6e00700 1 -- 172.21.15.31:0/1190034806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab01a54a0 con 0x7f5ab0103c40 2024-04-25T01:14:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.923+0000 7f5ab6e00700 1 -- 172.21.15.31:0/1190034806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ab0062900 con 0x7f5ab0103c40 2024-04-25T01:14:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.925+0000 7f5aae000700 1 -- 172.21.15.31:0/1190034806 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5a9c009d10 con 0x7f5ab0103c40 2024-04-25T01:14:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.926+0000 7f5aae000700 1 --2- 172.21.15.31:0/1190034806 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5a980a2cb0 0x7f5a980a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.926+0000 7f5aae000700 1 -- 172.21.15.31:0/1190034806 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f5a9c0d04a0 con 0x7f5ab0103c40 2024-04-25T01:14:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.926+0000 7f5ab5a00700 1 --2- 172.21.15.31:0/1190034806 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5a980a2cb0 0x7f5a980a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.927+0000 7f5aae000700 1 --2- 172.21.15.31:0/1190034806 >> [v2:172.21.15.31:6818/611507624,v1:172.21.15.31:6819/611507624] conn(0x7f5a980ac490 0x7f5a980ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.927+0000 7f5aae000700 1 -- 172.21.15.31:0/1190034806 --> [v2:172.21.15.31:6818/611507624,v1:172.21.15.31:6819/611507624] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5a9c0043d0 con 0x7f5a980ac490 2024-04-25T01:14:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.927+0000 7f5aae000700 1 -- 172.21.15.31:0/1190034806 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f5a9c0d0850 con 0x7f5ab0103c40 2024-04-25T01:14:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.927+0000 7f5ab6400700 1 --2- 172.21.15.31:0/1190034806 >> [v2:172.21.15.31:6818/611507624,v1:172.21.15.31:6819/611507624] conn(0x7f5a980ac490 0x7f5a980ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.927+0000 7f5ab5a00700 1 --2- 172.21.15.31:0/1190034806 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5a980a2cb0 0x7f5a980a5170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009c60 tx=0x7f5aa4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.928+0000 7f5ab6400700 1 --2- 172.21.15.31:0/1190034806 >> [v2:172.21.15.31:6818/611507624,v1:172.21.15.31:6819/611507624] conn(0x7f5a980ac490 0x7f5a980ae8b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.929+0000 7f5aae000700 1 -- 172.21.15.31:0/1190034806 <== osd.6 v2:172.21.15.31:6818/611507624 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5a9c0043d0 con 0x7f5a980ac490 2024-04-25T01:14:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.947+0000 7f5ab6e00700 1 -- 172.21.15.31:0/1190034806 --> [v2:172.21.15.31:6818/611507624,v1:172.21.15.31:6819/611507624] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5ab0062d80 con 0x7f5a980ac490 2024-04-25T01:14:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.948+0000 7f5aae000700 1 -- 172.21.15.31:0/1190034806 <== osd.6 v2:172.21.15.31:6818/611507624 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5ab0062d80 con 0x7f5a980ac490 2024-04-25T01:14:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.948+0000 7f5ab6e00700 1 -- 172.21.15.31:0/1190034806 >> [v2:172.21.15.31:6818/611507624,v1:172.21.15.31:6819/611507624] conn(0x7f5a980ac490 msgr2=0x7f5a980ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.948+0000 7f5ab6e00700 1 --2- 172.21.15.31:0/1190034806 >> [v2:172.21.15.31:6818/611507624,v1:172.21.15.31:6819/611507624] conn(0x7f5a980ac490 0x7f5a980ae8b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:15.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.948+0000 7f5ab6e00700 1 -- 172.21.15.31:0/1190034806 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5a980a2cb0 msgr2=0x7f5a980a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:15.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.948+0000 7f5ab6e00700 1 --2- 172.21.15.31:0/1190034806 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5a980a2cb0 0x7f5a980a5170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009c60 tx=0x7f5aa4009260 comp rx=0 tx=0).stop 2024-04-25T01:14:15.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.949+0000 7f5ab6e00700 1 -- 172.21.15.31:0/1190034806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ab0103c40 msgr2=0x7f5ab019eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:15.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.949+0000 7f5ab6e00700 1 --2- 172.21.15.31:0/1190034806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ab0103c40 0x7f5ab019eb80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00b2e0 tx=0x7f5a9c004790 comp rx=0 tx=0).stop 2024-04-25T01:14:15.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.949+0000 7f5ab6400700 1 -- 172.21.15.31:0/1190034806 reap_dead start 2024-04-25T01:14:15.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.949+0000 7f5ab6e00700 1 -- 172.21.15.31:0/1190034806 shutdown_connections 2024-04-25T01:14:15.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.949+0000 7f5ab6e00700 1 -- 172.21.15.31:0/1190034806 >> 172.21.15.31:0/1190034806 conn(0x7f5ab00fcb50 msgr2=0x7f5ab00fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:15.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.949+0000 7f5ab6e00700 1 -- 172.21.15.31:0/1190034806 shutdown_connections 2024-04-25T01:14:15.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:14.949+0000 7f5ab6e00700 1 -- 172.21.15.31:0/1190034806 wait complete. 2024-04-25T01:14:15.732 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:15 smithi031 ceph-mon[28053]: pgmap v338: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:14:16.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:15 smithi169 ceph-mon[32759]: pgmap v338: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:14:16.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:15 smithi040 ceph-mon[32518]: pgmap v338: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:14:17.773 INFO:teuthology.orchestra.run.smithi031.stdout:206158430259 2024-04-25T01:14:17.773 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T01:14:17.997 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:17 smithi169 ceph-mon[32759]: pgmap v339: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T01:14:18.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.016+0000 7f73a3600700 1 -- 172.21.15.31:0/1069366707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f739c102e90 msgr2=0x7f739c106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:18.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.016+0000 7f73a3600700 1 --2- 172.21.15.31:0/1069366707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f739c102e90 0x7f739c106e90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f73880098f0 tx=0x7f7388009c00 comp rx=0 tx=0).stop 2024-04-25T01:14:18.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.016+0000 7f73a3600700 1 -- 172.21.15.31:0/1069366707 shutdown_connections 2024-04-25T01:14:18.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.016+0000 7f73a3600700 1 --2- 172.21.15.31:0/1069366707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f739c107570 0x7f739c10f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:18.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.016+0000 7f73a3600700 1 --2- 172.21.15.31:0/1069366707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f739c1024e0 0x7f739c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:18.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.016+0000 7f73a3600700 1 --2- 172.21.15.31:0/1069366707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f739c102e90 0x7f739c106e90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:18.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.016+0000 7f73a3600700 1 -- 172.21.15.31:0/1069366707 >> 172.21.15.31:0/1069366707 conn(0x7f739c079390 msgr2=0x7f739c0797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:18.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.016+0000 7f73a3600700 1 -- 172.21.15.31:0/1069366707 shutdown_connections 2024-04-25T01:14:18.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.016+0000 7f73a3600700 1 -- 172.21.15.31:0/1069366707 wait complete. 2024-04-25T01:14:18.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:17 smithi031 ceph-mon[28053]: pgmap v339: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T01:14:18.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.017+0000 7f73a3600700 1 Processor -- start 2024-04-25T01:14:18.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.018+0000 7f73a3600700 1 -- start start 2024-04-25T01:14:18.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.018+0000 7f73a3600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f739c1024e0 0x7f739c19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:18.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.018+0000 7f73a3600700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f739c102e90 0x7f739c19ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:18.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.018+0000 7f73a3600700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f739c107570 0x7f739c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:18.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.018+0000 7f73a3600700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f739c112580 con 0x7f739c102e90 2024-04-25T01:14:18.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.018+0000 7f73a3600700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f739c112400 con 0x7f739c107570 2024-04-25T01:14:18.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.018+0000 7f73a3600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f739c112700 con 0x7f739c1024e0 2024-04-25T01:14:18.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.019+0000 7f73a0c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f739c1024e0 0x7f739c19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:18.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.019+0000 7f73a0c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f739c1024e0 0x7f739c19e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:45010/0 (socket says 172.21.15.31:45010) 2024-04-25T01:14:18.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.019+0000 7f73a0c00700 1 -- 172.21.15.31:0/2978406920 learned_addr learned my addr 172.21.15.31:0/2978406920 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:14:18.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.019+0000 7f73a1600700 1 --2- 172.21.15.31:0/2978406920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f739c107570 0x7f739c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:18.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.019+0000 7f739be00700 1 --2- 172.21.15.31:0/2978406920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f739c102e90 0x7f739c19ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:18.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.019+0000 7f73a0c00700 1 -- 172.21.15.31:0/2978406920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f739c107570 msgr2=0x7f739c1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:18.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.019+0000 7f73a0c00700 1 --2- 172.21.15.31:0/2978406920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f739c107570 0x7f739c1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:18.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.019+0000 7f73a0c00700 1 -- 172.21.15.31:0/2978406920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f739c102e90 msgr2=0x7f739c19ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:18.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.019+0000 7f73a0c00700 1 --2- 172.21.15.31:0/2978406920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f739c102e90 0x7f739c19ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:18.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.019+0000 7f73a0c00700 1 -- 172.21.15.31:0/2978406920 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739c199020 con 0x7f739c1024e0 2024-04-25T01:14:18.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.019+0000 7f73a0c00700 1 --2- 172.21.15.31:0/2978406920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f739c1024e0 0x7f739c19e700 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f739000e9c0 tx=0x7f739000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:18.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.020+0000 7f7399600700 1 -- 172.21.15.31:0/2978406920 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739000cbd0 con 0x7f739c1024e0 2024-04-25T01:14:18.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.020+0000 7f73a3600700 1 -- 172.21.15.31:0/2978406920 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f739c199310 con 0x7f739c1024e0 2024-04-25T01:14:18.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.020+0000 7f7399600700 1 -- 172.21.15.31:0/2978406920 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f739000ee60 con 0x7f739c1024e0 2024-04-25T01:14:18.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.020+0000 7f7399600700 1 -- 172.21.15.31:0/2978406920 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390005630 con 0x7f739c1024e0 2024-04-25T01:14:18.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.020+0000 7f73a3600700 1 -- 172.21.15.31:0/2978406920 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f739c1a55d0 con 0x7f739c1024e0 2024-04-25T01:14:18.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.021+0000 7f73a3600700 1 -- 172.21.15.31:0/2978406920 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f739c0629c0 con 0x7f739c1024e0 2024-04-25T01:14:18.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.023+0000 7f7399600700 1 -- 172.21.15.31:0/2978406920 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7390044b60 con 0x7f739c1024e0 2024-04-25T01:14:18.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.025+0000 7f7399600700 1 --2- 172.21.15.31:0/2978406920 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f73840a2d30 0x7f73840a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:18.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.025+0000 7f7399600700 1 -- 172.21.15.31:0/2978406920 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f73900cf440 con 0x7f739c1024e0 2024-04-25T01:14:18.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.025+0000 7f739be00700 1 --2- 172.21.15.31:0/2978406920 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f73840a2d30 0x7f73840a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:18.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.025+0000 7f7399600700 1 --2- 172.21.15.31:0/2978406920 >> [v2:172.21.15.169:6800/1955211197,v1:172.21.15.169:6801/1955211197] conn(0x7f73840ac510 0x7f73840ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:18.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.025+0000 7f7399600700 1 -- 172.21.15.31:0/2978406920 --> [v2:172.21.15.169:6800/1955211197,v1:172.21.15.169:6801/1955211197] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7390044e50 con 0x7f73840ac510 2024-04-25T01:14:18.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.025+0000 7f7399600700 1 -- 172.21.15.31:0/2978406920 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f73900cf7f0 con 0x7f739c1024e0 2024-04-25T01:14:18.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.026+0000 7f73a1600700 1 --2- 172.21.15.31:0/2978406920 >> [v2:172.21.15.169:6800/1955211197,v1:172.21.15.169:6801/1955211197] conn(0x7f73840ac510 0x7f73840ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:18.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.026+0000 7f739be00700 1 --2- 172.21.15.31:0/2978406920 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f73840a2d30 0x7f73840a51f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f73880098c0 tx=0x7f7388019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:18.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.026+0000 7f73a1600700 1 --2- 172.21.15.31:0/2978406920 >> [v2:172.21.15.169:6800/1955211197,v1:172.21.15.169:6801/1955211197] conn(0x7f73840ac510 0x7f73840ae930 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:18.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.027+0000 7f7399600700 1 -- 172.21.15.31:0/2978406920 <== osd.2 v2:172.21.15.169:6800/1955211197 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7390044e50 con 0x7f73840ac510 2024-04-25T01:14:18.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.046+0000 7f73a3600700 1 -- 172.21.15.31:0/2978406920 --> [v2:172.21.15.169:6800/1955211197,v1:172.21.15.169:6801/1955211197] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f739c062e40 con 0x7f73840ac510 2024-04-25T01:14:18.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.046+0000 7f7399600700 1 -- 172.21.15.31:0/2978406920 <== osd.2 v2:172.21.15.169:6800/1955211197 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f739c062e40 con 0x7f73840ac510 2024-04-25T01:14:18.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.047+0000 7f73a3600700 1 -- 172.21.15.31:0/2978406920 >> [v2:172.21.15.169:6800/1955211197,v1:172.21.15.169:6801/1955211197] conn(0x7f73840ac510 msgr2=0x7f73840ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:18.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.047+0000 7f73a3600700 1 --2- 172.21.15.31:0/2978406920 >> [v2:172.21.15.169:6800/1955211197,v1:172.21.15.169:6801/1955211197] conn(0x7f73840ac510 0x7f73840ae930 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:18.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.047+0000 7f73a3600700 1 -- 172.21.15.31:0/2978406920 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f73840a2d30 msgr2=0x7f73840a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:18.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.047+0000 7f73a3600700 1 --2- 172.21.15.31:0/2978406920 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f73840a2d30 0x7f73840a51f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f73880098c0 tx=0x7f7388019040 comp rx=0 tx=0).stop 2024-04-25T01:14:18.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.047+0000 7f73a3600700 1 -- 172.21.15.31:0/2978406920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f739c1024e0 msgr2=0x7f739c19e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:18.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.047+0000 7f73a3600700 1 --2- 172.21.15.31:0/2978406920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f739c1024e0 0x7f739c19e700 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f739000e9c0 tx=0x7f739000ecd0 comp rx=0 tx=0).stop 2024-04-25T01:14:18.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.047+0000 7f73a1600700 1 -- 172.21.15.31:0/2978406920 reap_dead start 2024-04-25T01:14:18.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.047+0000 7f73a3600700 1 -- 172.21.15.31:0/2978406920 shutdown_connections 2024-04-25T01:14:18.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.047+0000 7f73a3600700 1 -- 172.21.15.31:0/2978406920 >> 172.21.15.31:0/2978406920 conn(0x7f739c079390 msgr2=0x7f739c1086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:18.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.047+0000 7f73a3600700 1 -- 172.21.15.31:0/2978406920 shutdown_connections 2024-04-25T01:14:18.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.047+0000 7f73a3600700 1 -- 172.21.15.31:0/2978406920 wait complete. 2024-04-25T01:14:18.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:17 smithi040 ceph-mon[32518]: pgmap v339: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T01:14:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.240+0000 7f6bf6800700 1 -- 172.21.15.31:0/2912426354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bf0109220 msgr2=0x7f6bf0115730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.240+0000 7f6bf6800700 1 --2- 172.21.15.31:0/2912426354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bf0109220 0x7f6bf0115730 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6be800b0a0 tx=0x7f6be800b3b0 comp rx=0 tx=0).stop 2024-04-25T01:14:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.241+0000 7f6bf6800700 1 -- 172.21.15.31:0/2912426354 shutdown_connections 2024-04-25T01:14:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.241+0000 7f6bf6800700 1 --2- 172.21.15.31:0/2912426354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bf0109220 0x7f6bf0115730 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.241+0000 7f6bf6800700 1 --2- 172.21.15.31:0/2912426354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bf010e860 0x7f6bf010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.241+0000 7f6bf6800700 1 --2- 172.21.15.31:0/2912426354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bf0108860 0x7f6bf0108ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.241+0000 7f6bf6800700 1 -- 172.21.15.31:0/2912426354 >> 172.21.15.31:0/2912426354 conn(0x7f6bf0075a00 msgr2=0x7f6bf0107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:18.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.241+0000 7f6bf6800700 1 -- 172.21.15.31:0/2912426354 shutdown_connections 2024-04-25T01:14:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.241+0000 7f6bf6800700 1 -- 172.21.15.31:0/2912426354 wait complete. 2024-04-25T01:14:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.242+0000 7f6bf6800700 1 Processor -- start 2024-04-25T01:14:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.242+0000 7f6bf6800700 1 -- start start 2024-04-25T01:14:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.243+0000 7f6bf6800700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bf0108860 0x7f6bf019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.243+0000 7f6bf6800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bf0109220 0x7f6bf019f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.243+0000 7f6bf6800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bf010e860 0x7f6bf01a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.243+0000 7f6bf6800700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf0118be0 con 0x7f6bf0109220 2024-04-25T01:14:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.243+0000 7f6bf6800700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf0118a60 con 0x7f6bf0108860 2024-04-25T01:14:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.243+0000 7f6bf6800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf0118d60 con 0x7f6bf010e860 2024-04-25T01:14:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.243+0000 7f6be7e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bf0109220 0x7f6bf019f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.243+0000 7f6be7e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bf0109220 0x7f6bf019f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54978/0 (socket says 172.21.15.31:54978) 2024-04-25T01:14:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.243+0000 7f6be7e00700 1 -- 172.21.15.31:0/3855737826 learned_addr learned my addr 172.21.15.31:0/3855737826 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:14:18.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.243+0000 7f6befe00700 1 --2- 172.21.15.31:0/3855737826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bf010e860 0x7f6bf01a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.243+0000 7f6bef400700 1 --2- 172.21.15.31:0/3855737826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bf0108860 0x7f6bf019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.244+0000 7f6be7e00700 1 -- 172.21.15.31:0/3855737826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bf010e860 msgr2=0x7f6bf01a3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.244+0000 7f6be7e00700 1 --2- 172.21.15.31:0/3855737826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bf010e860 0x7f6bf01a3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.244+0000 7f6be7e00700 1 -- 172.21.15.31:0/3855737826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bf0108860 msgr2=0x7f6bf019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.244+0000 7f6be7e00700 1 --2- 172.21.15.31:0/3855737826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bf0108860 0x7f6bf019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.244+0000 7f6be7e00700 1 -- 172.21.15.31:0/3855737826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf01a3f70 con 0x7f6bf0109220 2024-04-25T01:14:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.244+0000 7f6be7e00700 1 --2- 172.21.15.31:0/3855737826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bf0109220 0x7f6bf019f4f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6bd800e9a0 tx=0x7f6bd800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.244+0000 7f6becc00700 1 -- 172.21.15.31:0/3855737826 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd800ca10 con 0x7f6bf0109220 2024-04-25T01:14:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.244+0000 7f6becc00700 1 -- 172.21.15.31:0/3855737826 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bd800cbb0 con 0x7f6bf0109220 2024-04-25T01:14:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.244+0000 7f6becc00700 1 -- 172.21.15.31:0/3855737826 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8018680 con 0x7f6bf0109220 2024-04-25T01:14:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.244+0000 7f6bf6800700 1 -- 172.21.15.31:0/3855737826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bf01a4260 con 0x7f6bf0109220 2024-04-25T01:14:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.244+0000 7f6bf6800700 1 -- 172.21.15.31:0/3855737826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bf01abae0 con 0x7f6bf0109220 2024-04-25T01:14:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.245+0000 7f6bf6800700 1 -- 172.21.15.31:0/3855737826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6bf0076940 con 0x7f6bf0109220 2024-04-25T01:14:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.247+0000 7f6becc00700 1 -- 172.21.15.31:0/3855737826 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6bd8010430 con 0x7f6bf0109220 2024-04-25T01:14:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.249+0000 7f6becc00700 1 --2- 172.21.15.31:0/3855737826 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6bd40a2cb0 0x7f6bd40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.249+0000 7f6becc00700 1 -- 172.21.15.31:0/3855737826 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f6bd80cfdc0 con 0x7f6bf0109220 2024-04-25T01:14:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.249+0000 7f6bef400700 1 --2- 172.21.15.31:0/3855737826 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6bd40a2cb0 0x7f6bd40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.249+0000 7f6becc00700 1 --2- 172.21.15.31:0/3855737826 >> [v2:172.21.15.31:6802/788174091,v1:172.21.15.31:6803/788174091] conn(0x7f6bd40ac490 0x7f6bd40ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.249+0000 7f6becc00700 1 -- 172.21.15.31:0/3855737826 --> [v2:172.21.15.31:6802/788174091,v1:172.21.15.31:6803/788174091] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6bd40aef60 con 0x7f6bd40ac490 2024-04-25T01:14:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.249+0000 7f6becc00700 1 -- 172.21.15.31:0/3855737826 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f6bd80d02a0 con 0x7f6bf0109220 2024-04-25T01:14:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.250+0000 7f6befe00700 1 --2- 172.21.15.31:0/3855737826 >> [v2:172.21.15.31:6802/788174091,v1:172.21.15.31:6803/788174091] conn(0x7f6bd40ac490 0x7f6bd40ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.250+0000 7f6bef400700 1 --2- 172.21.15.31:0/3855737826 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6bd40a2cb0 0x7f6bd40a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6be0009be0 tx=0x7f6be0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.250+0000 7f6befe00700 1 --2- 172.21.15.31:0/3855737826 >> [v2:172.21.15.31:6802/788174091,v1:172.21.15.31:6803/788174091] conn(0x7f6bd40ac490 0x7f6bd40ae8b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:18.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.251+0000 7f6becc00700 1 -- 172.21.15.31:0/3855737826 <== osd.0 v2:172.21.15.31:6802/788174091 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6bd40aef60 con 0x7f6bd40ac490 2024-04-25T01:14:18.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.270+0000 7f6bf6800700 1 -- 172.21.15.31:0/3855737826 --> [v2:172.21.15.31:6802/788174091,v1:172.21.15.31:6803/788174091] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6bf0109fa0 con 0x7f6bd40ac490 2024-04-25T01:14:18.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.271+0000 7f6becc00700 1 -- 172.21.15.31:0/3855737826 <== osd.0 v2:172.21.15.31:6802/788174091 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6bf0109fa0 con 0x7f6bd40ac490 2024-04-25T01:14:18.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.271+0000 7f6bf6800700 1 -- 172.21.15.31:0/3855737826 >> [v2:172.21.15.31:6802/788174091,v1:172.21.15.31:6803/788174091] conn(0x7f6bd40ac490 msgr2=0x7f6bd40ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:18.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.271+0000 7f6bf6800700 1 --2- 172.21.15.31:0/3855737826 >> [v2:172.21.15.31:6802/788174091,v1:172.21.15.31:6803/788174091] conn(0x7f6bd40ac490 0x7f6bd40ae8b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:18.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.272+0000 7f6bf6800700 1 -- 172.21.15.31:0/3855737826 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6bd40a2cb0 msgr2=0x7f6bd40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:18.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.272+0000 7f6bf6800700 1 --2- 172.21.15.31:0/3855737826 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6bd40a2cb0 0x7f6bd40a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6be0009be0 tx=0x7f6be0009260 comp rx=0 tx=0).stop 2024-04-25T01:14:18.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.272+0000 7f6bf6800700 1 -- 172.21.15.31:0/3855737826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bf0109220 msgr2=0x7f6bf019f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:18.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.272+0000 7f6bf6800700 1 --2- 172.21.15.31:0/3855737826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bf0109220 0x7f6bf019f4f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6bd800e9a0 tx=0x7f6bd800ecb0 comp rx=0 tx=0).stop 2024-04-25T01:14:18.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.272+0000 7f6befe00700 1 -- 172.21.15.31:0/3855737826 reap_dead start 2024-04-25T01:14:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.272+0000 7f6bf6800700 1 -- 172.21.15.31:0/3855737826 shutdown_connections 2024-04-25T01:14:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.272+0000 7f6bf6800700 1 -- 172.21.15.31:0/3855737826 >> 172.21.15.31:0/3855737826 conn(0x7f6bf0075a00 msgr2=0x7f6bf0075f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.272+0000 7f6bf6800700 1 -- 172.21.15.31:0/3855737826 shutdown_connections 2024-04-25T01:14:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:18.272+0000 7f6bf6800700 1 -- 172.21.15.31:0/3855737826 wait complete. 2024-04-25T01:14:20.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:19 smithi031 ceph-mon[28053]: pgmap v340: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:14:20.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:19 smithi169 ceph-mon[32759]: pgmap v340: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:14:20.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:19 smithi040 ceph-mon[32518]: pgmap v340: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:14:21.123 INFO:teuthology.orchestra.run.smithi031.stdout:317827579932 2024-04-25T01:14:21.124 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T01:14:21.238 INFO:teuthology.orchestra.run.smithi031.stdout:352187318297 2024-04-25T01:14:21.238 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T01:14:21.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.709+0000 7f7674e00700 1 -- 172.21.15.31:0/1160118481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76701096a0 msgr2=0x7f7670111270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:21.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.709+0000 7f7674e00700 1 --2- 172.21.15.31:0/1160118481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76701096a0 0x7f7670111270 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f766800b0d0 tx=0x7f766800b3e0 comp rx=0 tx=0).stop 2024-04-25T01:14:21.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.710+0000 7f7674e00700 1 -- 172.21.15.31:0/1160118481 shutdown_connections 2024-04-25T01:14:21.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.710+0000 7f7674e00700 1 --2- 172.21.15.31:0/1160118481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76701096a0 0x7f7670111270 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:21.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.710+0000 7f7674e00700 1 --2- 172.21.15.31:0/1160118481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76701045c0 0x7f76701049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:21.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.710+0000 7f7674e00700 1 --2- 172.21.15.31:0/1160118481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7670104f70 0x7f7670108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:21.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.710+0000 7f7674e00700 1 -- 172.21.15.31:0/1160118481 >> 172.21.15.31:0/1160118481 conn(0x7f76700ffe10 msgr2=0x7f7670102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:21.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.710+0000 7f7674e00700 1 -- 172.21.15.31:0/1160118481 shutdown_connections 2024-04-25T01:14:21.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.710+0000 7f7674e00700 1 -- 172.21.15.31:0/1160118481 wait complete. 2024-04-25T01:14:21.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.711+0000 7f7674e00700 1 Processor -- start 2024-04-25T01:14:21.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.711+0000 7f7674e00700 1 -- start start 2024-04-25T01:14:21.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.712+0000 7f7674e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76701045c0 0x7f76701a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:21.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.712+0000 7f7674e00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7670104f70 0x7f76701a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:21.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.712+0000 7f7674e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76701096a0 0x7f767019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:21.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.712+0000 7f7674e00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f76701146b0 con 0x7f76701045c0 2024-04-25T01:14:21.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.712+0000 7f7674e00700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7670114530 con 0x7f7670104f70 2024-04-25T01:14:21.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.712+0000 7f7674e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7670114830 con 0x7f76701096a0 2024-04-25T01:14:21.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.712+0000 7f766f400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76701045c0 0x7f76701a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:21.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.712+0000 7f766f400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76701045c0 0x7f76701a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54994/0 (socket says 172.21.15.31:54994) 2024-04-25T01:14:21.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.712+0000 7f766f400700 1 -- 172.21.15.31:0/1577178142 learned_addr learned my addr 172.21.15.31:0/1577178142 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:14:21.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.712+0000 7f766ea00700 1 --2- 172.21.15.31:0/1577178142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7670104f70 0x7f76701a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:21.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.712+0000 7f766fe00700 1 --2- 172.21.15.31:0/1577178142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76701096a0 0x7f767019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:21.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.713+0000 7f766f400700 1 -- 172.21.15.31:0/1577178142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76701096a0 msgr2=0x7f767019a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.713+0000 7f766f400700 1 --2- 172.21.15.31:0/1577178142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76701096a0 0x7f767019a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.713+0000 7f766f400700 1 -- 172.21.15.31:0/1577178142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7670104f70 msgr2=0x7f76701a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.713+0000 7f766f400700 1 --2- 172.21.15.31:0/1577178142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7670104f70 0x7f76701a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.713+0000 7f766f400700 1 -- 172.21.15.31:0/1577178142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767019b030 con 0x7f76701045c0 2024-04-25T01:14:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.713+0000 7f766fe00700 1 --2- 172.21.15.31:0/1577178142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76701096a0 0x7f767019a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:14:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.713+0000 7f766f400700 1 --2- 172.21.15.31:0/1577178142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76701045c0 0x7f76701a0710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f766000e9c0 tx=0x7f766000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.713+0000 7f7667e00700 1 -- 172.21.15.31:0/1577178142 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766000cbd0 con 0x7f76701045c0 2024-04-25T01:14:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.713+0000 7f7667e00700 1 -- 172.21.15.31:0/1577178142 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f766000ee60 con 0x7f76701045c0 2024-04-25T01:14:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.713+0000 7f7674e00700 1 -- 172.21.15.31:0/1577178142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f767019b320 con 0x7f76701045c0 2024-04-25T01:14:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.713+0000 7f7667e00700 1 -- 172.21.15.31:0/1577178142 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7660005630 con 0x7f76701045c0 2024-04-25T01:14:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.713+0000 7f7674e00700 1 -- 172.21.15.31:0/1577178142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76701a75e0 con 0x7f76701045c0 2024-04-25T01:14:21.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.714+0000 7f7674e00700 1 -- 172.21.15.31:0/1577178142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f76700629c0 con 0x7f76701045c0 2024-04-25T01:14:21.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.717+0000 7f7667e00700 1 -- 172.21.15.31:0/1577178142 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7660004750 con 0x7f76701045c0 2024-04-25T01:14:21.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.718+0000 7f7667e00700 1 --2- 172.21.15.31:0/1577178142 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f76540a2d30 0x7f76540a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.718+0000 7f7667e00700 1 -- 172.21.15.31:0/1577178142 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f76600cf760 con 0x7f76701045c0 2024-04-25T01:14:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.718+0000 7f7667e00700 1 --2- 172.21.15.31:0/1577178142 >> [v2:172.21.15.169:6808/2549650781,v1:172.21.15.169:6809/2549650781] conn(0x7f76540ac510 0x7f76540ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.718+0000 7f7667e00700 1 -- 172.21.15.31:0/1577178142 --> [v2:172.21.15.169:6808/2549650781,v1:172.21.15.169:6809/2549650781] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f766001ee50 con 0x7f76540ac510 2024-04-25T01:14:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.718+0000 7f766ea00700 1 --2- 172.21.15.31:0/1577178142 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f76540a2d30 0x7f76540a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.718+0000 7f7667e00700 1 -- 172.21.15.31:0/1577178142 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f76600cfb10 con 0x7f76701045c0 2024-04-25T01:14:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.719+0000 7f766fe00700 1 --2- 172.21.15.31:0/1577178142 >> [v2:172.21.15.169:6808/2549650781,v1:172.21.15.169:6809/2549650781] conn(0x7f76540ac510 0x7f76540ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:21.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.719+0000 7f766ea00700 1 --2- 172.21.15.31:0/1577178142 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f76540a2d30 0x7f76540a51f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f765c009680 tx=0x7f765c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:21.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.719+0000 7f766fe00700 1 --2- 172.21.15.31:0/1577178142 >> [v2:172.21.15.169:6808/2549650781,v1:172.21.15.169:6809/2549650781] conn(0x7f76540ac510 0x7f76540ae930 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:21.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.720+0000 7f7667e00700 1 -- 172.21.15.31:0/1577178142 <== osd.5 v2:172.21.15.169:6808/2549650781 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f766001ee50 con 0x7f76540ac510 2024-04-25T01:14:21.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.738+0000 7f7674e00700 1 -- 172.21.15.31:0/1577178142 --> [v2:172.21.15.169:6808/2549650781,v1:172.21.15.169:6809/2549650781] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7670062dc0 con 0x7f76540ac510 2024-04-25T01:14:21.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.739+0000 7f7667e00700 1 -- 172.21.15.31:0/1577178142 <== osd.5 v2:172.21.15.169:6808/2549650781 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7670062dc0 con 0x7f76540ac510 2024-04-25T01:14:21.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.740+0000 7f7674e00700 1 -- 172.21.15.31:0/1577178142 >> [v2:172.21.15.169:6808/2549650781,v1:172.21.15.169:6809/2549650781] conn(0x7f76540ac510 msgr2=0x7f76540ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:21.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.740+0000 7f7674e00700 1 --2- 172.21.15.31:0/1577178142 >> [v2:172.21.15.169:6808/2549650781,v1:172.21.15.169:6809/2549650781] conn(0x7f76540ac510 0x7f76540ae930 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:21.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.740+0000 7f7674e00700 1 -- 172.21.15.31:0/1577178142 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f76540a2d30 msgr2=0x7f76540a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:21.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.740+0000 7f7674e00700 1 --2- 172.21.15.31:0/1577178142 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f76540a2d30 0x7f76540a51f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f765c009680 tx=0x7f765c009260 comp rx=0 tx=0).stop 2024-04-25T01:14:21.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.740+0000 7f7674e00700 1 -- 172.21.15.31:0/1577178142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76701045c0 msgr2=0x7f76701a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:21.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.740+0000 7f7674e00700 1 --2- 172.21.15.31:0/1577178142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76701045c0 0x7f76701a0710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f766000e9c0 tx=0x7f766000ecd0 comp rx=0 tx=0).stop 2024-04-25T01:14:21.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.740+0000 7f766fe00700 1 -- 172.21.15.31:0/1577178142 reap_dead start 2024-04-25T01:14:21.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.740+0000 7f7674e00700 1 -- 172.21.15.31:0/1577178142 shutdown_connections 2024-04-25T01:14:21.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.740+0000 7f7674e00700 1 -- 172.21.15.31:0/1577178142 >> 172.21.15.31:0/1577178142 conn(0x7f76700ffe10 msgr2=0x7f7670100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:21.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.741+0000 7f7674e00700 1 -- 172.21.15.31:0/1577178142 shutdown_connections 2024-04-25T01:14:21.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:21.741+0000 7f7674e00700 1 -- 172.21.15.31:0/1577178142 wait complete. 2024-04-25T01:14:21.803 INFO:teuthology.orchestra.run.smithi031.stdout:77309411403 2024-04-25T01:14:21.804 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T01:14:21.921 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:21 smithi031 ceph-mon[28053]: pgmap v341: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:14:22.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:21 smithi169 ceph-mon[32759]: pgmap v341: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:14:22.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:21 smithi040 ceph-mon[32518]: pgmap v341: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:14:22.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.617+0000 7fcb25c00700 1 -- 172.21.15.31:0/2926439096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb20104f70 msgr2=0x7fcb20108fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:22.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.617+0000 7fcb25c00700 1 --2- 172.21.15.31:0/2926439096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb20104f70 0x7fcb20108fc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800b0d0 tx=0x7fcb0800b3e0 comp rx=0 tx=0).stop 2024-04-25T01:14:22.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.617+0000 7fcb25c00700 1 -- 172.21.15.31:0/2926439096 shutdown_connections 2024-04-25T01:14:22.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.617+0000 7fcb25c00700 1 --2- 172.21.15.31:0/2926439096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb201096a0 0x7fcb20111270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:22.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.617+0000 7fcb25c00700 1 --2- 172.21.15.31:0/2926439096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb201045c0 0x7fcb201049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:22.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.617+0000 7fcb25c00700 1 --2- 172.21.15.31:0/2926439096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb20104f70 0x7fcb20108fc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:22.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.617+0000 7fcb25c00700 1 -- 172.21.15.31:0/2926439096 >> 172.21.15.31:0/2926439096 conn(0x7fcb200ffe50 msgr2=0x7fcb20102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:22.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.617+0000 7fcb25c00700 1 -- 172.21.15.31:0/2926439096 shutdown_connections 2024-04-25T01:14:22.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.618+0000 7fcb25c00700 1 -- 172.21.15.31:0/2926439096 wait complete. 2024-04-25T01:14:22.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.619+0000 7fcb25c00700 1 Processor -- start 2024-04-25T01:14:22.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.619+0000 7fcb25c00700 1 -- start start 2024-04-25T01:14:22.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.619+0000 7fcb25c00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb201045c0 0x7fcb201a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:22.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.619+0000 7fcb25c00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb20104f70 0x7fcb201a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:22.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.619+0000 7fcb25c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb201096a0 0x7fcb2019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:22.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.619+0000 7fcb25c00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcb201146b0 con 0x7fcb20104f70 2024-04-25T01:14:22.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.619+0000 7fcb25c00700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcb20114530 con 0x7fcb201045c0 2024-04-25T01:14:22.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.619+0000 7fcb25c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcb20114830 con 0x7fcb201096a0 2024-04-25T01:14:22.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.620+0000 7fcb1fe00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb201045c0 0x7fcb201a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:22.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.620+0000 7fcb17400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb20104f70 0x7fcb201a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:22.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.620+0000 7fcb1fe00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb201045c0 0x7fcb201a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:41354/0 (socket says 172.21.15.31:41354) 2024-04-25T01:14:22.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.620+0000 7fcb1fe00700 1 -- 172.21.15.31:0/3710818509 learned_addr learned my addr 172.21.15.31:0/3710818509 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:14:22.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.620+0000 7fcb25200700 1 --2- 172.21.15.31:0/3710818509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb201096a0 0x7fcb2019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:22.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.620+0000 7fcb1fe00700 1 -- 172.21.15.31:0/3710818509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb201096a0 msgr2=0x7fcb2019a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:22.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.620+0000 7fcb1fe00700 1 --2- 172.21.15.31:0/3710818509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb201096a0 0x7fcb2019a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:22.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.620+0000 7fcb1fe00700 1 -- 172.21.15.31:0/3710818509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb20104f70 msgr2=0x7fcb201a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:22.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.620+0000 7fcb1fe00700 1 --2- 172.21.15.31:0/3710818509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb20104f70 0x7fcb201a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:22.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.620+0000 7fcb1fe00700 1 -- 172.21.15.31:0/3710818509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2019b0f0 con 0x7fcb201045c0 2024-04-25T01:14:22.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.620+0000 7fcb17400700 1 --2- 172.21.15.31:0/3710818509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb20104f70 0x7fcb201a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:14:22.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.621+0000 7fcb1fe00700 1 --2- 172.21.15.31:0/3710818509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb201045c0 0x7fcb201a0780 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000e9a0 tx=0x7fcb1000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:22.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.621+0000 7fcb1d600700 1 -- 172.21.15.31:0/3710818509 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1000ca10 con 0x7fcb201045c0 2024-04-25T01:14:22.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.621+0000 7fcb1d600700 1 -- 172.21.15.31:0/3710818509 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb1000cbb0 con 0x7fcb201045c0 2024-04-25T01:14:22.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.621+0000 7fcb1d600700 1 -- 172.21.15.31:0/3710818509 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb10018680 con 0x7fcb201045c0 2024-04-25T01:14:22.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.621+0000 7fcb25c00700 1 -- 172.21.15.31:0/3710818509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb2019b3e0 con 0x7fcb201045c0 2024-04-25T01:14:22.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.621+0000 7fcb25c00700 1 -- 172.21.15.31:0/3710818509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb201a7650 con 0x7fcb201045c0 2024-04-25T01:14:22.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.622+0000 7fcb25c00700 1 -- 172.21.15.31:0/3710818509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcb200629c0 con 0x7fcb201045c0 2024-04-25T01:14:22.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.624+0000 7fcb1d600700 1 -- 172.21.15.31:0/3710818509 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fcb10010430 con 0x7fcb201045c0 2024-04-25T01:14:22.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.625+0000 7fcb1d600700 1 --2- 172.21.15.31:0/3710818509 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fcb040a2d30 0x7fcb040a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:22.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.626+0000 7fcb1d600700 1 -- 172.21.15.31:0/3710818509 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fcb100cfaf0 con 0x7fcb201045c0 2024-04-25T01:14:22.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.626+0000 7fcb17400700 1 --2- 172.21.15.31:0/3710818509 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fcb040a2d30 0x7fcb040a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:22.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.626+0000 7fcb1d600700 1 --2- 172.21.15.31:0/3710818509 >> [v2:172.21.15.40:6800/3196394509,v1:172.21.15.40:6801/3196394509] conn(0x7fcb040ac510 0x7fcb040ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:22.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.626+0000 7fcb1d600700 1 -- 172.21.15.31:0/3710818509 --> [v2:172.21.15.40:6800/3196394509,v1:172.21.15.40:6801/3196394509] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcb1001bec0 con 0x7fcb040ac510 2024-04-25T01:14:22.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.626+0000 7fcb1d600700 1 -- 172.21.15.31:0/3710818509 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fcb100cfea0 con 0x7fcb201045c0 2024-04-25T01:14:22.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.626+0000 7fcb25200700 1 --2- 172.21.15.31:0/3710818509 >> [v2:172.21.15.40:6800/3196394509,v1:172.21.15.40:6801/3196394509] conn(0x7fcb040ac510 0x7fcb040ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:22.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.627+0000 7fcb17400700 1 --2- 172.21.15.31:0/3710818509 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fcb040a2d30 0x7fcb040a51f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009fd0 tx=0x7fcb0800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:22.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.627+0000 7fcb25200700 1 --2- 172.21.15.31:0/3710818509 >> [v2:172.21.15.40:6800/3196394509,v1:172.21.15.40:6801/3196394509] conn(0x7fcb040ac510 0x7fcb040ae930 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:22.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.628+0000 7fcb1d600700 1 -- 172.21.15.31:0/3710818509 <== osd.1 v2:172.21.15.40:6800/3196394509 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcb1001bec0 con 0x7fcb040ac510 2024-04-25T01:14:22.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.646+0000 7fcb25c00700 1 -- 172.21.15.31:0/3710818509 --> [v2:172.21.15.40:6800/3196394509,v1:172.21.15.40:6801/3196394509] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcb20062e10 con 0x7fcb040ac510 2024-04-25T01:14:22.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.647+0000 7fcb1d600700 1 -- 172.21.15.31:0/3710818509 <== osd.1 v2:172.21.15.40:6800/3196394509 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcb20062e10 con 0x7fcb040ac510 2024-04-25T01:14:22.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.647+0000 7fcb25c00700 1 -- 172.21.15.31:0/3710818509 >> [v2:172.21.15.40:6800/3196394509,v1:172.21.15.40:6801/3196394509] conn(0x7fcb040ac510 msgr2=0x7fcb040ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:22.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.647+0000 7fcb25c00700 1 --2- 172.21.15.31:0/3710818509 >> [v2:172.21.15.40:6800/3196394509,v1:172.21.15.40:6801/3196394509] conn(0x7fcb040ac510 0x7fcb040ae930 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:22.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.648+0000 7fcb25c00700 1 -- 172.21.15.31:0/3710818509 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fcb040a2d30 msgr2=0x7fcb040a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:22.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.648+0000 7fcb25c00700 1 --2- 172.21.15.31:0/3710818509 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fcb040a2d30 0x7fcb040a51f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009fd0 tx=0x7fcb0800b040 comp rx=0 tx=0).stop 2024-04-25T01:14:22.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.648+0000 7fcb25c00700 1 -- 172.21.15.31:0/3710818509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb201045c0 msgr2=0x7fcb201a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:22.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.648+0000 7fcb25c00700 1 --2- 172.21.15.31:0/3710818509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb201045c0 0x7fcb201a0780 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000e9a0 tx=0x7fcb1000ecb0 comp rx=0 tx=0).stop 2024-04-25T01:14:22.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.648+0000 7fcb25200700 1 -- 172.21.15.31:0/3710818509 reap_dead start 2024-04-25T01:14:22.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.648+0000 7fcb25c00700 1 -- 172.21.15.31:0/3710818509 shutdown_connections 2024-04-25T01:14:22.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.648+0000 7fcb25c00700 1 -- 172.21.15.31:0/3710818509 >> 172.21.15.31:0/3710818509 conn(0x7fcb200ffe50 msgr2=0x7fcb20100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:22.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.648+0000 7fcb25c00700 1 -- 172.21.15.31:0/3710818509 shutdown_connections 2024-04-25T01:14:22.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.648+0000 7fcb25c00700 1 -- 172.21.15.31:0/3710818509 wait complete. 2024-04-25T01:14:23.466 INFO:teuthology.orchestra.run.smithi031.stdout:261993005096 2024-04-25T01:14:23.466 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T01:14:23.874 INFO:teuthology.orchestra.run.smithi031.stdout:296352743458 2024-04-25T01:14:23.875 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T01:14:24.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:23 smithi031 ceph-mon[28053]: pgmap v342: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T01:14:24.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:23 smithi169 ceph-mon[32759]: pgmap v342: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T01:14:24.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:23 smithi040 ceph-mon[32518]: pgmap v342: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T01:14:24.390 INFO:teuthology.orchestra.run.smithi031.stdout:34359738453 2024-04-25T01:14:24.390 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T01:14:24.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.494+0000 7f3b15c00700 1 -- 172.21.15.31:0/3308426041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b10107160 msgr2=0x7f3b10069f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:24.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.494+0000 7f3b15c00700 1 --2- 172.21.15.31:0/3308426041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b10107160 0x7f3b10069f40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000b0a0 tx=0x7f3b0000b3b0 comp rx=0 tx=0).stop 2024-04-25T01:14:24.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.494+0000 7f3b15c00700 1 -- 172.21.15.31:0/3308426041 shutdown_connections 2024-04-25T01:14:24.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.494+0000 7f3b15c00700 1 --2- 172.21.15.31:0/3308426041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b10104c70 0x7f3b10105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:24.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.494+0000 7f3b15c00700 1 --2- 172.21.15.31:0/3308426041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b10107160 0x7f3b10069f40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:24.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.494+0000 7f3b15c00700 1 --2- 172.21.15.31:0/3308426041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b1006a480 0x7f3b1006a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:24.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.494+0000 7f3b15c00700 1 -- 172.21.15.31:0/3308426041 >> 172.21.15.31:0/3308426041 conn(0x7f3b100780e0 msgr2=0x7f3b100784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:24.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.494+0000 7f3b15c00700 1 -- 172.21.15.31:0/3308426041 shutdown_connections 2024-04-25T01:14:24.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.494+0000 7f3b15c00700 1 -- 172.21.15.31:0/3308426041 wait complete. 2024-04-25T01:14:24.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.495+0000 7f3b15c00700 1 Processor -- start 2024-04-25T01:14:24.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.496+0000 7f3b15c00700 1 -- start start 2024-04-25T01:14:24.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.496+0000 7f3b15c00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b1006a480 0x7f3b1019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:24.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.496+0000 7f3b15c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b10104c70 0x7f3b1019f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:24.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.496+0000 7f3b15c00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b10107160 0x7f3b101a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:24.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.496+0000 7f3b15c00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3b10118b60 con 0x7f3b1006a480 2024-04-25T01:14:24.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.496+0000 7f3b15c00700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3b101189e0 con 0x7f3b10107160 2024-04-25T01:14:24.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.496+0000 7f3b15c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3b10118ce0 con 0x7f3b10104c70 2024-04-25T01:14:24.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.496+0000 7f3b0ea00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b1006a480 0x7f3b1019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:24.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.496+0000 7f3b0ea00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b1006a480 0x7f3b1019f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55016/0 (socket says 172.21.15.31:55016) 2024-04-25T01:14:24.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.496+0000 7f3b0ea00700 1 -- 172.21.15.31:0/2388409821 learned_addr learned my addr 172.21.15.31:0/2388409821 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:14:24.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.496+0000 7f3b0f400700 1 --2- 172.21.15.31:0/2388409821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b10107160 0x7f3b101a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:24.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.497+0000 7f3b06000700 1 --2- 172.21.15.31:0/2388409821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b10104c70 0x7f3b1019f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:24.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.497+0000 7f3b0f400700 1 -- 172.21.15.31:0/2388409821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b10104c70 msgr2=0x7f3b1019f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:24.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.497+0000 7f3b0f400700 1 --2- 172.21.15.31:0/2388409821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b10104c70 0x7f3b1019f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:24.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.497+0000 7f3b0f400700 1 -- 172.21.15.31:0/2388409821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b1006a480 msgr2=0x7f3b1019f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:24.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.497+0000 7f3b0f400700 1 --2- 172.21.15.31:0/2388409821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b1006a480 0x7f3b1019f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:24.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.497+0000 7f3b0f400700 1 -- 172.21.15.31:0/2388409821 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b101a4080 con 0x7f3b10107160 2024-04-25T01:14:24.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.497+0000 7f3b0ea00700 1 --2- 172.21.15.31:0/2388409821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b1006a480 0x7f3b1019f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:14:24.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.497+0000 7f3b0f400700 1 --2- 172.21.15.31:0/2388409821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b10107160 0x7f3b101a3940 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3b0800e970 tx=0x7f3b0800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:24.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.498+0000 7f3b07e00700 1 -- 172.21.15.31:0/2388409821 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b08004220 con 0x7f3b10107160 2024-04-25T01:14:24.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.498+0000 7f3b07e00700 1 -- 172.21.15.31:0/2388409821 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b08004d10 con 0x7f3b10107160 2024-04-25T01:14:24.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.498+0000 7f3b15c00700 1 -- 172.21.15.31:0/2388409821 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b101a4370 con 0x7f3b10107160 2024-04-25T01:14:24.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.498+0000 7f3b07e00700 1 -- 172.21.15.31:0/2388409821 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b08010700 con 0x7f3b10107160 2024-04-25T01:14:24.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.498+0000 7f3b15c00700 1 -- 172.21.15.31:0/2388409821 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b101abbf0 con 0x7f3b10107160 2024-04-25T01:14:24.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.499+0000 7f3b15c00700 1 -- 172.21.15.31:0/2388409821 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3b10100b00 con 0x7f3b10107160 2024-04-25T01:14:24.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.501+0000 7f3b07e00700 1 -- 172.21.15.31:0/2388409821 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3b08004750 con 0x7f3b10107160 2024-04-25T01:14:24.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.502+0000 7f3b07e00700 1 --2- 172.21.15.31:0/2388409821 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f3af40a2cb0 0x7f3af40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:24.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.502+0000 7f3b07e00700 1 -- 172.21.15.31:0/2388409821 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f3b080cee80 con 0x7f3b10107160 2024-04-25T01:14:24.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.503+0000 7f3b0ea00700 1 --2- 172.21.15.31:0/2388409821 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f3af40a2cb0 0x7f3af40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:24.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.503+0000 7f3b07e00700 1 --2- 172.21.15.31:0/2388409821 >> [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643] conn(0x7f3af40ac4f0 0x7f3af40ae910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:24.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.503+0000 7f3b07e00700 1 -- 172.21.15.31:0/2388409821 --> [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3af40aefc0 con 0x7f3af40ac4f0 2024-04-25T01:14:24.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.503+0000 7f3b07e00700 1 -- 172.21.15.31:0/2388409821 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f3b080d14a0 con 0x7f3b10107160 2024-04-25T01:14:24.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.503+0000 7f3b06000700 1 --2- 172.21.15.31:0/2388409821 >> [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643] conn(0x7f3af40ac4f0 0x7f3af40ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:24.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.503+0000 7f3b0ea00700 1 --2- 172.21.15.31:0/2388409821 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f3af40a2cb0 0x7f3af40a5170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000b780 tx=0x7f3b00009c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:24.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.504+0000 7f3b06000700 1 --2- 172.21.15.31:0/2388409821 >> [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643] conn(0x7f3af40ac4f0 0x7f3af40ae910 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:24.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.505+0000 7f3b07e00700 1 -- 172.21.15.31:0/2388409821 <== osd.3 v2:172.21.15.31:6810/1228688643 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3af40aefc0 con 0x7f3af40ac4f0 2024-04-25T01:14:24.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.524+0000 7f3b15c00700 1 -- 172.21.15.31:0/2388409821 --> [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3b10101750 con 0x7f3af40ac4f0 2024-04-25T01:14:24.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.524+0000 7f3b07e00700 1 -- 172.21.15.31:0/2388409821 <== osd.3 v2:172.21.15.31:6810/1228688643 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3b10101750 con 0x7f3af40ac4f0 2024-04-25T01:14:24.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.525+0000 7f3b15c00700 1 -- 172.21.15.31:0/2388409821 >> [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643] conn(0x7f3af40ac4f0 msgr2=0x7f3af40ae910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:24.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.525+0000 7f3b15c00700 1 --2- 172.21.15.31:0/2388409821 >> [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643] conn(0x7f3af40ac4f0 0x7f3af40ae910 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:24.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.525+0000 7f3b15c00700 1 -- 172.21.15.31:0/2388409821 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f3af40a2cb0 msgr2=0x7f3af40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:24.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.525+0000 7f3b15c00700 1 --2- 172.21.15.31:0/2388409821 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f3af40a2cb0 0x7f3af40a5170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000b780 tx=0x7f3b00009c40 comp rx=0 tx=0).stop 2024-04-25T01:14:24.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.525+0000 7f3b15c00700 1 -- 172.21.15.31:0/2388409821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b10107160 msgr2=0x7f3b101a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:24.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.525+0000 7f3b15c00700 1 --2- 172.21.15.31:0/2388409821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b10107160 0x7f3b101a3940 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3b0800e970 tx=0x7f3b0800ec80 comp rx=0 tx=0).stop 2024-04-25T01:14:24.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.525+0000 7f3b0f400700 1 -- 172.21.15.31:0/2388409821 reap_dead start 2024-04-25T01:14:24.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.525+0000 7f3b15c00700 1 -- 172.21.15.31:0/2388409821 shutdown_connections 2024-04-25T01:14:24.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.525+0000 7f3b15c00700 1 -- 172.21.15.31:0/2388409821 >> 172.21.15.31:0/2388409821 conn(0x7f3b100780e0 msgr2=0x7f3b10100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:24.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.525+0000 7f3b15c00700 1 -- 172.21.15.31:0/2388409821 shutdown_connections 2024-04-25T01:14:24.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:22.525+0000 7f3b15c00700 1 -- 172.21.15.31:0/2388409821 wait complete. 2024-04-25T01:14:25.505 INFO:teuthology.orchestra.run.smithi031.stdout:55834574928 2024-04-25T01:14:25.505 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T01:14:25.823 INFO:teuthology.orchestra.run.smithi031.stdout:115964117060 2024-04-25T01:14:25.824 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T01:14:26.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:25 smithi031 ceph-mon[28053]: pgmap v343: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T01:14:26.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:25 smithi169 ceph-mon[32759]: pgmap v343: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T01:14:26.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:25 smithi040 ceph-mon[32518]: pgmap v343: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T01:14:26.402 INFO:teuthology.orchestra.run.smithi031.stdout:171798691898 2024-04-25T01:14:26.402 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T01:14:26.768 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:14:26.768 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:14:26.788 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:14:26.788 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:14:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:14:26.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.806+0000 7f7aa3200700 1 -- 172.21.15.31:0/4052902600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a9c096000 msgr2=0x7f7a9c0963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:26.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.807+0000 7f7aa3200700 1 --2- 172.21.15.31:0/4052902600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a9c096000 0x7f7a9c0963e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009920 tx=0x7f7a94009c30 comp rx=0 tx=0).stop 2024-04-25T01:14:26.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.807+0000 7f7aa3200700 1 -- 172.21.15.31:0/4052902600 shutdown_connections 2024-04-25T01:14:26.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.807+0000 7f7aa3200700 1 --2- 172.21.15.31:0/4052902600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a9c096000 0x7f7a9c0963e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:26.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.807+0000 7f7aa3200700 1 --2- 172.21.15.31:0/4052902600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a9c09b090 0x7f7a9c0a2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:26.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.807+0000 7f7aa3200700 1 --2- 172.21.15.31:0/4052902600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a9c0969b0 0x7f7a9c09a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:26.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.807+0000 7f7aa3200700 1 -- 172.21.15.31:0/4052902600 >> 172.21.15.31:0/4052902600 conn(0x7f7a9c00cec0 msgr2=0x7f7a9c00d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:26.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.808+0000 7f7aa3200700 1 -- 172.21.15.31:0/4052902600 shutdown_connections 2024-04-25T01:14:26.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.808+0000 7f7aa3200700 1 -- 172.21.15.31:0/4052902600 wait complete. 2024-04-25T01:14:26.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.811+0000 7f7aa3200700 1 Processor -- start 2024-04-25T01:14:26.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.811+0000 7f7aa3200700 1 -- start start 2024-04-25T01:14:26.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.812+0000 7f7aa3200700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a9c096000 0x7f7a9c134420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:26.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.812+0000 7f7aa3200700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a9c0969b0 0x7f7a9c134960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:26.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.812+0000 7f7aa3200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a9c09b090 0x7f7a9c12e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:26.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.813+0000 7f7aa3200700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c0a60c0 con 0x7f7a9c096000 2024-04-25T01:14:26.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.813+0000 7f7aa3200700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c0a5f40 con 0x7f7a9c0969b0 2024-04-25T01:14:26.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.813+0000 7f7aa3200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c0a6240 con 0x7f7a9c09b090 2024-04-25T01:14:26.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.815+0000 7f7aa2800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a9c09b090 0x7f7a9c12e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:26.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.815+0000 7f7aa2800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a9c09b090 0x7f7a9c12e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:48740/0 (socket says 172.21.15.31:48740) 2024-04-25T01:14:26.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.815+0000 7f7aa1400700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a9c0969b0 0x7f7a9c134960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:26.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.815+0000 7f7aa1400700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a9c0969b0 0x7f7a9c134960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:36726/0 (socket says 172.21.15.31:36726) 2024-04-25T01:14:26.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.815+0000 7f7aa2800700 1 -- 172.21.15.31:0/1307496759 learned_addr learned my addr 172.21.15.31:0/1307496759 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:14:26.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.815+0000 7f7aa1e00700 1 --2- 172.21.15.31:0/1307496759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a9c096000 0x7f7a9c134420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:26.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.817+0000 7f7aa1400700 1 -- 172.21.15.31:0/1307496759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a9c09b090 msgr2=0x7f7a9c12e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:26.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.817+0000 7f7aa1400700 1 --2- 172.21.15.31:0/1307496759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a9c09b090 0x7f7a9c12e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:26.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.818+0000 7f7aa1400700 1 -- 172.21.15.31:0/1307496759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a9c096000 msgr2=0x7f7a9c134420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:26.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.818+0000 7f7aa1400700 1 --2- 172.21.15.31:0/1307496759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a9c096000 0x7f7a9c134420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:26.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.818+0000 7f7aa1400700 1 -- 172.21.15.31:0/1307496759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a9c12ed40 con 0x7f7a9c0969b0 2024-04-25T01:14:26.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.818+0000 7f7aa1e00700 1 --2- 172.21.15.31:0/1307496759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a9c096000 0x7f7a9c134420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:14:26.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.818+0000 7f7aa1400700 1 --2- 172.21.15.31:0/1307496759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a9c0969b0 0x7f7a9c134960 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7a90005fd0 tx=0x7f7a9000eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:26.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.819+0000 7f7a9aa00700 1 -- 172.21.15.31:0/1307496759 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9000c9c0 con 0x7f7a9c0969b0 2024-04-25T01:14:26.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.819+0000 7f7a9aa00700 1 -- 172.21.15.31:0/1307496759 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a9000cb60 con 0x7f7a9c0969b0 2024-04-25T01:14:26.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.819+0000 7f7aa3200700 1 -- 172.21.15.31:0/1307496759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a9c12f030 con 0x7f7a9c0969b0 2024-04-25T01:14:26.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.819+0000 7f7aa2800700 1 --2- 172.21.15.31:0/1307496759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a9c09b090 0x7f7a9c12e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:14:26.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.820+0000 7f7a9aa00700 1 -- 172.21.15.31:0/1307496759 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90018680 con 0x7f7a9c0969b0 2024-04-25T01:14:26.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.822+0000 7f7aa3200700 1 -- 172.21.15.31:0/1307496759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a9c13b2f0 con 0x7f7a9c0969b0 2024-04-25T01:14:26.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.823+0000 7f7aa3200700 1 -- 172.21.15.31:0/1307496759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7a9c0946e0 con 0x7f7a9c0969b0 2024-04-25T01:14:26.812 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:14:26.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.830+0000 7f7a9aa00700 1 -- 172.21.15.31:0/1307496759 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7a90044b60 con 0x7f7a9c0969b0 2024-04-25T01:14:26.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.834+0000 7f7a9aa00700 1 --2- 172.21.15.31:0/1307496759 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7a880a2d30 0x7f7a880a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:26.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.834+0000 7f7aa1e00700 1 --2- 172.21.15.31:0/1307496759 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7a880a2d30 0x7f7a880a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:26.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.838+0000 7f7a9aa00700 1 -- 172.21.15.31:0/1307496759 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f7a900cf400 con 0x7f7a9c0969b0 2024-04-25T01:14:26.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.838+0000 7f7a9aa00700 1 --2- 172.21.15.31:0/1307496759 >> [v2:172.21.15.40:6816/3776085849,v1:172.21.15.40:6817/3776085849] conn(0x7f7a880ac510 0x7f7a880ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:26.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.838+0000 7f7aa1e00700 1 --2- 172.21.15.31:0/1307496759 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7a880a2d30 0x7f7a880a51f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009680 tx=0x7f7a94019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:26.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.839+0000 7f7a9aa00700 1 -- 172.21.15.31:0/1307496759 --> [v2:172.21.15.40:6816/3776085849,v1:172.21.15.40:6817/3776085849] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7a90045e50 con 0x7f7a880ac510 2024-04-25T01:14:26.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.839+0000 7f7aa2800700 1 --2- 172.21.15.31:0/1307496759 >> [v2:172.21.15.40:6816/3776085849,v1:172.21.15.40:6817/3776085849] conn(0x7f7a880ac510 0x7f7a880ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:26.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.839+0000 7f7a9aa00700 1 -- 172.21.15.31:0/1307496759 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f7a900cf8e0 con 0x7f7a9c0969b0 2024-04-25T01:14:26.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.840+0000 7f7aa2800700 1 --2- 172.21.15.31:0/1307496759 >> [v2:172.21.15.40:6816/3776085849,v1:172.21.15.40:6817/3776085849] conn(0x7f7a880ac510 0x7f7a880ae930 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:26.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.841+0000 7f7a9aa00700 1 -- 172.21.15.31:0/1307496759 <== osd.7 v2:172.21.15.40:6816/3776085849 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7a90045e50 con 0x7f7a880ac510 2024-04-25T01:14:26.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.890+0000 7f7aa3200700 1 -- 172.21.15.31:0/1307496759 --> [v2:172.21.15.40:6816/3776085849,v1:172.21.15.40:6817/3776085849] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7a9c097490 con 0x7f7a880ac510 2024-04-25T01:14:26.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.891+0000 7f7a9aa00700 1 -- 172.21.15.31:0/1307496759 <== osd.7 v2:172.21.15.40:6816/3776085849 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7a9c097490 con 0x7f7a880ac510 2024-04-25T01:14:26.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.891+0000 7f7aa3200700 1 -- 172.21.15.31:0/1307496759 >> [v2:172.21.15.40:6816/3776085849,v1:172.21.15.40:6817/3776085849] conn(0x7f7a880ac510 msgr2=0x7f7a880ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:26.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.891+0000 7f7aa3200700 1 --2- 172.21.15.31:0/1307496759 >> [v2:172.21.15.40:6816/3776085849,v1:172.21.15.40:6817/3776085849] conn(0x7f7a880ac510 0x7f7a880ae930 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:26.882 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:14:26.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.892+0000 7f7aa3200700 1 -- 172.21.15.31:0/1307496759 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7a880a2d30 msgr2=0x7f7a880a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:26.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.892+0000 7f7aa3200700 1 --2- 172.21.15.31:0/1307496759 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7a880a2d30 0x7f7a880a51f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009680 tx=0x7f7a94019040 comp rx=0 tx=0).stop 2024-04-25T01:14:26.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.892+0000 7f7aa3200700 1 -- 172.21.15.31:0/1307496759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a9c0969b0 msgr2=0x7f7a9c134960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:26.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.892+0000 7f7aa3200700 1 --2- 172.21.15.31:0/1307496759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a9c0969b0 0x7f7a9c134960 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7a90005fd0 tx=0x7f7a9000eca0 comp rx=0 tx=0).stop 2024-04-25T01:14:26.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.892+0000 7f7aa2800700 1 -- 172.21.15.31:0/1307496759 reap_dead start 2024-04-25T01:14:26.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.892+0000 7f7aa3200700 1 -- 172.21.15.31:0/1307496759 shutdown_connections 2024-04-25T01:14:26.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.892+0000 7f7aa3200700 1 -- 172.21.15.31:0/1307496759 >> 172.21.15.31:0/1307496759 conn(0x7f7a9c00cec0 msgr2=0x7f7a9c093cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:26.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.893+0000 7f7aa3200700 1 -- 172.21.15.31:0/1307496759 shutdown_connections 2024-04-25T01:14:26.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:26.893+0000 7f7aa3200700 1 -- 172.21.15.31:0/1307496759 wait complete. 2024-04-25T01:14:27.987 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:27 smithi031 ceph-mon[28053]: pgmap v344: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T01:14:28.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:27 smithi169 ceph-mon[32759]: pgmap v344: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T01:14:28.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:27 smithi040 ceph-mon[32518]: pgmap v344: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T01:14:30.049 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:29 smithi031 ceph-mon[28053]: pgmap v345: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T01:14:30.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:29 smithi169 ceph-mon[32759]: pgmap v345: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T01:14:30.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:29 smithi040 ceph-mon[32518]: pgmap v345: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T01:14:30.914 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:14:30.915 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:14:31.944 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:31 smithi169 ceph-mon[32759]: pgmap v346: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:14:31.945 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:31 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:14:31.951 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:31 smithi040 ceph-mon[32518]: pgmap v346: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:14:31.951 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:31 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:14:32.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:31 smithi031 ceph-mon[28053]: pgmap v346: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:14:32.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:31 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:14:33.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:32 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T01:14:33.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:32 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]: dispatch 2024-04-25T01:14:33.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:32 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-04-25T01:14:33.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:32 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-25T01:14:33.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:32 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-25T01:14:33.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:32 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-25T01:14:33.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:32 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:14:33.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:32 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:14:33.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:32 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:14:33.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:32 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:14:33.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:32 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T01:14:33.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:32 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]: dispatch 2024-04-25T01:14:33.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:32 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-04-25T01:14:33.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:32 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-25T01:14:33.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:32 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-25T01:14:33.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:32 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-25T01:14:33.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:32 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:14:33.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:32 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:14:33.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:32 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:14:33.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:32 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:14:33.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:32 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T01:14:33.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:32 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]: dispatch 2024-04-25T01:14:33.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:32 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-04-25T01:14:33.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:32 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-25T01:14:33.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:32 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-25T01:14:33.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:32 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-25T01:14:33.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:32 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:14:33.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:32 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:14:33.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:32 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:14:33.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:32 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:14:33.360 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:14:33.538 INFO:teuthology.orchestra.run.smithi031.stdout:227633266736 2024-04-25T01:14:33.538 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T01:14:34.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:33 smithi031 ceph-mon[28053]: pgmap v347: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:14:34.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:33 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T01:14:34.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:33 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]': finished 2024-04-25T01:14:34.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:33 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-04-25T01:14:34.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:33 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-25T01:14:34.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:33 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-25T01:14:34.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:33 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-25T01:14:34.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:33 smithi031 ceph-mon[28053]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T01:14:34.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:33 smithi169 ceph-mon[32759]: pgmap v347: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:14:34.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:33 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T01:14:34.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:33 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]': finished 2024-04-25T01:14:34.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:33 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-04-25T01:14:34.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:33 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-25T01:14:34.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:33 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-25T01:14:34.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:33 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-25T01:14:34.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:33 smithi169 ceph-mon[32759]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T01:14:34.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:33 smithi040 ceph-mon[32518]: pgmap v347: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:14:34.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:33 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T01:14:34.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:33 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]': finished 2024-04-25T01:14:34.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:33 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-04-25T01:14:34.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:33 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-25T01:14:34.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:33 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-25T01:14:34.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:33 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-25T01:14:34.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:33 smithi040 ceph-mon[32518]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T01:14:34.991 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:34 smithi031 ceph-mon[28053]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T01:14:35.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:34 smithi169 ceph-mon[32759]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T01:14:35.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:34 smithi040 ceph-mon[32518]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T01:14:36.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:35 smithi031 ceph-mon[28053]: pgmap v350: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T01:14:36.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:35 smithi031 ceph-mon[28053]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-25T01:14:36.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:35 smithi169 ceph-mon[32759]: pgmap v350: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T01:14:36.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:35 smithi169 ceph-mon[32759]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-25T01:14:36.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:35 smithi040 ceph-mon[32518]: pgmap v350: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T01:14:36.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:35 smithi040 ceph-mon[32518]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-25T01:14:36.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.556+0000 7fc49d600700 1 -- 172.21.15.31:0/2099530812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4981096a0 msgr2=0x7fc498111270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:36.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.556+0000 7fc49d600700 1 --2- 172.21.15.31:0/2099530812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4981096a0 0x7fc498111270 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc49000b0a0 tx=0x7fc49000b3b0 comp rx=0 tx=0).stop 2024-04-25T01:14:36.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.557+0000 7fc49d600700 1 -- 172.21.15.31:0/2099530812 shutdown_connections 2024-04-25T01:14:36.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.557+0000 7fc49d600700 1 --2- 172.21.15.31:0/2099530812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4981096a0 0x7fc498111270 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:36.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.557+0000 7fc49d600700 1 --2- 172.21.15.31:0/2099530812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4981045c0 0x7fc4981049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:36.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.557+0000 7fc49d600700 1 --2- 172.21.15.31:0/2099530812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc498104f70 0x7fc498108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:36.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.557+0000 7fc49d600700 1 -- 172.21.15.31:0/2099530812 >> 172.21.15.31:0/2099530812 conn(0x7fc4980ffe50 msgr2=0x7fc498102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:36.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.557+0000 7fc49d600700 1 -- 172.21.15.31:0/2099530812 shutdown_connections 2024-04-25T01:14:36.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.557+0000 7fc49d600700 1 -- 172.21.15.31:0/2099530812 wait complete. 2024-04-25T01:14:36.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.558+0000 7fc49d600700 1 Processor -- start 2024-04-25T01:14:36.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.558+0000 7fc49d600700 1 -- start start 2024-04-25T01:14:36.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.558+0000 7fc49d600700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4981045c0 0x7fc4981a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:36.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.559+0000 7fc49d600700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc498104f70 0x7fc4981a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:36.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.559+0000 7fc49d600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4981096a0 0x7fc49819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:36.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.559+0000 7fc49d600700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc498114690 con 0x7fc4981045c0 2024-04-25T01:14:36.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.559+0000 7fc49d600700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc498114510 con 0x7fc498104f70 2024-04-25T01:14:36.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.559+0000 7fc49d600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc498114810 con 0x7fc4981096a0 2024-04-25T01:14:36.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.559+0000 7fc497e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4981045c0 0x7fc4981a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:36.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.559+0000 7fc497e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4981045c0 0x7fc4981a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58594/0 (socket says 172.21.15.31:58594) 2024-04-25T01:14:36.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.559+0000 7fc497e00700 1 -- 172.21.15.31:0/2669957942 learned_addr learned my addr 172.21.15.31:0/2669957942 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:14:36.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.559+0000 7fc49cc00700 1 --2- 172.21.15.31:0/2669957942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4981096a0 0x7fc49819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:36.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.559+0000 7fc48fe00700 1 --2- 172.21.15.31:0/2669957942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc498104f70 0x7fc4981a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:36.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.559+0000 7fc497e00700 1 -- 172.21.15.31:0/2669957942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4981096a0 msgr2=0x7fc49819a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:36.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.559+0000 7fc497e00700 1 --2- 172.21.15.31:0/2669957942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4981096a0 0x7fc49819a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:36.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.559+0000 7fc497e00700 1 -- 172.21.15.31:0/2669957942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc498104f70 msgr2=0x7fc4981a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:36.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.559+0000 7fc497e00700 1 --2- 172.21.15.31:0/2669957942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc498104f70 0x7fc4981a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:36.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.559+0000 7fc497e00700 1 -- 172.21.15.31:0/2669957942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49819aff0 con 0x7fc4981045c0 2024-04-25T01:14:36.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.559+0000 7fc48fe00700 1 --2- 172.21.15.31:0/2669957942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc498104f70 0x7fc4981a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:14:36.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.560+0000 7fc497e00700 1 --2- 172.21.15.31:0/2669957942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4981045c0 0x7fc4981a06d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc48800e9a0 tx=0x7fc48800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:36.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.560+0000 7fc495600700 1 -- 172.21.15.31:0/2669957942 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48800cbd0 con 0x7fc4981045c0 2024-04-25T01:14:36.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.560+0000 7fc495600700 1 -- 172.21.15.31:0/2669957942 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc48800cd70 con 0x7fc4981045c0 2024-04-25T01:14:36.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.560+0000 7fc49d600700 1 -- 172.21.15.31:0/2669957942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc49819b2e0 con 0x7fc4981045c0 2024-04-25T01:14:36.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.560+0000 7fc495600700 1 -- 172.21.15.31:0/2669957942 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc488018750 con 0x7fc4981045c0 2024-04-25T01:14:36.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.560+0000 7fc49d600700 1 -- 172.21.15.31:0/2669957942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4981a75a0 con 0x7fc4981045c0 2024-04-25T01:14:36.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.562+0000 7fc49d600700 1 -- 172.21.15.31:0/2669957942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc474005330 con 0x7fc4981045c0 2024-04-25T01:14:36.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.563+0000 7fc495600700 1 -- 172.21.15.31:0/2669957942 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc4880188f0 con 0x7fc4981045c0 2024-04-25T01:14:36.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.565+0000 7fc495600700 1 --2- 172.21.15.31:0/2669957942 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc47c0a2cb0 0x7fc47c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:36.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.565+0000 7fc495600700 1 -- 172.21.15.31:0/2669957942 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fc4880cf450 con 0x7fc4981045c0 2024-04-25T01:14:36.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.565+0000 7fc48fe00700 1 --2- 172.21.15.31:0/2669957942 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc47c0a2cb0 0x7fc47c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:36.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.566+0000 7fc48fe00700 1 --2- 172.21.15.31:0/2669957942 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc47c0a2cb0 0x7fc47c0a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc498103500 tx=0x7fc480005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:36.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.570+0000 7fc495600700 1 -- 172.21.15.31:0/2669957942 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc48809c240 con 0x7fc4981045c0 2024-04-25T01:14:36.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.804+0000 7fc49d600700 1 -- 172.21.15.31:0/2669957942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fc4740058f0 con 0x7fc4981045c0 2024-04-25T01:14:36.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.805+0000 7fc495600700 1 -- 172.21.15.31:0/2669957942 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc48809bb40 con 0x7fc4981045c0 2024-04-25T01:14:36.791 INFO:teuthology.orchestra.run.smithi031.stdout:261993005100 2024-04-25T01:14:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.808+0000 7fc49d600700 1 -- 172.21.15.31:0/2669957942 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc47c0a2cb0 msgr2=0x7fc47c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.808+0000 7fc49d600700 1 --2- 172.21.15.31:0/2669957942 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc47c0a2cb0 0x7fc47c0a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc498103500 tx=0x7fc480005d80 comp rx=0 tx=0).stop 2024-04-25T01:14:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.808+0000 7fc49d600700 1 -- 172.21.15.31:0/2669957942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4981045c0 msgr2=0x7fc4981a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.808+0000 7fc49d600700 1 --2- 172.21.15.31:0/2669957942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4981045c0 0x7fc4981a06d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc48800e9a0 tx=0x7fc48800ecb0 comp rx=0 tx=0).stop 2024-04-25T01:14:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.808+0000 7fc49d600700 1 -- 172.21.15.31:0/2669957942 shutdown_connections 2024-04-25T01:14:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.808+0000 7fc49d600700 1 --2- 172.21.15.31:0/2669957942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4981045c0 0x7fc4981a06d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.808+0000 7fc49d600700 1 --2- 172.21.15.31:0/2669957942 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc47c0a2cb0 0x7fc47c0a5170 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.808+0000 7fc49d600700 1 --2- 172.21.15.31:0/2669957942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4981096a0 0x7fc49819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:36.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.808+0000 7fc49d600700 1 --2- 172.21.15.31:0/2669957942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc498104f70 0x7fc4981a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:36.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.808+0000 7fc49d600700 1 -- 172.21.15.31:0/2669957942 >> 172.21.15.31:0/2669957942 conn(0x7fc4980ffe50 msgr2=0x7fc498100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:36.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.808+0000 7fc49d600700 1 -- 172.21.15.31:0/2669957942 shutdown_connections 2024-04-25T01:14:36.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:36.809+0000 7fc49d600700 1 -- 172.21.15.31:0/2669957942 wait complete. 2024-04-25T01:14:37.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:36 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2669957942' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T01:14:37.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:36 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2669957942' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T01:14:37.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:36 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2669957942' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T01:14:38.111 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:37 smithi031 ceph-mon[28053]: pgmap v351: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T01:14:38.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:37 smithi169 ceph-mon[32759]: pgmap v351: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T01:14:38.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:37 smithi040 ceph-mon[32518]: pgmap v351: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T01:14:40.066 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:39 smithi031 ceph-mon[28053]: pgmap v352: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T01:14:40.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:39 smithi169 ceph-mon[32759]: pgmap v352: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T01:14:40.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:39 smithi040 ceph-mon[32518]: pgmap v352: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T01:14:40.921 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:40 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3999951451' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T01:14:40.921 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:40 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/4181206357' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T01:14:41.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:40 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3999951451' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T01:14:41.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:40 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/4181206357' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T01:14:41.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:40 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3999951451' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T01:14:41.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:40 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/4181206357' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T01:14:41.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.429+0000 7f72db600700 1 -- 172.21.15.31:0/2229367437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72d4113400 msgr2=0x7f72d41157f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.429+0000 7f72db600700 1 --2- 172.21.15.31:0/2229367437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72d4113400 0x7f72d41157f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00b0a0 tx=0x7f72cc00b3b0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.429+0000 7f72db600700 1 -- 172.21.15.31:0/2229367437 shutdown_connections 2024-04-25T01:14:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.429+0000 7f72db600700 1 --2- 172.21.15.31:0/2229367437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72d4113400 0x7f72d41157f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.429+0000 7f72db600700 1 --2- 172.21.15.31:0/2229367437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72d4074460 0x7f72d4074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.429+0000 7f72db600700 1 --2- 172.21.15.31:0/2229367437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72d4074d80 0x7f72d4112ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.429+0000 7f72db600700 1 -- 172.21.15.31:0/2229367437 >> 172.21.15.31:0/2229367437 conn(0x7f72d4079390 msgr2=0x7f72d40797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.429+0000 7f72db600700 1 -- 172.21.15.31:0/2229367437 shutdown_connections 2024-04-25T01:14:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.429+0000 7f72db600700 1 -- 172.21.15.31:0/2229367437 wait complete. 2024-04-25T01:14:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.430+0000 7f72db600700 1 Processor -- start 2024-04-25T01:14:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.431+0000 7f72db600700 1 -- start start 2024-04-25T01:14:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.431+0000 7f72db600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72d4074460 0x7f72d419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:41.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.431+0000 7f72db600700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72d4074d80 0x7f72d419f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:41.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.431+0000 7f72db600700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72d4113400 0x7f72d41a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:41.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.431+0000 7f72db600700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f72d4118b60 con 0x7f72d4074d80 2024-04-25T01:14:41.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.431+0000 7f72db600700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f72d41189e0 con 0x7f72d4113400 2024-04-25T01:14:41.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.431+0000 7f72db600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f72d4118ce0 con 0x7f72d4074460 2024-04-25T01:14:41.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.431+0000 7f72cbe00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72d4074d80 0x7f72d419f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:41.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.431+0000 7f72cbe00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72d4074d80 0x7f72d419f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42648/0 (socket says 172.21.15.31:42648) 2024-04-25T01:14:41.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.431+0000 7f72cbe00700 1 -- 172.21.15.31:0/4181206357 learned_addr learned my addr 172.21.15.31:0/4181206357 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:14:41.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.431+0000 7f72d8c00700 1 --2- 172.21.15.31:0/4181206357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72d4074460 0x7f72d419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:41.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.431+0000 7f72d9600700 1 --2- 172.21.15.31:0/4181206357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72d4113400 0x7f72d41a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:41.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.432+0000 7f72cbe00700 1 -- 172.21.15.31:0/4181206357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72d4074460 msgr2=0x7f72d419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:41.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.432+0000 7f72cbe00700 1 --2- 172.21.15.31:0/4181206357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72d4074460 0x7f72d419eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.432+0000 7f72cbe00700 1 -- 172.21.15.31:0/4181206357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72d4113400 msgr2=0x7f72d41a3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:41.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.432+0000 7f72cbe00700 1 --2- 172.21.15.31:0/4181206357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72d4113400 0x7f72d41a3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.432+0000 7f72cbe00700 1 -- 172.21.15.31:0/4181206357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d41a3fb0 con 0x7f72d4074d80 2024-04-25T01:14:41.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.432+0000 7f72cbe00700 1 --2- 172.21.15.31:0/4181206357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72d4074d80 0x7f72d419f530 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f72c400e9a0 tx=0x7f72c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:41.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.432+0000 7f72d2000700 1 -- 172.21.15.31:0/4181206357 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c400ca10 con 0x7f72d4074d80 2024-04-25T01:14:41.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.432+0000 7f72d2000700 1 -- 172.21.15.31:0/4181206357 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72c400cbb0 con 0x7f72d4074d80 2024-04-25T01:14:41.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.432+0000 7f72d2000700 1 -- 172.21.15.31:0/4181206357 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c4018680 con 0x7f72d4074d80 2024-04-25T01:14:41.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.432+0000 7f72db600700 1 -- 172.21.15.31:0/4181206357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72d41a42a0 con 0x7f72d4074d80 2024-04-25T01:14:41.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.433+0000 7f72db600700 1 -- 172.21.15.31:0/4181206357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d41abb20 con 0x7f72d4074d80 2024-04-25T01:14:41.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.434+0000 7f72db600700 1 -- 172.21.15.31:0/4181206357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72d4068f20 con 0x7f72d4074d80 2024-04-25T01:14:41.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.436+0000 7f72d2000700 1 -- 172.21.15.31:0/4181206357 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f72c4010430 con 0x7f72d4074d80 2024-04-25T01:14:41.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.437+0000 7f72d2000700 1 --2- 172.21.15.31:0/4181206357 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f72b40a2cb0 0x7f72b40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:41.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.437+0000 7f72d2000700 1 -- 172.21.15.31:0/4181206357 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f72c40cf390 con 0x7f72d4074d80 2024-04-25T01:14:41.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.438+0000 7f72d8c00700 1 --2- 172.21.15.31:0/4181206357 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f72b40a2cb0 0x7f72b40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:41.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.438+0000 7f72d8c00700 1 --2- 172.21.15.31:0/4181206357 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f72b40a2cb0 0x7f72b40a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f72bc009680 tx=0x7f72bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:41.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.441+0000 7f72d2000700 1 -- 172.21.15.31:0/4181206357 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f72c409c180 con 0x7f72d4074d80 2024-04-25T01:14:41.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.675+0000 7f72db600700 1 -- 172.21.15.31:0/4181206357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f72d41abd50 con 0x7f72d4074d80 2024-04-25T01:14:41.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.676+0000 7f72d2000700 1 -- 172.21.15.31:0/4181206357 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f72c409ba80 con 0x7f72d4074d80 2024-04-25T01:14:41.419 INFO:teuthology.orchestra.run.smithi031.stdout:115964117063 2024-04-25T01:14:41.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.679+0000 7f72db600700 1 -- 172.21.15.31:0/4181206357 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f72b40a2cb0 msgr2=0x7f72b40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:41.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.679+0000 7f72db600700 1 --2- 172.21.15.31:0/4181206357 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f72b40a2cb0 0x7f72b40a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f72bc009680 tx=0x7f72bc009260 comp rx=0 tx=0).stop 2024-04-25T01:14:41.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.679+0000 7f72db600700 1 -- 172.21.15.31:0/4181206357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72d4074d80 msgr2=0x7f72d419f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:41.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.679+0000 7f72db600700 1 --2- 172.21.15.31:0/4181206357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72d4074d80 0x7f72d419f530 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f72c400e9a0 tx=0x7f72c400ecb0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.679+0000 7f72db600700 1 -- 172.21.15.31:0/4181206357 shutdown_connections 2024-04-25T01:14:41.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.679+0000 7f72db600700 1 --2- 172.21.15.31:0/4181206357 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f72b40a2cb0 0x7f72b40a5170 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.679+0000 7f72db600700 1 --2- 172.21.15.31:0/4181206357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72d4113400 0x7f72d41a3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.679+0000 7f72db600700 1 --2- 172.21.15.31:0/4181206357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72d4074460 0x7f72d419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.680+0000 7f72db600700 1 --2- 172.21.15.31:0/4181206357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72d4074d80 0x7f72d419f530 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.680+0000 7f72db600700 1 -- 172.21.15.31:0/4181206357 >> 172.21.15.31:0/4181206357 conn(0x7f72d4079390 msgr2=0x7f72d4112160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:41.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.680+0000 7f72db600700 1 -- 172.21.15.31:0/4181206357 shutdown_connections 2024-04-25T01:14:41.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.680+0000 7f72db600700 1 -- 172.21.15.31:0/4181206357 wait complete. 2024-04-25T01:14:41.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.829+0000 7ff6aae00700 1 -- 172.21.15.31:0/3543485670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6a410c7c0 msgr2=0x7ff6a410ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:41.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.829+0000 7ff6aae00700 1 --2- 172.21.15.31:0/3543485670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6a410c7c0 0x7ff6a410ec80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00b0a0 tx=0x7ff69c00b3b0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.829+0000 7ff6aae00700 1 -- 172.21.15.31:0/3543485670 shutdown_connections 2024-04-25T01:14:41.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.829+0000 7ff6aae00700 1 --2- 172.21.15.31:0/3543485670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a4103390 0x7ff6a4103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.829+0000 7ff6aae00700 1 --2- 172.21.15.31:0/3543485670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6a410c7c0 0x7ff6a410ec80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.829+0000 7ff6aae00700 1 --2- 172.21.15.31:0/3543485670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a4103cb0 0x7ff6a410c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.829+0000 7ff6aae00700 1 -- 172.21.15.31:0/3543485670 >> 172.21.15.31:0/3543485670 conn(0x7ff6a40fcbe0 msgr2=0x7ff6a40ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:41.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.829+0000 7ff6aae00700 1 -- 172.21.15.31:0/3543485670 shutdown_connections 2024-04-25T01:14:41.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.829+0000 7ff6aae00700 1 -- 172.21.15.31:0/3543485670 wait complete. 2024-04-25T01:14:41.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.830+0000 7ff6aae00700 1 Processor -- start 2024-04-25T01:14:41.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.830+0000 7ff6aae00700 1 -- start start 2024-04-25T01:14:41.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.831+0000 7ff6aae00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a4103390 0x7ff6a41a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:41.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.831+0000 7ff6aae00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6a4103cb0 0x7ff6a41a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:41.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.831+0000 7ff6aae00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a410c7c0 0x7ff6a419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:41.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.831+0000 7ff6aae00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a4112530 con 0x7ff6a4103390 2024-04-25T01:14:41.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.831+0000 7ff6aae00700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a41123b0 con 0x7ff6a410c7c0 2024-04-25T01:14:41.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.831+0000 7ff6aae00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a41126b0 con 0x7ff6a4103cb0 2024-04-25T01:14:41.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.831+0000 7ff6a9a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a4103390 0x7ff6a41a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:41.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.831+0000 7ff6a9a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a4103390 0x7ff6a41a0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42620/0 (socket says 172.21.15.31:42620) 2024-04-25T01:14:41.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.831+0000 7ff6a9a00700 1 -- 172.21.15.31:0/3999951451 learned_addr learned my addr 172.21.15.31:0/3999951451 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:14:41.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.831+0000 7ff6a9000700 1 --2- 172.21.15.31:0/3999951451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6a4103cb0 0x7ff6a41a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:41.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.831+0000 7ff6aa400700 1 --2- 172.21.15.31:0/3999951451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a410c7c0 0x7ff6a419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:41.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.831+0000 7ff6a9a00700 1 -- 172.21.15.31:0/3999951451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6a4103cb0 msgr2=0x7ff6a41a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:41.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.831+0000 7ff6a9a00700 1 --2- 172.21.15.31:0/3999951451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6a4103cb0 0x7ff6a41a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.831+0000 7ff6a9a00700 1 -- 172.21.15.31:0/3999951451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a410c7c0 msgr2=0x7ff6a419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:41.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.831+0000 7ff6a9a00700 1 --2- 172.21.15.31:0/3999951451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a410c7c0 0x7ff6a419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.831+0000 7ff6a9a00700 1 -- 172.21.15.31:0/3999951451 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a419b120 con 0x7ff6a4103390 2024-04-25T01:14:41.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.832+0000 7ff6a9a00700 1 --2- 172.21.15.31:0/3999951451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a4103390 0x7ff6a41a0800 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff69800e9a0 tx=0x7ff69800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:41.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.832+0000 7ff6a2000700 1 -- 172.21.15.31:0/3999951451 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69800ca10 con 0x7ff6a4103390 2024-04-25T01:14:41.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.832+0000 7ff6aae00700 1 -- 172.21.15.31:0/3999951451 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a419b410 con 0x7ff6a4103390 2024-04-25T01:14:41.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.832+0000 7ff6a2000700 1 -- 172.21.15.31:0/3999951451 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff69800cbb0 con 0x7ff6a4103390 2024-04-25T01:14:41.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.832+0000 7ff6a2000700 1 -- 172.21.15.31:0/3999951451 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff698018680 con 0x7ff6a4103390 2024-04-25T01:14:41.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.832+0000 7ff6aae00700 1 -- 172.21.15.31:0/3999951451 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a41a76d0 con 0x7ff6a4103390 2024-04-25T01:14:41.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.833+0000 7ff6aae00700 1 -- 172.21.15.31:0/3999951451 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6a4068f20 con 0x7ff6a4103390 2024-04-25T01:14:41.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.835+0000 7ff6a2000700 1 -- 172.21.15.31:0/3999951451 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff698010430 con 0x7ff6a4103390 2024-04-25T01:14:41.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.837+0000 7ff6a2000700 1 --2- 172.21.15.31:0/3999951451 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff68c0a2cb0 0x7ff68c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:41.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.837+0000 7ff6a2000700 1 -- 172.21.15.31:0/3999951451 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7ff6980cf400 con 0x7ff6a4103390 2024-04-25T01:14:41.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.837+0000 7ff6a9000700 1 --2- 172.21.15.31:0/3999951451 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff68c0a2cb0 0x7ff68c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:41.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.838+0000 7ff6a9000700 1 --2- 172.21.15.31:0/3999951451 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff68c0a2cb0 0x7ff68c0a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff6a410e030 tx=0x7ff690009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:41.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:39.841+0000 7ff6a2000700 1 -- 172.21.15.31:0/3999951451 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff69809c1f0 con 0x7ff6a4103390 2024-04-25T01:14:41.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.079+0000 7ff6aae00700 1 -- 172.21.15.31:0/3999951451 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7ff6a41a79c0 con 0x7ff6a4103390 2024-04-25T01:14:41.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.080+0000 7ff6a2000700 1 -- 172.21.15.31:0/3999951451 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff69809baf0 con 0x7ff6a4103390 2024-04-25T01:14:41.580 INFO:teuthology.orchestra.run.smithi031.stdout:352187318301 2024-04-25T01:14:41.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.082+0000 7ff6aae00700 1 -- 172.21.15.31:0/3999951451 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff68c0a2cb0 msgr2=0x7ff68c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:41.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.082+0000 7ff6aae00700 1 --2- 172.21.15.31:0/3999951451 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff68c0a2cb0 0x7ff68c0a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff6a410e030 tx=0x7ff690009260 comp rx=0 tx=0).stop 2024-04-25T01:14:41.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.083+0000 7ff6aae00700 1 -- 172.21.15.31:0/3999951451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a4103390 msgr2=0x7ff6a41a0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:41.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.083+0000 7ff6aae00700 1 --2- 172.21.15.31:0/3999951451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a4103390 0x7ff6a41a0800 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff69800e9a0 tx=0x7ff69800ecb0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.083+0000 7ff6aae00700 1 -- 172.21.15.31:0/3999951451 shutdown_connections 2024-04-25T01:14:41.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.083+0000 7ff6aae00700 1 --2- 172.21.15.31:0/3999951451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a4103390 0x7ff6a41a0800 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.083+0000 7ff6aae00700 1 --2- 172.21.15.31:0/3999951451 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff68c0a2cb0 0x7ff68c0a5170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.083+0000 7ff6aae00700 1 --2- 172.21.15.31:0/3999951451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6a410c7c0 0x7ff6a419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.083+0000 7ff6aae00700 1 --2- 172.21.15.31:0/3999951451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6a4103cb0 0x7ff6a41a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.083+0000 7ff6aae00700 1 -- 172.21.15.31:0/3999951451 >> 172.21.15.31:0/3999951451 conn(0x7ff6a40fcbe0 msgr2=0x7ff6a410ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:41.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.083+0000 7ff6aae00700 1 -- 172.21.15.31:0/3999951451 shutdown_connections 2024-04-25T01:14:41.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:40.083+0000 7ff6aae00700 1 -- 172.21.15.31:0/3999951451 wait complete. 2024-04-25T01:14:41.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.668+0000 7f4c77600700 1 -- 172.21.15.31:0/2482950562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c701024e0 msgr2=0x7f4c701028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:41.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.668+0000 7f4c77600700 1 --2- 172.21.15.31:0/2482950562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c701024e0 0x7f4c701028c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4c54009990 tx=0x7f4c54009ca0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.668+0000 7f4c77600700 1 -- 172.21.15.31:0/2482950562 shutdown_connections 2024-04-25T01:14:41.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.668+0000 7f4c77600700 1 --2- 172.21.15.31:0/2482950562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c70107570 0x7f4c7010f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.668+0000 7f4c77600700 1 --2- 172.21.15.31:0/2482950562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c701024e0 0x7f4c701028c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.668+0000 7f4c77600700 1 --2- 172.21.15.31:0/2482950562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c70102e90 0x7f4c70106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.668+0000 7f4c77600700 1 -- 172.21.15.31:0/2482950562 >> 172.21.15.31:0/2482950562 conn(0x7f4c70079390 msgr2=0x7f4c700797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:41.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.668+0000 7f4c77600700 1 -- 172.21.15.31:0/2482950562 shutdown_connections 2024-04-25T01:14:41.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.668+0000 7f4c77600700 1 -- 172.21.15.31:0/2482950562 wait complete. 2024-04-25T01:14:41.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.670+0000 7f4c77600700 1 Processor -- start 2024-04-25T01:14:41.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.670+0000 7f4c77600700 1 -- start start 2024-04-25T01:14:41.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.671+0000 7f4c77600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c701024e0 0x7f4c701a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:41.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.671+0000 7f4c77600700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c70102e90 0x7f4c701a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:41.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.671+0000 7f4c77600700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c70107570 0x7f4c7019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:41.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.671+0000 7f4c77600700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4c70112580 con 0x7f4c70102e90 2024-04-25T01:14:41.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.671+0000 7f4c77600700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4c70112400 con 0x7f4c70107570 2024-04-25T01:14:41.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.671+0000 7f4c77600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4c70112700 con 0x7f4c701024e0 2024-04-25T01:14:41.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.671+0000 7f4c74c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c701024e0 0x7f4c701a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:41.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.671+0000 7f4c74c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c701024e0 0x7f4c701a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:35720/0 (socket says 172.21.15.31:35720) 2024-04-25T01:14:41.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.671+0000 7f4c74c00700 1 -- 172.21.15.31:0/129426684 learned_addr learned my addr 172.21.15.31:0/129426684 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:14:41.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.671+0000 7f4c67e00700 1 --2- 172.21.15.31:0/129426684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c70102e90 0x7f4c701a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.671+0000 7f4c75600700 1 --2- 172.21.15.31:0/129426684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c70107570 0x7f4c7019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.671+0000 7f4c74c00700 1 -- 172.21.15.31:0/129426684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c70107570 msgr2=0x7f4c7019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.671+0000 7f4c74c00700 1 --2- 172.21.15.31:0/129426684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c70107570 0x7f4c7019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.671+0000 7f4c74c00700 1 -- 172.21.15.31:0/129426684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c70102e90 msgr2=0x7f4c701a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.671+0000 7f4c74c00700 1 --2- 172.21.15.31:0/129426684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c70102e90 0x7f4c701a0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.671+0000 7f4c74c00700 1 -- 172.21.15.31:0/129426684 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c7019b220 con 0x7f4c701024e0 2024-04-25T01:14:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.672+0000 7f4c75600700 1 --2- 172.21.15.31:0/129426684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c70107570 0x7f4c7019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:14:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.672+0000 7f4c74c00700 1 --2- 172.21.15.31:0/129426684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c701024e0 0x7f4c701a0900 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4c54005910 tx=0x7f4c5400bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.672+0000 7f4c6e000700 1 -- 172.21.15.31:0/129426684 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c54027070 con 0x7f4c701024e0 2024-04-25T01:14:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.672+0000 7f4c6e000700 1 -- 172.21.15.31:0/129426684 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c54003d70 con 0x7f4c701024e0 2024-04-25T01:14:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.672+0000 7f4c6e000700 1 -- 172.21.15.31:0/129426684 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5400f460 con 0x7f4c701024e0 2024-04-25T01:14:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.672+0000 7f4c77600700 1 -- 172.21.15.31:0/129426684 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c7019b4b0 con 0x7f4c701024e0 2024-04-25T01:14:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.673+0000 7f4c77600700 1 -- 172.21.15.31:0/129426684 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c701a77d0 con 0x7f4c701024e0 2024-04-25T01:14:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.674+0000 7f4c77600700 1 -- 172.21.15.31:0/129426684 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c70068f20 con 0x7f4c701024e0 2024-04-25T01:14:41.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.676+0000 7f4c6e000700 1 -- 172.21.15.31:0/129426684 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4c5402a070 con 0x7f4c701024e0 2024-04-25T01:14:41.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.677+0000 7f4c6e000700 1 --2- 172.21.15.31:0/129426684 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f4c4c0a2d30 0x7f4c4c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:41.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.677+0000 7f4c6e000700 1 -- 172.21.15.31:0/129426684 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f4c540d0200 con 0x7f4c701024e0 2024-04-25T01:14:41.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.677+0000 7f4c67e00700 1 --2- 172.21.15.31:0/129426684 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f4c4c0a2d30 0x7f4c4c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:41.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.678+0000 7f4c67e00700 1 --2- 172.21.15.31:0/129426684 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f4c4c0a2d30 0x7f4c4c0a51f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4c700764c0 tx=0x7f4c5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:41.913 INFO:teuthology.orchestra.run.smithi031.stdout:77309411407 2024-04-25T01:14:41.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.680+0000 7f4c6e000700 1 -- 172.21.15.31:0/129426684 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4c54013070 con 0x7f4c701024e0 2024-04-25T01:14:41.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.919+0000 7f4c77600700 1 -- 172.21.15.31:0/129426684 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f4c701a79a0 con 0x7f4c701024e0 2024-04-25T01:14:41.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.920+0000 7f4c6e000700 1 -- 172.21.15.31:0/129426684 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4c5409d1a0 con 0x7f4c701024e0 2024-04-25T01:14:41.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.923+0000 7f4c77600700 1 -- 172.21.15.31:0/129426684 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f4c4c0a2d30 msgr2=0x7f4c4c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:41.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.923+0000 7f4c77600700 1 --2- 172.21.15.31:0/129426684 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f4c4c0a2d30 0x7f4c4c0a51f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4c700764c0 tx=0x7f4c5c009260 comp rx=0 tx=0).stop 2024-04-25T01:14:41.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.923+0000 7f4c77600700 1 -- 172.21.15.31:0/129426684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c701024e0 msgr2=0x7f4c701a0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:41.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.923+0000 7f4c77600700 1 --2- 172.21.15.31:0/129426684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c701024e0 0x7f4c701a0900 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4c54005910 tx=0x7f4c5400bab0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.923+0000 7f4c77600700 1 -- 172.21.15.31:0/129426684 shutdown_connections 2024-04-25T01:14:41.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.923+0000 7f4c77600700 1 --2- 172.21.15.31:0/129426684 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f4c4c0a2d30 0x7f4c4c0a51f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.923+0000 7f4c77600700 1 --2- 172.21.15.31:0/129426684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c70107570 0x7f4c7019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.923+0000 7f4c77600700 1 --2- 172.21.15.31:0/129426684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c701024e0 0x7f4c701a0900 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.923+0000 7f4c77600700 1 --2- 172.21.15.31:0/129426684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c70102e90 0x7f4c701a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:41.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.923+0000 7f4c77600700 1 -- 172.21.15.31:0/129426684 >> 172.21.15.31:0/129426684 conn(0x7f4c70079390 msgr2=0x7f4c701086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:41.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.924+0000 7f4c77600700 1 -- 172.21.15.31:0/129426684 shutdown_connections 2024-04-25T01:14:41.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:41.924+0000 7f4c77600700 1 -- 172.21.15.31:0/129426684 wait complete. 2024-04-25T01:14:42.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:41 smithi169 ceph-mon[32759]: pgmap v353: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T01:14:42.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:41 smithi169 ceph-mon[32759]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-25T01:14:42.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:41 smithi169 ceph-mon[32759]: Cluster is now healthy 2024-04-25T01:14:42.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:41 smithi040 ceph-mon[32518]: pgmap v353: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T01:14:42.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:41 smithi040 ceph-mon[32518]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-25T01:14:42.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:41 smithi040 ceph-mon[32518]: Cluster is now healthy 2024-04-25T01:14:42.248 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:41 smithi031 ceph-mon[28053]: pgmap v353: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T01:14:42.248 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:41 smithi031 ceph-mon[28053]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-25T01:14:42.248 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:41 smithi031 ceph-mon[28053]: Cluster is now healthy 2024-04-25T01:14:42.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.771+0000 7f0bca600700 1 -- 172.21.15.31:0/2163894738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bc4102e90 msgr2=0x7f0bc4106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:42.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.771+0000 7f0bca600700 1 --2- 172.21.15.31:0/2163894738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bc4102e90 0x7f0bc4106e90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009990 tx=0x7f0bb0009ca0 comp rx=0 tx=0).stop 2024-04-25T01:14:42.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.771+0000 7f0bca600700 1 -- 172.21.15.31:0/2163894738 shutdown_connections 2024-04-25T01:14:42.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.771+0000 7f0bca600700 1 --2- 172.21.15.31:0/2163894738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bc4107570 0x7f0bc410f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:42.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.771+0000 7f0bca600700 1 --2- 172.21.15.31:0/2163894738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bc41024e0 0x7f0bc41028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:42.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.771+0000 7f0bca600700 1 --2- 172.21.15.31:0/2163894738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bc4102e90 0x7f0bc4106e90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:42.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.771+0000 7f0bca600700 1 -- 172.21.15.31:0/2163894738 >> 172.21.15.31:0/2163894738 conn(0x7f0bc4079390 msgr2=0x7f0bc40797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:42.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.771+0000 7f0bca600700 1 -- 172.21.15.31:0/2163894738 shutdown_connections 2024-04-25T01:14:42.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.771+0000 7f0bca600700 1 -- 172.21.15.31:0/2163894738 wait complete. 2024-04-25T01:14:42.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.772+0000 7f0bca600700 1 Processor -- start 2024-04-25T01:14:42.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.773+0000 7f0bca600700 1 -- start start 2024-04-25T01:14:42.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.773+0000 7f0bca600700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bc41024e0 0x7f0bc4074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:42.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.773+0000 7f0bca600700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bc4102e90 0x7f0bc4074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:42.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.773+0000 7f0bca600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bc4107570 0x7f0bc4074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:42.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.773+0000 7f0bca600700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc4112580 con 0x7f0bc4102e90 2024-04-25T01:14:42.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.773+0000 7f0bca600700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc4112400 con 0x7f0bc41024e0 2024-04-25T01:14:42.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.773+0000 7f0bca600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc4112700 con 0x7f0bc4107570 2024-04-25T01:14:42.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.773+0000 7f0bc2a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bc4102e90 0x7f0bc4074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:42.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.773+0000 7f0bc2a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bc4102e90 0x7f0bc4074a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42694/0 (socket says 172.21.15.31:42694) 2024-04-25T01:14:42.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.773+0000 7f0bc2a00700 1 -- 172.21.15.31:0/1179946149 learned_addr learned my addr 172.21.15.31:0/1179946149 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:14:42.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.773+0000 7f0bc3400700 1 --2- 172.21.15.31:0/1179946149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bc41024e0 0x7f0bc4074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:42.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.773+0000 7f0bc3e00700 1 --2- 172.21.15.31:0/1179946149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bc4107570 0x7f0bc4074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:42.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.774+0000 7f0bc3400700 1 -- 172.21.15.31:0/1179946149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bc4107570 msgr2=0x7f0bc4074fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:42.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.774+0000 7f0bc3400700 1 --2- 172.21.15.31:0/1179946149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bc4107570 0x7f0bc4074fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:42.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.774+0000 7f0bc3400700 1 -- 172.21.15.31:0/1179946149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bc4102e90 msgr2=0x7f0bc4074a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:42.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.774+0000 7f0bc3400700 1 --2- 172.21.15.31:0/1179946149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bc4102e90 0x7f0bc4074a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:42.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.774+0000 7f0bc3400700 1 -- 172.21.15.31:0/1179946149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc41a6f80 con 0x7f0bc41024e0 2024-04-25T01:14:42.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.774+0000 7f0bc3400700 1 --2- 172.21.15.31:0/1179946149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bc41024e0 0x7f0bc4074530 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400e9f0 tx=0x7f0bb400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:42.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.774+0000 7f0bbbe00700 1 -- 172.21.15.31:0/1179946149 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4004220 con 0x7f0bc41024e0 2024-04-25T01:14:42.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.774+0000 7f0bca600700 1 -- 172.21.15.31:0/1179946149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc41a7270 con 0x7f0bc41024e0 2024-04-25T01:14:42.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.774+0000 7f0bbbe00700 1 -- 172.21.15.31:0/1179946149 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb4004d10 con 0x7f0bc41024e0 2024-04-25T01:14:42.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.774+0000 7f0bbbe00700 1 -- 172.21.15.31:0/1179946149 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4005020 con 0x7f0bc41024e0 2024-04-25T01:14:42.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.775+0000 7f0bca600700 1 -- 172.21.15.31:0/1179946149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc41a7760 con 0x7f0bc41024e0 2024-04-25T01:14:42.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.776+0000 7f0bca600700 1 -- 172.21.15.31:0/1179946149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc4068f20 con 0x7f0bc41024e0 2024-04-25T01:14:42.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.778+0000 7f0bbbe00700 1 -- 172.21.15.31:0/1179946149 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0bb4042b10 con 0x7f0bc41024e0 2024-04-25T01:14:42.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.779+0000 7f0bbbe00700 1 --2- 172.21.15.31:0/1179946149 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0ba80a2cb0 0x7f0ba80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:42.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.780+0000 7f0bbbe00700 1 -- 172.21.15.31:0/1179946149 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f0bb40cf090 con 0x7f0bc41024e0 2024-04-25T01:14:42.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.780+0000 7f0bc2a00700 1 --2- 172.21.15.31:0/1179946149 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0ba80a2cb0 0x7f0ba80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:42.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.780+0000 7f0bc2a00700 1 --2- 172.21.15.31:0/1179946149 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0ba80a2cb0 0x7f0ba80a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009750 tx=0x7f0bb000b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:42.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:42.782+0000 7f0bbbe00700 1 -- 172.21.15.31:0/1179946149 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0bb401f3a0 con 0x7f0bc41024e0 2024-04-25T01:14:43.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:43.027+0000 7f0bca600700 1 -- 172.21.15.31:0/1179946149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f0bc402db30 con 0x7f0bc41024e0 2024-04-25T01:14:43.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:43.028+0000 7f0bbbe00700 1 -- 172.21.15.31:0/1179946149 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0bb409be80 con 0x7f0bc41024e0 2024-04-25T01:14:43.014 INFO:teuthology.orchestra.run.smithi031.stdout:150323855428 2024-04-25T01:14:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:43.030+0000 7f0bca600700 1 -- 172.21.15.31:0/1179946149 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0ba80a2cb0 msgr2=0x7f0ba80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:43.030+0000 7f0bca600700 1 --2- 172.21.15.31:0/1179946149 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0ba80a2cb0 0x7f0ba80a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009750 tx=0x7f0bb000b280 comp rx=0 tx=0).stop 2024-04-25T01:14:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:43.030+0000 7f0bca600700 1 -- 172.21.15.31:0/1179946149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bc41024e0 msgr2=0x7f0bc4074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:43.030+0000 7f0bca600700 1 --2- 172.21.15.31:0/1179946149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bc41024e0 0x7f0bc4074530 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400e9f0 tx=0x7f0bb400ed00 comp rx=0 tx=0).stop 2024-04-25T01:14:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:43.031+0000 7f0bca600700 1 -- 172.21.15.31:0/1179946149 shutdown_connections 2024-04-25T01:14:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:43.031+0000 7f0bca600700 1 --2- 172.21.15.31:0/1179946149 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f0ba80a2cb0 0x7f0ba80a5170 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:43.031+0000 7f0bca600700 1 --2- 172.21.15.31:0/1179946149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bc4107570 0x7f0bc4074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:43.031+0000 7f0bca600700 1 --2- 172.21.15.31:0/1179946149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bc41024e0 0x7f0bc4074530 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:43.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:43.031+0000 7f0bca600700 1 --2- 172.21.15.31:0/1179946149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bc4102e90 0x7f0bc4074a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:43.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:43.031+0000 7f0bca600700 1 -- 172.21.15.31:0/1179946149 >> 172.21.15.31:0/1179946149 conn(0x7f0bc4079390 msgr2=0x7f0bc4101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:43.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:43.031+0000 7f0bca600700 1 -- 172.21.15.31:0/1179946149 shutdown_connections 2024-04-25T01:14:43.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:43.031+0000 7f0bca600700 1 -- 172.21.15.31:0/1179946149 wait complete. 2024-04-25T01:14:43.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:42 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/129426684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T01:14:43.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:42 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/129426684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T01:14:43.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:42 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/129426684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T01:14:44.067 INFO:tasks.cephadm.ceph_manager.ceph:need seq 352187318297 got 352187318301 for osd.11 2024-04-25T01:14:44.068 DEBUG:teuthology.parallel:result is None 2024-04-25T01:14:44.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.102+0000 7f10a2e00700 1 -- 172.21.15.31:0/631808801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f109c104490 msgr2=0x7f109c104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:44.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.102+0000 7f10a2e00700 1 --2- 172.21.15.31:0/631808801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f109c104490 0x7f109c104910 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f10880099c0 tx=0x7f1088009cd0 comp rx=0 tx=0).stop 2024-04-25T01:14:44.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.102+0000 7f10a2e00700 1 -- 172.21.15.31:0/631808801 shutdown_connections 2024-04-25T01:14:44.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.102+0000 7f10a2e00700 1 --2- 172.21.15.31:0/631808801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f109c104e50 0x7f109c111310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:44.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.102+0000 7f10a2e00700 1 --2- 172.21.15.31:0/631808801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f109c10a540 0x7f109c10a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:44.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.102+0000 7f10a2e00700 1 --2- 172.21.15.31:0/631808801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f109c104490 0x7f109c104910 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:44.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.102+0000 7f10a2e00700 1 -- 172.21.15.31:0/631808801 >> 172.21.15.31:0/631808801 conn(0x7f109c0fff60 msgr2=0x7f109c102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:44.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.102+0000 7f10a2e00700 1 -- 172.21.15.31:0/631808801 shutdown_connections 2024-04-25T01:14:44.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.102+0000 7f10a2e00700 1 -- 172.21.15.31:0/631808801 wait complete. 2024-04-25T01:14:44.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.103+0000 7f10a2e00700 1 Processor -- start 2024-04-25T01:14:44.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.103+0000 7f10a2e00700 1 -- start start 2024-04-25T01:14:44.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.104+0000 7f10a2e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f109c104490 0x7f109c1a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:44.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.104+0000 7f10a2e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f109c104e50 0x7f109c1a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:44.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.104+0000 7f10a2e00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f109c10a540 0x7f109c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:44.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.104+0000 7f10a2e00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f109c1147e0 con 0x7f109c104490 2024-04-25T01:14:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.104+0000 7f10a2e00700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f109c114660 con 0x7f109c10a540 2024-04-25T01:14:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.104+0000 7f10a2e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f109c114960 con 0x7f109c104e50 2024-04-25T01:14:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.104+0000 7f109be00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f109c104490 0x7f109c1a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.104+0000 7f109be00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f109c104490 0x7f109c1a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42706/0 (socket says 172.21.15.31:42706) 2024-04-25T01:14:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.104+0000 7f109be00700 1 -- 172.21.15.31:0/3040687608 learned_addr learned my addr 172.21.15.31:0/3040687608 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:14:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.104+0000 7f109b400700 1 --2- 172.21.15.31:0/3040687608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f109c104e50 0x7f109c1a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.104+0000 7f10a0e00700 1 --2- 172.21.15.31:0/3040687608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f109c10a540 0x7f109c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.104+0000 7f109be00700 1 -- 172.21.15.31:0/3040687608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f109c104e50 msgr2=0x7f109c1a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.104+0000 7f109be00700 1 --2- 172.21.15.31:0/3040687608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f109c104e50 0x7f109c1a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.104+0000 7f109be00700 1 -- 172.21.15.31:0/3040687608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f109c10a540 msgr2=0x7f109c19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.104+0000 7f109be00700 1 --2- 172.21.15.31:0/3040687608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f109c10a540 0x7f109c19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.104+0000 7f109be00700 1 -- 172.21.15.31:0/3040687608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f109c19b1e0 con 0x7f109c104490 2024-04-25T01:14:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.104+0000 7f10a0e00700 1 --2- 172.21.15.31:0/3040687608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f109c10a540 0x7f109c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:14:44.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.104+0000 7f109b400700 1 --2- 172.21.15.31:0/3040687608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f109c104e50 0x7f109c1a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:14:44.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.105+0000 7f109be00700 1 --2- 172.21.15.31:0/3040687608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f109c104490 0x7f109c1a08c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f108c00e9c0 tx=0x7f108c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:44.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.105+0000 7f1098c00700 1 -- 172.21.15.31:0/3040687608 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c00cbd0 con 0x7f109c104490 2024-04-25T01:14:44.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.105+0000 7f1098c00700 1 -- 172.21.15.31:0/3040687608 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f108c00cd70 con 0x7f109c104490 2024-04-25T01:14:44.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.105+0000 7f1098c00700 1 -- 172.21.15.31:0/3040687608 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c010430 con 0x7f109c104490 2024-04-25T01:14:44.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.105+0000 7f10a2e00700 1 -- 172.21.15.31:0/3040687608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f109c19b4d0 con 0x7f109c104490 2024-04-25T01:14:44.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.105+0000 7f10a2e00700 1 -- 172.21.15.31:0/3040687608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f109c1a7790 con 0x7f109c104490 2024-04-25T01:14:44.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.107+0000 7f10a2e00700 1 -- 172.21.15.31:0/3040687608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f109c068f20 con 0x7f109c104490 2024-04-25T01:14:44.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.109+0000 7f1098c00700 1 -- 172.21.15.31:0/3040687608 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f108c004750 con 0x7f109c104490 2024-04-25T01:14:44.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.110+0000 7f1098c00700 1 --2- 172.21.15.31:0/3040687608 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f10800a2d30 0x7f10800a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:44.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.110+0000 7f1098c00700 1 -- 172.21.15.31:0/3040687608 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f108c0cf3a0 con 0x7f109c104490 2024-04-25T01:14:44.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.110+0000 7f109b400700 1 --2- 172.21.15.31:0/3040687608 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f10800a2d30 0x7f10800a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:44.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.111+0000 7f109b400700 1 --2- 172.21.15.31:0/3040687608 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f10800a2d30 0x7f10800a51f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f109c109bb0 tx=0x7f1088005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:44.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.115+0000 7f1098c00700 1 -- 172.21.15.31:0/3040687608 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f108c09c190 con 0x7f109c104490 2024-04-25T01:14:44.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:43 smithi169 ceph-mon[32759]: pgmap v354: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T01:14:44.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:43 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1179946149' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T01:14:44.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:43 smithi040 ceph-mon[32518]: pgmap v354: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T01:14:44.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:43 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1179946149' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T01:14:44.252 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:43 smithi031 ceph-mon[28053]: pgmap v354: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T01:14:44.252 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:43 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1179946149' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T01:14:44.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.350+0000 7f10a2e00700 1 -- 172.21.15.31:0/3040687608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f109c02db30 con 0x7f109c104490 2024-04-25T01:14:44.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.351+0000 7f1098c00700 1 -- 172.21.15.31:0/3040687608 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f108c09ba90 con 0x7f109c104490 2024-04-25T01:14:44.337 INFO:teuthology.orchestra.run.smithi031.stdout:296352743463 2024-04-25T01:14:44.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.354+0000 7f10a2e00700 1 -- 172.21.15.31:0/3040687608 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f10800a2d30 msgr2=0x7f10800a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:44.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.354+0000 7f10a2e00700 1 --2- 172.21.15.31:0/3040687608 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f10800a2d30 0x7f10800a51f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f109c109bb0 tx=0x7f1088005fb0 comp rx=0 tx=0).stop 2024-04-25T01:14:44.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.354+0000 7f10a2e00700 1 -- 172.21.15.31:0/3040687608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f109c104490 msgr2=0x7f109c1a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:44.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.354+0000 7f10a2e00700 1 --2- 172.21.15.31:0/3040687608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f109c104490 0x7f109c1a08c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f108c00e9c0 tx=0x7f108c00ecd0 comp rx=0 tx=0).stop 2024-04-25T01:14:44.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.354+0000 7f10a2e00700 1 -- 172.21.15.31:0/3040687608 shutdown_connections 2024-04-25T01:14:44.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.354+0000 7f10a2e00700 1 --2- 172.21.15.31:0/3040687608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f109c104490 0x7f109c1a08c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:44.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.354+0000 7f10a2e00700 1 --2- 172.21.15.31:0/3040687608 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f10800a2d30 0x7f10800a51f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:44.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.354+0000 7f10a2e00700 1 --2- 172.21.15.31:0/3040687608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f109c10a540 0x7f109c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:44.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.354+0000 7f10a2e00700 1 --2- 172.21.15.31:0/3040687608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f109c104e50 0x7f109c1a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:44.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.354+0000 7f10a2e00700 1 -- 172.21.15.31:0/3040687608 >> 172.21.15.31:0/3040687608 conn(0x7f109c0fff60 msgr2=0x7f109c101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:44.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.355+0000 7f10a2e00700 1 -- 172.21.15.31:0/3040687608 shutdown_connections 2024-04-25T01:14:44.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.355+0000 7f10a2e00700 1 -- 172.21.15.31:0/3040687608 wait complete. 2024-04-25T01:14:44.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.651+0000 7fa01f200700 1 -- 172.21.15.31:0/660809650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0181023b0 msgr2=0x7fa018102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:44.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.651+0000 7fa01f200700 1 --2- 172.21.15.31:0/660809650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0181023b0 0x7fa018102790 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa00c009990 tx=0x7fa00c009ca0 comp rx=0 tx=0).stop 2024-04-25T01:14:44.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.651+0000 7fa01f200700 1 -- 172.21.15.31:0/660809650 shutdown_connections 2024-04-25T01:14:44.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.651+0000 7fa01f200700 1 --2- 172.21.15.31:0/660809650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa018107490 0x7fa01810f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:44.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.651+0000 7fa01f200700 1 --2- 172.21.15.31:0/660809650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0181023b0 0x7fa018102790 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:44.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.651+0000 7fa01f200700 1 --2- 172.21.15.31:0/660809650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa018102d60 0x7fa018106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:44.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.651+0000 7fa01f200700 1 -- 172.21.15.31:0/660809650 >> 172.21.15.31:0/660809650 conn(0x7fa0180fdc80 msgr2=0x7fa0181000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:44.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.651+0000 7fa01f200700 1 -- 172.21.15.31:0/660809650 shutdown_connections 2024-04-25T01:14:44.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.651+0000 7fa01f200700 1 -- 172.21.15.31:0/660809650 wait complete. 2024-04-25T01:14:44.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.652+0000 7fa01f200700 1 Processor -- start 2024-04-25T01:14:44.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.652+0000 7fa01f200700 1 -- start start 2024-04-25T01:14:44.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.653+0000 7fa01f200700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0181023b0 0x7fa01819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:44.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.653+0000 7fa01f200700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa018102d60 0x7fa01819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:44.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.653+0000 7fa01f200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa018107490 0x7fa0181987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:44.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.653+0000 7fa01f200700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa0181124c0 con 0x7fa0181023b0 2024-04-25T01:14:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.653+0000 7fa01f200700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa018112340 con 0x7fa018102d60 2024-04-25T01:14:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.653+0000 7fa01f200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa018112640 con 0x7fa018107490 2024-04-25T01:14:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.653+0000 7fa01de00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0181023b0 0x7fa01819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.653+0000 7fa01de00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0181023b0 0x7fa01819e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42726/0 (socket says 172.21.15.31:42726) 2024-04-25T01:14:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.653+0000 7fa01de00700 1 -- 172.21.15.31:0/2070614925 learned_addr learned my addr 172.21.15.31:0/2070614925 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:14:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.653+0000 7fa01e800700 1 --2- 172.21.15.31:0/2070614925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa018107490 0x7fa0181987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.653+0000 7fa01d400700 1 --2- 172.21.15.31:0/2070614925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa018102d60 0x7fa01819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.653+0000 7fa01de00700 1 -- 172.21.15.31:0/2070614925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa018107490 msgr2=0x7fa0181987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.653+0000 7fa01de00700 1 --2- 172.21.15.31:0/2070614925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa018107490 0x7fa0181987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.653+0000 7fa01de00700 1 -- 172.21.15.31:0/2070614925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa018102d60 msgr2=0x7fa01819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.653+0000 7fa01de00700 1 --2- 172.21.15.31:0/2070614925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa018102d60 0x7fa01819eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.653+0000 7fa01de00700 1 -- 172.21.15.31:0/2070614925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa018198f80 con 0x7fa0181023b0 2024-04-25T01:14:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.654+0000 7fa01de00700 1 --2- 172.21.15.31:0/2070614925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0181023b0 0x7fa01819e620 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa00c000c00 tx=0x7fa00c005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.654+0000 7fa012a00700 1 -- 172.21.15.31:0/2070614925 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c027070 con 0x7fa0181023b0 2024-04-25T01:14:44.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.654+0000 7fa012a00700 1 -- 172.21.15.31:0/2070614925 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa00c0056b0 con 0x7fa0181023b0 2024-04-25T01:14:44.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.654+0000 7fa01f200700 1 -- 172.21.15.31:0/2070614925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa018199210 con 0x7fa0181023b0 2024-04-25T01:14:44.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.654+0000 7fa012a00700 1 -- 172.21.15.31:0/2070614925 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c017650 con 0x7fa0181023b0 2024-04-25T01:14:44.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.654+0000 7fa01f200700 1 -- 172.21.15.31:0/2070614925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0181a5480 con 0x7fa0181023b0 2024-04-25T01:14:44.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.656+0000 7fa01f200700 1 -- 172.21.15.31:0/2070614925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa018103890 con 0x7fa0181023b0 2024-04-25T01:14:44.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.659+0000 7fa012a00700 1 -- 172.21.15.31:0/2070614925 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa00c009d10 con 0x7fa0181023b0 2024-04-25T01:14:44.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.661+0000 7fa012a00700 1 --2- 172.21.15.31:0/2070614925 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa0000a2d30 0x7fa0000a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:44.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.661+0000 7fa012a00700 1 -- 172.21.15.31:0/2070614925 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fa00c0cfff0 con 0x7fa0181023b0 2024-04-25T01:14:44.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.661+0000 7fa01d400700 1 --2- 172.21.15.31:0/2070614925 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa0000a2d30 0x7fa0000a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:44.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.662+0000 7fa01d400700 1 --2- 172.21.15.31:0/2070614925 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa0000a2d30 0x7fa0000a51f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa008009680 tx=0x7fa008009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:44.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.663+0000 7fa012a00700 1 -- 172.21.15.31:0/2070614925 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa00c013070 con 0x7fa0181023b0 2024-04-25T01:14:44.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.899+0000 7fa01f200700 1 -- 172.21.15.31:0/2070614925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fa01802db30 con 0x7fa0181023b0 2024-04-25T01:14:44.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.899+0000 7fa012a00700 1 -- 172.21.15.31:0/2070614925 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa00c09cff0 con 0x7fa0181023b0 2024-04-25T01:14:44.886 INFO:teuthology.orchestra.run.smithi031.stdout:206158430265 2024-04-25T01:14:44.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.902+0000 7fa01f200700 1 -- 172.21.15.31:0/2070614925 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa0000a2d30 msgr2=0x7fa0000a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:44.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.902+0000 7fa01f200700 1 --2- 172.21.15.31:0/2070614925 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa0000a2d30 0x7fa0000a51f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa008009680 tx=0x7fa008009260 comp rx=0 tx=0).stop 2024-04-25T01:14:44.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.902+0000 7fa01f200700 1 -- 172.21.15.31:0/2070614925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0181023b0 msgr2=0x7fa01819e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:44.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.902+0000 7fa01f200700 1 --2- 172.21.15.31:0/2070614925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0181023b0 0x7fa01819e620 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa00c000c00 tx=0x7fa00c005590 comp rx=0 tx=0).stop 2024-04-25T01:14:44.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.902+0000 7fa01f200700 1 -- 172.21.15.31:0/2070614925 shutdown_connections 2024-04-25T01:14:44.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.902+0000 7fa01f200700 1 --2- 172.21.15.31:0/2070614925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0181023b0 0x7fa01819e620 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:44.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.902+0000 7fa01f200700 1 --2- 172.21.15.31:0/2070614925 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa0000a2d30 0x7fa0000a51f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:44.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.902+0000 7fa01f200700 1 --2- 172.21.15.31:0/2070614925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa018107490 0x7fa0181987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:44.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.902+0000 7fa01f200700 1 --2- 172.21.15.31:0/2070614925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa018102d60 0x7fa01819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:44.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.902+0000 7fa01f200700 1 -- 172.21.15.31:0/2070614925 >> 172.21.15.31:0/2070614925 conn(0x7fa0180fdc80 msgr2=0x7fa0180fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:44.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.903+0000 7fa01f200700 1 -- 172.21.15.31:0/2070614925 shutdown_connections 2024-04-25T01:14:44.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:44.903+0000 7fa01f200700 1 -- 172.21.15.31:0/2070614925 wait complete. 2024-04-25T01:14:45.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:44 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3040687608' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T01:14:45.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:44 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3040687608' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T01:14:45.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:44 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3040687608' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T01:14:46.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:45 smithi031 ceph-mon[28053]: pgmap v355: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T01:14:46.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:45 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2070614925' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T01:14:46.076 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005096 got 261993005100 for osd.8 2024-04-25T01:14:46.076 DEBUG:teuthology.parallel:result is None 2024-04-25T01:14:46.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:45 smithi169 ceph-mon[32759]: pgmap v355: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T01:14:46.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:45 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2070614925' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T01:14:46.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:45 smithi040 ceph-mon[32518]: pgmap v355: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T01:14:46.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:45 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2070614925' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T01:14:46.511 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743458 got 296352743463 for osd.9 2024-04-25T01:14:46.511 DEBUG:teuthology.parallel:result is None 2024-04-25T01:14:46.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.945+0000 7f37c0000700 1 -- 172.21.15.31:0/4054865433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37b8069f40 msgr2=0x7f37b806a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:46.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.945+0000 7f37c0000700 1 --2- 172.21.15.31:0/4054865433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37b8069f40 0x7f37b806a3a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009990 tx=0x7f37a4009ca0 comp rx=0 tx=0).stop 2024-04-25T01:14:46.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.946+0000 7f37c0000700 1 -- 172.21.15.31:0/4054865433 shutdown_connections 2024-04-25T01:14:46.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.946+0000 7f37c0000700 1 --2- 172.21.15.31:0/4054865433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37b806a8e0 0x7f37b810f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:46.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.946+0000 7f37c0000700 1 --2- 172.21.15.31:0/4054865433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37b81082b0 0x7f37b8108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:46.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.946+0000 7f37c0000700 1 --2- 172.21.15.31:0/4054865433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37b8069f40 0x7f37b806a3a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:46.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.946+0000 7f37c0000700 1 -- 172.21.15.31:0/4054865433 >> 172.21.15.31:0/4054865433 conn(0x7f37b80780e0 msgr2=0x7f37b80784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:46.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.946+0000 7f37c0000700 1 -- 172.21.15.31:0/4054865433 shutdown_connections 2024-04-25T01:14:46.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.946+0000 7f37c0000700 1 -- 172.21.15.31:0/4054865433 wait complete. 2024-04-25T01:14:46.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.947+0000 7f37c0000700 1 Processor -- start 2024-04-25T01:14:46.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.948+0000 7f37c0000700 1 -- start start 2024-04-25T01:14:46.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.948+0000 7f37c0000700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37b8069f40 0x7f37b81a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:46.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.948+0000 7f37c0000700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37b806a8e0 0x7f37b81a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:46.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.948+0000 7f37c0000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37b81082b0 0x7f37b819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:46.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.948+0000 7f37c0000700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f37b8112510 con 0x7f37b806a8e0 2024-04-25T01:14:46.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.948+0000 7f37c0000700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f37b8112390 con 0x7f37b8069f40 2024-04-25T01:14:46.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.948+0000 7f37c0000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f37b8112690 con 0x7f37b81082b0 2024-04-25T01:14:46.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.949+0000 7f37be000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37b81082b0 0x7f37b819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:46.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.949+0000 7f37bd600700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37b8069f40 0x7f37b81a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:46.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.949+0000 7f37be000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37b81082b0 0x7f37b819aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:45750/0 (socket says 172.21.15.31:45750) 2024-04-25T01:14:46.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.949+0000 7f37be000700 1 -- 172.21.15.31:0/1343010699 learned_addr learned my addr 172.21.15.31:0/1343010699 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:14:46.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.949+0000 7f37bd600700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37b8069f40 0x7f37b81a08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:50116/0 (socket says 172.21.15.31:50116) 2024-04-25T01:14:46.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.949+0000 7f37bcc00700 1 --2- 172.21.15.31:0/1343010699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37b806a8e0 0x7f37b81a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:46.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.949+0000 7f37be000700 1 -- 172.21.15.31:0/1343010699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37b8069f40 msgr2=0x7f37b81a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:46.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.949+0000 7f37be000700 1 --2- 172.21.15.31:0/1343010699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37b8069f40 0x7f37b81a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:46.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.949+0000 7f37be000700 1 -- 172.21.15.31:0/1343010699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37b806a8e0 msgr2=0x7f37b81a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:46.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.949+0000 7f37be000700 1 --2- 172.21.15.31:0/1343010699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37b806a8e0 0x7f37b81a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:46.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.949+0000 7f37be000700 1 -- 172.21.15.31:0/1343010699 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b819b200 con 0x7f37b81082b0 2024-04-25T01:14:46.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.949+0000 7f37be000700 1 --2- 172.21.15.31:0/1343010699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37b81082b0 0x7f37b819aaa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f37b000ef30 tx=0x7f37b000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:46.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.950+0000 7f37b6000700 1 -- 172.21.15.31:0/1343010699 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b00092e0 con 0x7f37b81082b0 2024-04-25T01:14:46.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.950+0000 7f37b6000700 1 -- 172.21.15.31:0/1343010699 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b0004bb0 con 0x7f37b81082b0 2024-04-25T01:14:46.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.950+0000 7f37c0000700 1 -- 172.21.15.31:0/1343010699 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b819b490 con 0x7f37b81082b0 2024-04-25T01:14:46.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.950+0000 7f37b6000700 1 -- 172.21.15.31:0/1343010699 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b0004280 con 0x7f37b81082b0 2024-04-25T01:14:46.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.950+0000 7f37c0000700 1 -- 172.21.15.31:0/1343010699 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b81a77b0 con 0x7f37b81082b0 2024-04-25T01:14:46.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.951+0000 7f37c0000700 1 -- 172.21.15.31:0/1343010699 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3798005330 con 0x7f37b81082b0 2024-04-25T01:14:46.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.953+0000 7f37b6000700 1 -- 172.21.15.31:0/1343010699 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f37b001b030 con 0x7f37b81082b0 2024-04-25T01:14:46.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.955+0000 7f37b6000700 1 --2- 172.21.15.31:0/1343010699 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f37a00a2d30 0x7f37a00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:46.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.955+0000 7f37b6000700 1 -- 172.21.15.31:0/1343010699 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f37b00ceb30 con 0x7f37b81082b0 2024-04-25T01:14:46.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.955+0000 7f37bd600700 1 --2- 172.21.15.31:0/1343010699 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f37a00a2d30 0x7f37a00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:46.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.956+0000 7f37bd600700 1 --2- 172.21.15.31:0/1343010699 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f37a00a2d30 0x7f37a00a51f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f37b819bbe0 tx=0x7f37ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:46.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:46.959+0000 7f37b6000700 1 -- 172.21.15.31:0/1343010699 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f37b0022020 con 0x7f37b81082b0 2024-04-25T01:14:47.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:47.232+0000 7f37c0000700 1 -- 172.21.15.31:0/1343010699 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f37980058f0 con 0x7f37b81082b0 2024-04-25T01:14:47.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:47.233+0000 7f37b6000700 1 -- 172.21.15.31:0/1343010699 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f37b0095e20 con 0x7f37b81082b0 2024-04-25T01:14:47.219 INFO:teuthology.orchestra.run.smithi031.stdout:34359738458 2024-04-25T01:14:47.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:47.235+0000 7f37c0000700 1 -- 172.21.15.31:0/1343010699 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f37a00a2d30 msgr2=0x7f37a00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:47.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:47.235+0000 7f37c0000700 1 --2- 172.21.15.31:0/1343010699 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f37a00a2d30 0x7f37a00a51f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f37b819bbe0 tx=0x7f37ac009260 comp rx=0 tx=0).stop 2024-04-25T01:14:47.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:47.235+0000 7f37c0000700 1 -- 172.21.15.31:0/1343010699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37b81082b0 msgr2=0x7f37b819aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:47.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:47.235+0000 7f37c0000700 1 --2- 172.21.15.31:0/1343010699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37b81082b0 0x7f37b819aaa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f37b000ef30 tx=0x7f37b000c3a0 comp rx=0 tx=0).stop 2024-04-25T01:14:47.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:47.235+0000 7f37c0000700 1 -- 172.21.15.31:0/1343010699 shutdown_connections 2024-04-25T01:14:47.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:47.235+0000 7f37c0000700 1 --2- 172.21.15.31:0/1343010699 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f37a00a2d30 0x7f37a00a51f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:47.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:47.235+0000 7f37c0000700 1 --2- 172.21.15.31:0/1343010699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37b81082b0 0x7f37b819aaa0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:47.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:47.235+0000 7f37c0000700 1 --2- 172.21.15.31:0/1343010699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37b8069f40 0x7f37b81a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:47.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:47.235+0000 7f37c0000700 1 --2- 172.21.15.31:0/1343010699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37b806a8e0 0x7f37b81a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:47.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:47.235+0000 7f37c0000700 1 -- 172.21.15.31:0/1343010699 >> 172.21.15.31:0/1343010699 conn(0x7f37b80780e0 msgr2=0x7f37b80ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:47.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:47.236+0000 7f37c0000700 1 -- 172.21.15.31:0/1343010699 shutdown_connections 2024-04-25T01:14:47.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:47.236+0000 7f37c0000700 1 -- 172.21.15.31:0/1343010699 wait complete. 2024-04-25T01:14:47.780 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855422 got 150323855428 for osd.4 2024-04-25T01:14:47.780 DEBUG:teuthology.parallel:result is None 2024-04-25T01:14:47.917 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411403 got 77309411407 for osd.2 2024-04-25T01:14:47.917 DEBUG:teuthology.parallel:result is None 2024-04-25T01:14:48.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:47 smithi031 ceph-mon[28053]: pgmap v356: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T01:14:48.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:47 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1343010699' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T01:14:48.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:47 smithi169 ceph-mon[32759]: pgmap v356: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T01:14:48.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:47 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1343010699' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T01:14:48.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:47 smithi040 ceph-mon[32518]: pgmap v356: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T01:14:48.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:47 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1343010699' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T01:14:48.966 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:14:49.009 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738453 got 34359738458 for osd.0 2024-04-25T01:14:49.009 DEBUG:teuthology.parallel:result is None 2024-04-25T01:14:49.563 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117060 got 115964117063 for osd.3 2024-04-25T01:14:49.564 DEBUG:teuthology.parallel:result is None 2024-04-25T01:14:49.640 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:14:50.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:49 smithi031 ceph-mon[28053]: pgmap v357: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:14:50.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:49 smithi169 ceph-mon[32759]: pgmap v357: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:14:50.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:49 smithi040 ceph-mon[32518]: pgmap v357: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:14:50.318 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430259 got 206158430265 for osd.6 2024-04-25T01:14:50.319 DEBUG:teuthology.parallel:result is None 2024-04-25T01:14:51.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.116+0000 7fc17f800700 1 -- 172.21.15.31:0/3098632395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1781082b0 msgr2=0x7fc178108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:51.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.116+0000 7fc17f800700 1 --2- 172.21.15.31:0/3098632395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1781082b0 0x7fc178108690 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009990 tx=0x7fc16c009ca0 comp rx=0 tx=0).stop 2024-04-25T01:14:51.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.116+0000 7fc17f800700 1 -- 172.21.15.31:0/3098632395 shutdown_connections 2024-04-25T01:14:51.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.116+0000 7fc17f800700 1 --2- 172.21.15.31:0/3098632395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc17806a8e0 0x7fc17810f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:51.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.116+0000 7fc17f800700 1 --2- 172.21.15.31:0/3098632395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1781082b0 0x7fc178108690 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:51.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.116+0000 7fc17f800700 1 --2- 172.21.15.31:0/3098632395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc178069f40 0x7fc17806a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:51.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.116+0000 7fc17f800700 1 -- 172.21.15.31:0/3098632395 >> 172.21.15.31:0/3098632395 conn(0x7fc1780780e0 msgr2=0x7fc1780784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:51.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.116+0000 7fc17f800700 1 -- 172.21.15.31:0/3098632395 shutdown_connections 2024-04-25T01:14:51.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.117+0000 7fc17f800700 1 -- 172.21.15.31:0/3098632395 wait complete. 2024-04-25T01:14:51.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.118+0000 7fc17f800700 1 Processor -- start 2024-04-25T01:14:51.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.118+0000 7fc17f800700 1 -- start start 2024-04-25T01:14:51.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.118+0000 7fc17f800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc178069f40 0x7fc1781a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:51.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.118+0000 7fc17f800700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc17806a8e0 0x7fc1781a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:51.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.118+0000 7fc17f800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1781082b0 0x7fc17819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:51.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.118+0000 7fc17f800700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc178112540 con 0x7fc178069f40 2024-04-25T01:14:51.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.118+0000 7fc17f800700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc1781123c0 con 0x7fc17806a8e0 2024-04-25T01:14:51.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.118+0000 7fc17f800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc1781126c0 con 0x7fc1781082b0 2024-04-25T01:14:51.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.119+0000 7fc17d800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1781082b0 0x7fc17819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:51.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.119+0000 7fc17ce00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc178069f40 0x7fc1781a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:51.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.119+0000 7fc17d800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1781082b0 0x7fc17819aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:45762/0 (socket says 172.21.15.31:45762) 2024-04-25T01:14:51.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.119+0000 7fc17d800700 1 -- 172.21.15.31:0/4125587757 learned_addr learned my addr 172.21.15.31:0/4125587757 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:14:51.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.119+0000 7fc177e00700 1 --2- 172.21.15.31:0/4125587757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc17806a8e0 0x7fc1781a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:51.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.119+0000 7fc17d800700 1 -- 172.21.15.31:0/4125587757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc17806a8e0 msgr2=0x7fc1781a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:51.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.119+0000 7fc17d800700 1 --2- 172.21.15.31:0/4125587757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc17806a8e0 0x7fc1781a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:51.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.119+0000 7fc17d800700 1 -- 172.21.15.31:0/4125587757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc178069f40 msgr2=0x7fc1781a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:51.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.119+0000 7fc17d800700 1 --2- 172.21.15.31:0/4125587757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc178069f40 0x7fc1781a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:51.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.119+0000 7fc17d800700 1 -- 172.21.15.31:0/4125587757 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc17819b1a0 con 0x7fc1781082b0 2024-04-25T01:14:51.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.119+0000 7fc17d800700 1 --2- 172.21.15.31:0/4125587757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1781082b0 0x7fc17819aa40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc17000ef30 tx=0x7fc17000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:51.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.119+0000 7fc175600700 1 -- 172.21.15.31:0/4125587757 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1700092e0 con 0x7fc1781082b0 2024-04-25T01:14:51.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.120+0000 7fc175600700 1 -- 172.21.15.31:0/4125587757 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc170004bb0 con 0x7fc1781082b0 2024-04-25T01:14:51.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.120+0000 7fc175600700 1 -- 172.21.15.31:0/4125587757 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170004280 con 0x7fc1781082b0 2024-04-25T01:14:51.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.120+0000 7fc17f800700 1 -- 172.21.15.31:0/4125587757 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc17819b430 con 0x7fc1781082b0 2024-04-25T01:14:51.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.120+0000 7fc17f800700 1 -- 172.21.15.31:0/4125587757 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1781a7750 con 0x7fc1781082b0 2024-04-25T01:14:51.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.121+0000 7fc17f800700 1 -- 172.21.15.31:0/4125587757 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc178068f20 con 0x7fc1781082b0 2024-04-25T01:14:51.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.123+0000 7fc175600700 1 -- 172.21.15.31:0/4125587757 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc17001b030 con 0x7fc1781082b0 2024-04-25T01:14:51.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.124+0000 7fc175600700 1 --2- 172.21.15.31:0/4125587757 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc1600a2d30 0x7fc1600a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:51.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.124+0000 7fc175600700 1 -- 172.21.15.31:0/4125587757 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fc1700cefe0 con 0x7fc1781082b0 2024-04-25T01:14:51.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.124+0000 7fc17ce00700 1 --2- 172.21.15.31:0/4125587757 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc1600a2d30 0x7fc1600a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:51.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.125+0000 7fc17ce00700 1 --2- 172.21.15.31:0/4125587757 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc1600a2d30 0x7fc1600a51f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc17819bb80 tx=0x7fc16c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:51.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.127+0000 7fc175600700 1 -- 172.21.15.31:0/4125587757 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc170022020 con 0x7fc1781082b0 2024-04-25T01:14:51.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.372+0000 7fc17f800700 1 -- 172.21.15.31:0/4125587757 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc1781a7920 con 0x7fc1781082b0 2024-04-25T01:14:51.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.373+0000 7fc175600700 1 -- 172.21.15.31:0/4125587757 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc170010040 con 0x7fc1781082b0 2024-04-25T01:14:51.359 INFO:teuthology.orchestra.run.smithi031.stdout:55834574934 2024-04-25T01:14:51.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.376+0000 7fc17f800700 1 -- 172.21.15.31:0/4125587757 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc1600a2d30 msgr2=0x7fc1600a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:51.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.376+0000 7fc17f800700 1 --2- 172.21.15.31:0/4125587757 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc1600a2d30 0x7fc1600a51f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc17819bb80 tx=0x7fc16c01a040 comp rx=0 tx=0).stop 2024-04-25T01:14:51.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.376+0000 7fc17f800700 1 -- 172.21.15.31:0/4125587757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1781082b0 msgr2=0x7fc17819aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:51.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.376+0000 7fc17f800700 1 --2- 172.21.15.31:0/4125587757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1781082b0 0x7fc17819aa40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc17000ef30 tx=0x7fc17000c3a0 comp rx=0 tx=0).stop 2024-04-25T01:14:51.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.376+0000 7fc17f800700 1 -- 172.21.15.31:0/4125587757 shutdown_connections 2024-04-25T01:14:51.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.376+0000 7fc17f800700 1 --2- 172.21.15.31:0/4125587757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc178069f40 0x7fc1781a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:51.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.376+0000 7fc17f800700 1 --2- 172.21.15.31:0/4125587757 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc1600a2d30 0x7fc1600a51f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:51.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.376+0000 7fc17f800700 1 --2- 172.21.15.31:0/4125587757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1781082b0 0x7fc17819aa40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:51.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.376+0000 7fc17f800700 1 --2- 172.21.15.31:0/4125587757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc17806a8e0 0x7fc1781a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:51.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.376+0000 7fc17f800700 1 -- 172.21.15.31:0/4125587757 >> 172.21.15.31:0/4125587757 conn(0x7fc1780780e0 msgr2=0x7fc17810d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:51.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.376+0000 7fc17f800700 1 -- 172.21.15.31:0/4125587757 shutdown_connections 2024-04-25T01:14:51.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.376+0000 7fc17f800700 1 -- 172.21.15.31:0/4125587757 wait complete. 2024-04-25T01:14:51.988 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:51 smithi031 ceph-mon[28053]: pgmap v358: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:14:51.988 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:51 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/4125587757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T01:14:52.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:51 smithi169 ceph-mon[32759]: pgmap v358: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:14:52.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:51 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/4125587757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T01:14:52.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:51 smithi040 ceph-mon[32518]: pgmap v358: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:14:52.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:51 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/4125587757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T01:14:52.887 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574928 got 55834574934 for osd.1 2024-04-25T01:14:52.887 DEBUG:teuthology.parallel:result is None 2024-04-25T01:14:53.180 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:52 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2150477258' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T01:14:53.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:52 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2150477258' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T01:14:53.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:52 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2150477258' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T01:14:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.749+0000 7f1182200700 1 -- 172.21.15.31:0/371863154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f117c074d80 msgr2=0x7f117c112ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.749+0000 7f1182200700 1 --2- 172.21.15.31:0/371863154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f117c074d80 0x7f117c112ec0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1164009990 tx=0x7f1164009ca0 comp rx=0 tx=0).stop 2024-04-25T01:14:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.750+0000 7f1182200700 1 -- 172.21.15.31:0/371863154 shutdown_connections 2024-04-25T01:14:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.750+0000 7f1182200700 1 --2- 172.21.15.31:0/371863154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f117c113400 0x7f117c1157f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.750+0000 7f1182200700 1 --2- 172.21.15.31:0/371863154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f117c074460 0x7f117c074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.750+0000 7f1182200700 1 --2- 172.21.15.31:0/371863154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f117c074d80 0x7f117c112ec0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.750+0000 7f1182200700 1 -- 172.21.15.31:0/371863154 >> 172.21.15.31:0/371863154 conn(0x7f117c0793e0 msgr2=0x7f117c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:53.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.750+0000 7f1182200700 1 -- 172.21.15.31:0/371863154 shutdown_connections 2024-04-25T01:14:53.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.750+0000 7f1182200700 1 -- 172.21.15.31:0/371863154 wait complete. 2024-04-25T01:14:53.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.751+0000 7f1182200700 1 Processor -- start 2024-04-25T01:14:53.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.751+0000 7f1182200700 1 -- start start 2024-04-25T01:14:53.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.751+0000 7f1182200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f117c074460 0x7f117c19f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:53.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.751+0000 7f1182200700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f117c074d80 0x7f117c19f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:53.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.752+0000 7f1182200700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f117c113400 0x7f117c1a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:53.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.752+0000 7f1182200700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f117c118be0 con 0x7f117c074d80 2024-04-25T01:14:53.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.752+0000 7f1182200700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f117c118a60 con 0x7f117c113400 2024-04-25T01:14:53.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.752+0000 7f1182200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f117c118d60 con 0x7f117c074460 2024-04-25T01:14:53.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.752+0000 7f1172a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f117c074d80 0x7f117c19f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:53.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.752+0000 7f1172a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f117c074d80 0x7f117c19f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41426/0 (socket says 172.21.15.31:41426) 2024-04-25T01:14:53.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.752+0000 7f1172a00700 1 -- 172.21.15.31:0/2150477258 learned_addr learned my addr 172.21.15.31:0/2150477258 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:14:53.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.752+0000 7f117be00700 1 --2- 172.21.15.31:0/2150477258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f117c113400 0x7f117c1a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:53.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.752+0000 7f117b400700 1 --2- 172.21.15.31:0/2150477258 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f117c074460 0x7f117c19f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:53.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.752+0000 7f1172a00700 1 -- 172.21.15.31:0/2150477258 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f117c074460 msgr2=0x7f117c19f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:53.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.752+0000 7f1172a00700 1 --2- 172.21.15.31:0/2150477258 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f117c074460 0x7f117c19f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:53.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.752+0000 7f1172a00700 1 -- 172.21.15.31:0/2150477258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f117c113400 msgr2=0x7f117c1a3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:53.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.752+0000 7f1172a00700 1 --2- 172.21.15.31:0/2150477258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f117c113400 0x7f117c1a3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:53.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.752+0000 7f1172a00700 1 -- 172.21.15.31:0/2150477258 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f117c1a4040 con 0x7f117c074d80 2024-04-25T01:14:53.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.752+0000 7f117be00700 1 --2- 172.21.15.31:0/2150477258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f117c113400 0x7f117c1a3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:14:53.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.753+0000 7f1172a00700 1 --2- 172.21.15.31:0/2150477258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f117c074d80 0x7f117c19f570 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1164000c00 tx=0x7f116400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:53.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.753+0000 7f1178c00700 1 -- 172.21.15.31:0/2150477258 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164027070 con 0x7f117c074d80 2024-04-25T01:14:53.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.753+0000 7f1182200700 1 -- 172.21.15.31:0/2150477258 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f117c1a42d0 con 0x7f117c074d80 2024-04-25T01:14:53.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.753+0000 7f1178c00700 1 -- 172.21.15.31:0/2150477258 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11640056d0 con 0x7f117c074d80 2024-04-25T01:14:53.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.753+0000 7f1178c00700 1 -- 172.21.15.31:0/2150477258 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164022e10 con 0x7f117c074d80 2024-04-25T01:14:53.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.753+0000 7f1182200700 1 -- 172.21.15.31:0/2150477258 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f117c1abbb0 con 0x7f117c074d80 2024-04-25T01:14:53.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.755+0000 7f1182200700 1 -- 172.21.15.31:0/2150477258 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f117c068f20 con 0x7f117c074d80 2024-04-25T01:14:53.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.757+0000 7f1178c00700 1 -- 172.21.15.31:0/2150477258 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f116402a070 con 0x7f117c074d80 2024-04-25T01:14:53.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.759+0000 7f1178c00700 1 --2- 172.21.15.31:0/2150477258 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f11600a2cb0 0x7f11600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:53.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.759+0000 7f1178c00700 1 -- 172.21.15.31:0/2150477258 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f11640d0ab0 con 0x7f117c074d80 2024-04-25T01:14:53.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.759+0000 7f117b400700 1 --2- 172.21.15.31:0/2150477258 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f11600a2cb0 0x7f11600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:53.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.760+0000 7f117b400700 1 --2- 172.21.15.31:0/2150477258 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f11600a2cb0 0x7f11600a5170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f116c009c60 tx=0x7f116c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:53.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.762+0000 7f1178c00700 1 -- 172.21.15.31:0/2150477258 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1164031070 con 0x7f117c074d80 2024-04-25T01:14:53.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.999+0000 7f1182200700 1 -- 172.21.15.31:0/2150477258 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f117c1abd80 con 0x7f117c074d80 2024-04-25T01:14:53.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:51.999+0000 7f1178c00700 1 -- 172.21.15.31:0/2150477258 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f1164013070 con 0x7f117c074d80 2024-04-25T01:14:53.402 INFO:teuthology.orchestra.run.smithi031.stdout:317827579940 2024-04-25T01:14:53.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:52.002+0000 7f1182200700 1 -- 172.21.15.31:0/2150477258 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f11600a2cb0 msgr2=0x7f11600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:53.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:52.002+0000 7f1182200700 1 --2- 172.21.15.31:0/2150477258 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f11600a2cb0 0x7f11600a5170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f116c009c60 tx=0x7f116c009260 comp rx=0 tx=0).stop 2024-04-25T01:14:53.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:52.002+0000 7f1182200700 1 -- 172.21.15.31:0/2150477258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f117c074d80 msgr2=0x7f117c19f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:53.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:52.002+0000 7f1182200700 1 --2- 172.21.15.31:0/2150477258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f117c074d80 0x7f117c19f570 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1164000c00 tx=0x7f116400bab0 comp rx=0 tx=0).stop 2024-04-25T01:14:53.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:52.002+0000 7f1182200700 1 -- 172.21.15.31:0/2150477258 shutdown_connections 2024-04-25T01:14:53.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:52.002+0000 7f1182200700 1 --2- 172.21.15.31:0/2150477258 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f11600a2cb0 0x7f11600a5170 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:53.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:52.002+0000 7f1182200700 1 --2- 172.21.15.31:0/2150477258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f117c113400 0x7f117c1a3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:53.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:52.002+0000 7f1182200700 1 --2- 172.21.15.31:0/2150477258 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f117c074460 0x7f117c19f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:53.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:52.002+0000 7f1182200700 1 --2- 172.21.15.31:0/2150477258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f117c074d80 0x7f117c19f570 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:53.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:52.002+0000 7f1182200700 1 -- 172.21.15.31:0/2150477258 >> 172.21.15.31:0/2150477258 conn(0x7f117c0793e0 msgr2=0x7f117c112150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:53.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:52.003+0000 7f1182200700 1 -- 172.21.15.31:0/2150477258 shutdown_connections 2024-04-25T01:14:53.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:52.003+0000 7f1182200700 1 -- 172.21.15.31:0/2150477258 wait complete. 2024-04-25T01:14:54.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:53 smithi031 ceph-mon[28053]: pgmap v359: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:14:54.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:53 smithi169 ceph-mon[32759]: pgmap v359: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:14:54.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:53 smithi040 ceph-mon[32518]: pgmap v359: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:14:54.371 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579932 got 317827579940 for osd.10 2024-04-25T01:14:54.371 DEBUG:teuthology.parallel:result is None 2024-04-25T01:14:56.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:55 smithi031 ceph-mon[28053]: pgmap v360: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:14:56.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:55 smithi169 ceph-mon[32759]: pgmap v360: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:14:56.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:55 smithi040 ceph-mon[32518]: pgmap v360: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:14:56.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.595+0000 7f972ea00700 1 -- 172.21.15.31:0/4253088209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9728074d80 msgr2=0x7f972810e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:56.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.595+0000 7f972ea00700 1 --2- 172.21.15.31:0/4253088209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9728074d80 0x7f972810e6f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9710009990 tx=0x7f9710009ca0 comp rx=0 tx=0).stop 2024-04-25T01:14:56.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.595+0000 7f972ea00700 1 -- 172.21.15.31:0/4253088209 shutdown_connections 2024-04-25T01:14:56.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.595+0000 7f972ea00700 1 --2- 172.21.15.31:0/4253088209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972810ec30 0x7f9728111020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:56.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.595+0000 7f972ea00700 1 --2- 172.21.15.31:0/4253088209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9728074460 0x7f9728074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:56.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.595+0000 7f972ea00700 1 --2- 172.21.15.31:0/4253088209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9728074d80 0x7f972810e6f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:56.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.595+0000 7f972ea00700 1 -- 172.21.15.31:0/4253088209 >> 172.21.15.31:0/4253088209 conn(0x7f9728079390 msgr2=0x7f97280797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:56.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.596+0000 7f972ea00700 1 -- 172.21.15.31:0/4253088209 shutdown_connections 2024-04-25T01:14:56.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.596+0000 7f972ea00700 1 -- 172.21.15.31:0/4253088209 wait complete. 2024-04-25T01:14:56.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.597+0000 7f972ea00700 1 Processor -- start 2024-04-25T01:14:56.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.597+0000 7f972ea00700 1 -- start start 2024-04-25T01:14:56.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.598+0000 7f972ea00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9728074460 0x7f97281a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:56.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.598+0000 7f972ea00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9728074d80 0x7f97281a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:56.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.598+0000 7f972ea00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972810ec30 0x7f972819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:56.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.598+0000 7f972ea00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f97281147a0 con 0x7f9728074d80 2024-04-25T01:14:56.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.598+0000 7f972ea00700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9728114620 con 0x7f972810ec30 2024-04-25T01:14:56.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.598+0000 7f972ea00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9728114920 con 0x7f9728074460 2024-04-25T01:14:56.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.598+0000 7f972ca00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972810ec30 0x7f972819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:56.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.598+0000 7f9727e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9728074460 0x7f97281a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:56.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.598+0000 7f972ca00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972810ec30 0x7f972819aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:50182/0 (socket says 172.21.15.31:50182) 2024-04-25T01:14:56.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.598+0000 7f972ca00700 1 -- 172.21.15.31:0/945964919 learned_addr learned my addr 172.21.15.31:0/945964919 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:14:56.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.599+0000 7f971f400700 1 --2- 172.21.15.31:0/945964919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9728074d80 0x7f97281a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:56.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.599+0000 7f972ca00700 1 -- 172.21.15.31:0/945964919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9728074460 msgr2=0x7f97281a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:56.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.599+0000 7f972ca00700 1 --2- 172.21.15.31:0/945964919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9728074460 0x7f97281a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:56.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.599+0000 7f972ca00700 1 -- 172.21.15.31:0/945964919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9728074d80 msgr2=0x7f97281a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:56.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.599+0000 7f972ca00700 1 --2- 172.21.15.31:0/945964919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9728074d80 0x7f97281a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:56.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.599+0000 7f972ca00700 1 -- 172.21.15.31:0/945964919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972819b200 con 0x7f972810ec30 2024-04-25T01:14:56.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.599+0000 7f9727e00700 1 --2- 172.21.15.31:0/945964919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9728074460 0x7f97281a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:14:56.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.599+0000 7f972ca00700 1 --2- 172.21.15.31:0/945964919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972810ec30 0x7f972819aaa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f972000d740 tx=0x7f972000da50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:56.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.599+0000 7f9725600700 1 -- 172.21.15.31:0/945964919 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9720016070 con 0x7f972810ec30 2024-04-25T01:14:56.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.599+0000 7f9725600700 1 -- 172.21.15.31:0/945964919 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f972000fb70 con 0x7f972810ec30 2024-04-25T01:14:56.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.599+0000 7f972ea00700 1 -- 172.21.15.31:0/945964919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f972819b4f0 con 0x7f972810ec30 2024-04-25T01:14:56.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.600+0000 7f9725600700 1 -- 172.21.15.31:0/945964919 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9720003b50 con 0x7f972810ec30 2024-04-25T01:14:56.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.600+0000 7f972ea00700 1 -- 172.21.15.31:0/945964919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97281a7760 con 0x7f972810ec30 2024-04-25T01:14:56.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.601+0000 7f972ea00700 1 -- 172.21.15.31:0/945964919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9728068f20 con 0x7f972810ec30 2024-04-25T01:14:56.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.605+0000 7f9725600700 1 -- 172.21.15.31:0/945964919 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9720003cf0 con 0x7f972810ec30 2024-04-25T01:14:56.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.606+0000 7f9725600700 1 --2- 172.21.15.31:0/945964919 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f970c0a2d30 0x7f970c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:56.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.606+0000 7f9725600700 1 -- 172.21.15.31:0/945964919 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f97200cebf0 con 0x7f972810ec30 2024-04-25T01:14:56.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.606+0000 7f9727e00700 1 --2- 172.21.15.31:0/945964919 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f970c0a2d30 0x7f970c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:56.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.607+0000 7f9727e00700 1 --2- 172.21.15.31:0/945964919 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f970c0a2d30 0x7f970c0a51f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f97281017f0 tx=0x7f971800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:56.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.611+0000 7f9725600700 1 -- 172.21.15.31:0/945964919 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f972009b9e0 con 0x7f972810ec30 2024-04-25T01:14:56.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.850+0000 7f972ea00700 1 -- 172.21.15.31:0/945964919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f972802db30 con 0x7f972810ec30 2024-04-25T01:14:56.836 INFO:teuthology.orchestra.run.smithi031.stdout:227633266741 2024-04-25T01:14:56.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.850+0000 7f9725600700 1 -- 172.21.15.31:0/945964919 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f972009b2e0 con 0x7f972810ec30 2024-04-25T01:14:56.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.853+0000 7f972ea00700 1 -- 172.21.15.31:0/945964919 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f970c0a2d30 msgr2=0x7f970c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:56.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.853+0000 7f972ea00700 1 --2- 172.21.15.31:0/945964919 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f970c0a2d30 0x7f970c0a51f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f97281017f0 tx=0x7f971800b040 comp rx=0 tx=0).stop 2024-04-25T01:14:56.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.853+0000 7f972ea00700 1 -- 172.21.15.31:0/945964919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972810ec30 msgr2=0x7f972819aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:56.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.853+0000 7f972ea00700 1 --2- 172.21.15.31:0/945964919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972810ec30 0x7f972819aaa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f972000d740 tx=0x7f972000da50 comp rx=0 tx=0).stop 2024-04-25T01:14:56.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.853+0000 7f972ea00700 1 -- 172.21.15.31:0/945964919 shutdown_connections 2024-04-25T01:14:56.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.853+0000 7f972ea00700 1 --2- 172.21.15.31:0/945964919 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f970c0a2d30 0x7f970c0a51f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:56.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.853+0000 7f972ea00700 1 --2- 172.21.15.31:0/945964919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972810ec30 0x7f972819aaa0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:56.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.853+0000 7f972ea00700 1 --2- 172.21.15.31:0/945964919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9728074460 0x7f97281a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:56.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.853+0000 7f972ea00700 1 --2- 172.21.15.31:0/945964919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9728074d80 0x7f97281a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:56.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.853+0000 7f972ea00700 1 -- 172.21.15.31:0/945964919 >> 172.21.15.31:0/945964919 conn(0x7f9728079390 msgr2=0x7f97280ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:56.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.854+0000 7f972ea00700 1 -- 172.21.15.31:0/945964919 shutdown_connections 2024-04-25T01:14:56.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.854+0000 7f972ea00700 1 -- 172.21.15.31:0/945964919 wait complete. 2024-04-25T01:14:56.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.456+0000 7f6c8d400700 1 -- 172.21.15.31:0/3282615279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c881023c0 msgr2=0x7f6c881027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:56.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.456+0000 7f6c8d400700 1 --2- 172.21.15.31:0/3282615279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c881023c0 0x7f6c881027a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6c780099c0 tx=0x7f6c78009cd0 comp rx=0 tx=0).stop 2024-04-25T01:14:56.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.456+0000 7f6c8d400700 1 -- 172.21.15.31:0/3282615279 shutdown_connections 2024-04-25T01:14:56.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.456+0000 7f6c8d400700 1 --2- 172.21.15.31:0/3282615279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c881074a0 0x7f6c8810f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:56.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.456+0000 7f6c8d400700 1 --2- 172.21.15.31:0/3282615279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c881023c0 0x7f6c881027a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:56.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.456+0000 7f6c8d400700 1 --2- 172.21.15.31:0/3282615279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c88102d70 0x7f6c88106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:56.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.456+0000 7f6c8d400700 1 -- 172.21.15.31:0/3282615279 >> 172.21.15.31:0/3282615279 conn(0x7f6c880fdcb0 msgr2=0x7f6c881000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:56.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.456+0000 7f6c8d400700 1 -- 172.21.15.31:0/3282615279 shutdown_connections 2024-04-25T01:14:56.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.456+0000 7f6c8d400700 1 -- 172.21.15.31:0/3282615279 wait complete. 2024-04-25T01:14:56.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.457+0000 7f6c8d400700 1 Processor -- start 2024-04-25T01:14:56.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.457+0000 7f6c8d400700 1 -- start start 2024-04-25T01:14:56.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.458+0000 7f6c8d400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c881023c0 0x7f6c8819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:56.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.458+0000 7f6c8d400700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c88102d70 0x7f6c8819b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:56.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.458+0000 7f6c8d400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c881074a0 0x7f6c8819f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:56.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.458+0000 7f6c8d400700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6c881124d0 con 0x7f6c881023c0 2024-04-25T01:14:56.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.458+0000 7f6c8d400700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6c88112350 con 0x7f6c88102d70 2024-04-25T01:14:56.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.458+0000 7f6c8d400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6c88112650 con 0x7f6c881074a0 2024-04-25T01:14:56.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.458+0000 7f6c87e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c881023c0 0x7f6c8819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:56.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.458+0000 7f6c87e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c881023c0 0x7f6c8819abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41442/0 (socket says 172.21.15.31:41442) 2024-04-25T01:14:56.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.458+0000 7f6c87400700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c88102d70 0x7f6c8819b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:56.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.458+0000 7f6c87e00700 1 -- 172.21.15.31:0/3044421441 learned_addr learned my addr 172.21.15.31:0/3044421441 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:14:56.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.458+0000 7f6c8ca00700 1 --2- 172.21.15.31:0/3044421441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c881074a0 0x7f6c8819f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:56.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.458+0000 7f6c87400700 1 -- 172.21.15.31:0/3044421441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c881074a0 msgr2=0x7f6c8819f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:56.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.458+0000 7f6c87400700 1 --2- 172.21.15.31:0/3044421441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c881074a0 0x7f6c8819f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:56.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.459+0000 7f6c87400700 1 -- 172.21.15.31:0/3044421441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c881023c0 msgr2=0x7f6c8819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:56.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.459+0000 7f6c87400700 1 --2- 172.21.15.31:0/3044421441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c881023c0 0x7f6c8819abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:56.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.459+0000 7f6c87400700 1 -- 172.21.15.31:0/3044421441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8819fbc0 con 0x7f6c88102d70 2024-04-25T01:14:56.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.459+0000 7f6c8ca00700 1 --2- 172.21.15.31:0/3044421441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c881074a0 0x7f6c8819f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:14:56.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.459+0000 7f6c87e00700 1 --2- 172.21.15.31:0/3044421441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c881023c0 0x7f6c8819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:14:56.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.459+0000 7f6c87400700 1 --2- 172.21.15.31:0/3044421441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c88102d70 0x7f6c8819b100 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6c7400e9c0 tx=0x7f6c7400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:56.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.459+0000 7f6c84c00700 1 -- 172.21.15.31:0/3044421441 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7400c9c0 con 0x7f6c88102d70 2024-04-25T01:14:56.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.459+0000 7f6c8d400700 1 -- 172.21.15.31:0/3044421441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c8819feb0 con 0x7f6c88102d70 2024-04-25T01:14:56.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.459+0000 7f6c84c00700 1 -- 172.21.15.31:0/3044421441 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c7400cb60 con 0x7f6c88102d70 2024-04-25T01:14:56.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.459+0000 7f6c84c00700 1 -- 172.21.15.31:0/3044421441 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c74018680 con 0x7f6c88102d70 2024-04-25T01:14:56.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.459+0000 7f6c8d400700 1 -- 172.21.15.31:0/3044421441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c881a7770 con 0x7f6c88102d70 2024-04-25T01:14:56.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.461+0000 7f6c8d400700 1 -- 172.21.15.31:0/3044421441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c88068f20 con 0x7f6c88102d70 2024-04-25T01:14:56.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.464+0000 7f6c84c00700 1 -- 172.21.15.31:0/3044421441 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6c74010430 con 0x7f6c88102d70 2024-04-25T01:14:56.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.465+0000 7f6c84c00700 1 --2- 172.21.15.31:0/3044421441 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6c6c0a2cb0 0x7f6c6c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:14:56.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.465+0000 7f6c84c00700 1 -- 172.21.15.31:0/3044421441 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f6c740cf2c0 con 0x7f6c88102d70 2024-04-25T01:14:56.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.465+0000 7f6c87e00700 1 --2- 172.21.15.31:0/3044421441 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6c6c0a2cb0 0x7f6c6c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:14:56.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.466+0000 7f6c87e00700 1 --2- 172.21.15.31:0/3044421441 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6c6c0a2cb0 0x7f6c6c0a5170 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6c78000c00 tx=0x7f6c78005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:14:56.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.469+0000 7f6c84c00700 1 -- 172.21.15.31:0/3044421441 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6c7409c0b0 con 0x7f6c88102d70 2024-04-25T01:14:56.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.708+0000 7f6c8d400700 1 -- 172.21.15.31:0/3044421441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f6c881a7940 con 0x7f6c88102d70 2024-04-25T01:14:56.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.709+0000 7f6c84c00700 1 -- 172.21.15.31:0/3044421441 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6c7409b9b0 con 0x7f6c88102d70 2024-04-25T01:14:56.915 INFO:teuthology.orchestra.run.smithi031.stdout:171798691904 2024-04-25T01:14:56.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.712+0000 7f6c8d400700 1 -- 172.21.15.31:0/3044421441 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6c6c0a2cb0 msgr2=0x7f6c6c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:56.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.712+0000 7f6c8d400700 1 --2- 172.21.15.31:0/3044421441 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6c6c0a2cb0 0x7f6c6c0a5170 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6c78000c00 tx=0x7f6c78005fb0 comp rx=0 tx=0).stop 2024-04-25T01:14:56.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.712+0000 7f6c8d400700 1 -- 172.21.15.31:0/3044421441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c88102d70 msgr2=0x7f6c8819b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:14:56.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.712+0000 7f6c8d400700 1 --2- 172.21.15.31:0/3044421441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c88102d70 0x7f6c8819b100 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6c7400e9c0 tx=0x7f6c7400ecd0 comp rx=0 tx=0).stop 2024-04-25T01:14:56.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.712+0000 7f6c8d400700 1 -- 172.21.15.31:0/3044421441 shutdown_connections 2024-04-25T01:14:56.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.712+0000 7f6c8d400700 1 --2- 172.21.15.31:0/3044421441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c881023c0 0x7f6c8819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:56.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.712+0000 7f6c8d400700 1 --2- 172.21.15.31:0/3044421441 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6c6c0a2cb0 0x7f6c6c0a5170 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:56.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.712+0000 7f6c8d400700 1 --2- 172.21.15.31:0/3044421441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c881074a0 0x7f6c8819f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:56.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.712+0000 7f6c8d400700 1 --2- 172.21.15.31:0/3044421441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c88102d70 0x7f6c8819b100 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:14:56.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.712+0000 7f6c8d400700 1 -- 172.21.15.31:0/3044421441 >> 172.21.15.31:0/3044421441 conn(0x7f6c880fdcb0 msgr2=0x7f6c880fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:14:56.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.713+0000 7f6c8d400700 1 -- 172.21.15.31:0/3044421441 shutdown_connections 2024-04-25T01:14:56.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:14:56.713+0000 7f6c8d400700 1 -- 172.21.15.31:0/3044421441 wait complete. 2024-04-25T01:14:57.144 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:56 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3044421441' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T01:14:57.144 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:56 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/945964919' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T01:14:57.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:56 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3044421441' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T01:14:57.195 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:56 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/945964919' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T01:14:57.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:56 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3044421441' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T01:14:57.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:56 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/945964919' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T01:14:57.966 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691898 got 171798691904 for osd.5 2024-04-25T01:14:57.966 DEBUG:teuthology.parallel:result is None 2024-04-25T01:14:58.154 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:57 smithi031 ceph-mon[28053]: pgmap v361: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T01:14:58.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:57 smithi169 ceph-mon[32759]: pgmap v361: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T01:14:58.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:57 smithi040 ceph-mon[32518]: pgmap v361: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T01:14:58.579 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266736 got 227633266741 for osd.7 2024-04-25T01:14:58.579 DEBUG:teuthology.parallel:result is None 2024-04-25T01:14:58.580 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T01:14:58.580 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T01:14:59.036 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:15:00.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:59 smithi169 ceph-mon[32759]: pgmap v362: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:00.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:59 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:15:00.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:14:59 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:15:00.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:59 smithi040 ceph-mon[32518]: pgmap v362: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:00.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:59 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:15:00.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:14:59 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:15:00.220 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:59 smithi031 ceph-mon[28053]: pgmap v362: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:00.220 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:59 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:15:00.220 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:14:59 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:15:01.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.292+0000 7f263b600700 1 -- 172.21.15.31:0/2624124623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26341082f0 msgr2=0x7f26341086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:01.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.292+0000 7f263b600700 1 --2- 172.21.15.31:0/2624124623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26341082f0 0x7f26341086d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2618009990 tx=0x7f2618009ca0 comp rx=0 tx=0).stop 2024-04-25T01:15:01.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.293+0000 7f263b600700 1 -- 172.21.15.31:0/2624124623 shutdown_connections 2024-04-25T01:15:01.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.293+0000 7f263b600700 1 --2- 172.21.15.31:0/2624124623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2634102ca0 0x7f263410f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.293+0000 7f263b600700 1 --2- 172.21.15.31:0/2624124623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26341082f0 0x7f26341086d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.293+0000 7f263b600700 1 --2- 172.21.15.31:0/2624124623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26341022e0 0x7f2634102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.293+0000 7f263b600700 1 -- 172.21.15.31:0/2624124623 >> 172.21.15.31:0/2624124623 conn(0x7f2634079390 msgr2=0x7f26340797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.293+0000 7f263b600700 1 -- 172.21.15.31:0/2624124623 shutdown_connections 2024-04-25T01:15:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.293+0000 7f263b600700 1 -- 172.21.15.31:0/2624124623 wait complete. 2024-04-25T01:15:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.294+0000 7f263b600700 1 Processor -- start 2024-04-25T01:15:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.295+0000 7f263b600700 1 -- start start 2024-04-25T01:15:01.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.295+0000 7f263b600700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26341022e0 0x7f26341a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:01.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.295+0000 7f263b600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2634102ca0 0x7f26341a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:01.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.295+0000 7f263b600700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26341082f0 0x7f263419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:01.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.295+0000 7f263b600700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f26341125c0 con 0x7f26341082f0 2024-04-25T01:15:01.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.295+0000 7f263b600700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2634112440 con 0x7f26341022e0 2024-04-25T01:15:01.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.295+0000 7f263b600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2634112740 con 0x7f2634102ca0 2024-04-25T01:15:01.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.295+0000 7f262be00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2634102ca0 0x7f26341a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:01.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.295+0000 7f262be00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2634102ca0 0x7f26341a0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:52508/0 (socket says 172.21.15.31:52508) 2024-04-25T01:15:01.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.295+0000 7f262be00700 1 -- 172.21.15.31:0/1488823956 learned_addr learned my addr 172.21.15.31:0/1488823956 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:01.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.295+0000 7f2639600700 1 --2- 172.21.15.31:0/1488823956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26341082f0 0x7f263419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:01.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.296+0000 7f2638c00700 1 --2- 172.21.15.31:0/1488823956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26341022e0 0x7f26341a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:01.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.296+0000 7f262be00700 1 -- 172.21.15.31:0/1488823956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26341022e0 msgr2=0x7f26341a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:01.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.296+0000 7f262be00700 1 --2- 172.21.15.31:0/1488823956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26341022e0 0x7f26341a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:01.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.296+0000 7f262be00700 1 -- 172.21.15.31:0/1488823956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26341082f0 msgr2=0x7f263419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:01.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.296+0000 7f262be00700 1 --2- 172.21.15.31:0/1488823956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26341082f0 0x7f263419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:01.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.296+0000 7f262be00700 1 -- 172.21.15.31:0/1488823956 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263419b160 con 0x7f2634102ca0 2024-04-25T01:15:01.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.296+0000 7f262be00700 1 --2- 172.21.15.31:0/1488823956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2634102ca0 0x7f26341a0d80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f262000e9a0 tx=0x7f262000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:01.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.297+0000 7f2632000700 1 -- 172.21.15.31:0/1488823956 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262000ca10 con 0x7f2634102ca0 2024-04-25T01:15:01.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.297+0000 7f2632000700 1 -- 172.21.15.31:0/1488823956 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f262000cbb0 con 0x7f2634102ca0 2024-04-25T01:15:01.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.297+0000 7f2632000700 1 -- 172.21.15.31:0/1488823956 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620018680 con 0x7f2634102ca0 2024-04-25T01:15:01.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.297+0000 7f263b600700 1 -- 172.21.15.31:0/1488823956 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f263419b450 con 0x7f2634102ca0 2024-04-25T01:15:01.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.297+0000 7f263b600700 1 -- 172.21.15.31:0/1488823956 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26341a7710 con 0x7f2634102ca0 2024-04-25T01:15:01.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.298+0000 7f263b600700 1 -- 172.21.15.31:0/1488823956 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2634068f20 con 0x7f2634102ca0 2024-04-25T01:15:01.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.300+0000 7f2632000700 1 -- 172.21.15.31:0/1488823956 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2620010430 con 0x7f2634102ca0 2024-04-25T01:15:01.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.301+0000 7f2632000700 1 --2- 172.21.15.31:0/1488823956 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f26100a2d30 0x7f26100a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:01.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.302+0000 7f2632000700 1 -- 172.21.15.31:0/1488823956 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f26200cf6d0 con 0x7f2634102ca0 2024-04-25T01:15:01.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.302+0000 7f2638c00700 1 --2- 172.21.15.31:0/1488823956 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f26100a2d30 0x7f26100a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:01.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.302+0000 7f2638c00700 1 --2- 172.21.15.31:0/1488823956 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f26100a2d30 0x7f26100a51f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f263419bb40 tx=0x7f261801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:01.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.304+0000 7f2632000700 1 -- 172.21.15.31:0/1488823956 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f262009c4c0 con 0x7f2634102ca0 2024-04-25T01:15:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.530+0000 7f263b600700 1 -- 172.21.15.31:0/1488823956 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f263410eb60 con 0x7f26100a2d30 2024-04-25T01:15:01.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.543+0000 7f2632000700 1 -- 172.21.15.31:0/1488823956 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258715 (secure 0 0 0) 0x7f26180672b0 con 0x7f26100a2d30 2024-04-25T01:15:01.530 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:15:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.548+0000 7f263b600700 1 -- 172.21.15.31:0/1488823956 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f26100a2d30 msgr2=0x7f26100a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.548+0000 7f263b600700 1 --2- 172.21.15.31:0/1488823956 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f26100a2d30 0x7f26100a51f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f263419bb40 tx=0x7f261801a040 comp rx=0 tx=0).stop 2024-04-25T01:15:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.548+0000 7f263b600700 1 -- 172.21.15.31:0/1488823956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2634102ca0 msgr2=0x7f26341a0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.548+0000 7f263b600700 1 --2- 172.21.15.31:0/1488823956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2634102ca0 0x7f26341a0d80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f262000e9a0 tx=0x7f262000ecb0 comp rx=0 tx=0).stop 2024-04-25T01:15:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.549+0000 7f263b600700 1 -- 172.21.15.31:0/1488823956 shutdown_connections 2024-04-25T01:15:01.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.549+0000 7f263b600700 1 --2- 172.21.15.31:0/1488823956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26341082f0 0x7f263419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:01.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.549+0000 7f263b600700 1 --2- 172.21.15.31:0/1488823956 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f26100a2d30 0x7f26100a51f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:01.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.549+0000 7f263b600700 1 --2- 172.21.15.31:0/1488823956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26341022e0 0x7f26341a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:01.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.549+0000 7f263b600700 1 --2- 172.21.15.31:0/1488823956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2634102ca0 0x7f26341a0d80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:01.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.549+0000 7f263b600700 1 -- 172.21.15.31:0/1488823956 >> 172.21.15.31:0/1488823956 conn(0x7f2634079390 msgr2=0x7f263410d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:01.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.549+0000 7f263b600700 1 -- 172.21.15.31:0/1488823956 shutdown_connections 2024-04-25T01:15:01.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:01.549+0000 7f263b600700 1 -- 172.21.15.31:0/1488823956 wait complete. 2024-04-25T01:15:01.537 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-04-25T01:15:01.922 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:01 smithi031 ceph-mon[28053]: pgmap v363: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:01.922 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:01 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:15:01.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:01 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:15:01.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:01 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:15:02.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:01 smithi169 ceph-mon[32759]: pgmap v363: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:02.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:01 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:15:02.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:01 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:15:02.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:01 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:15:02.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:01 smithi040 ceph-mon[32518]: pgmap v363: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:02.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:01 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:15:02.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:01 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:15:02.198 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:01 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:15:02.459 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-04-25T01:15:00.106321+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653483,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":648,"num_read_kb":435,"num_write":341,"num_write_kb":3046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375068,"kb_used_data":13356,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566756,"statfs":{"total":1151940427776,"available":1151556358144,"internally_reserved":0,"allocated":13676544,"data_stored":7506105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005680"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.800410+0000","last_change":"2024-04-25T01:13:31.536782+0000","last_active":"2024-04-25T01:14:33.800410+0000","last_peered":"2024-04-25T01:14:33.800410+0000","last_clean":"2024-04-25T01:14:33.800410+0000","last_became_active":"2024-04-25T01:13:31.536426+0000","last_became_peered":"2024-04-25T01:13:31.536426+0000","last_unstale":"2024-04-25T01:14:33.800410+0000","last_undegraded":"2024-04-25T01:14:33.800410+0000","last_fullsized":"2024-04-25T01:14:33.800410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:07.489179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.384567+0000","last_change":"2024-04-25T01:13:31.543521+0000","last_active":"2024-04-25T01:14:34.384567+0000","last_peered":"2024-04-25T01:14:34.384567+0000","last_clean":"2024-04-25T01:14:34.384567+0000","last_became_active":"2024-04-25T01:13:31.542767+0000","last_became_peered":"2024-04-25T01:13:31.542767+0000","last_unstale":"2024-04-25T01:14:34.384567+0000","last_undegraded":"2024-04-25T01:14:34.384567+0000","last_fullsized":"2024-04-25T01:14:34.384567+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:33.287564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.383601+0000","last_change":"2024-04-25T01:13:31.537297+0000","last_active":"2024-04-25T01:14:34.383601+0000","last_peered":"2024-04-25T01:14:34.383601+0000","last_clean":"2024-04-25T01:14:34.383601+0000","last_became_active":"2024-04-25T01:13:31.536930+0000","last_became_peered":"2024-04-25T01:13:31.536930+0000","last_unstale":"2024-04-25T01:14:34.383601+0000","last_undegraded":"2024-04-25T01:14:34.383601+0000","last_fullsized":"2024-04-25T01:14:34.383601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:58:35.465287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.380366+0000","last_change":"2024-04-25T01:13:31.516514+0000","last_active":"2024-04-25T01:14:34.380366+0000","last_peered":"2024-04-25T01:14:34.380366+0000","last_clean":"2024-04-25T01:14:34.380366+0000","last_became_active":"2024-04-25T01:13:31.516192+0000","last_became_peered":"2024-04-25T01:13:31.516192+0000","last_unstale":"2024-04-25T01:14:34.380366+0000","last_undegraded":"2024-04-25T01:14:34.380366+0000","last_fullsized":"2024-04-25T01:14:34.380366+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:27.416626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802262+0000","last_change":"2024-04-25T01:13:31.538662+0000","last_active":"2024-04-25T01:14:33.802262+0000","last_peered":"2024-04-25T01:14:33.802262+0000","last_clean":"2024-04-25T01:14:33.802262+0000","last_became_active":"2024-04-25T01:13:31.538362+0000","last_became_peered":"2024-04-25T01:13:31.538362+0000","last_unstale":"2024-04-25T01:14:33.802262+0000","last_undegraded":"2024-04-25T01:14:33.802262+0000","last_fullsized":"2024-04-25T01:14:33.802262+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:10.744891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.811985+0000","last_change":"2024-04-25T01:14:33.811985+0000","last_active":"2024-04-25T01:14:33.811985+0000","last_peered":"2024-04-25T01:14:33.811985+0000","last_clean":"2024-04-25T01:14:33.811985+0000","last_became_active":"2024-04-25T01:14:33.811680+0000","last_became_peered":"2024-04-25T01:14:33.811680+0000","last_unstale":"2024-04-25T01:14:33.811985+0000","last_undegraded":"2024-04-25T01:14:33.811985+0000","last_fullsized":"2024-04-25T01:14:33.811985+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:25.004679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802564+0000","last_change":"2024-04-25T01:13:31.530362+0000","last_active":"2024-04-25T01:14:33.802564+0000","last_peered":"2024-04-25T01:14:33.802564+0000","last_clean":"2024-04-25T01:14:33.802564+0000","last_became_active":"2024-04-25T01:13:31.529869+0000","last_became_peered":"2024-04-25T01:13:31.529869+0000","last_unstale":"2024-04-25T01:14:33.802564+0000","last_undegraded":"2024-04-25T01:14:33.802564+0000","last_fullsized":"2024-04-25T01:14:33.802564+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:54.499230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.383073+0000","last_change":"2024-04-25T01:14:34.383073+0000","last_active":"2024-04-25T01:14:34.383073+0000","last_peered":"2024-04-25T01:14:34.383073+0000","last_clean":"2024-04-25T01:14:34.383073+0000","last_became_active":"2024-04-25T01:14:34.381510+0000","last_became_peered":"2024-04-25T01:14:34.381510+0000","last_unstale":"2024-04-25T01:14:34.383073+0000","last_undegraded":"2024-04-25T01:14:34.383073+0000","last_fullsized":"2024-04-25T01:14:34.383073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:10.130178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.191393+0000","last_change":"2024-04-25T01:13:31.518589+0000","last_active":"2024-04-25T01:14:34.191393+0000","last_peered":"2024-04-25T01:14:34.191393+0000","last_clean":"2024-04-25T01:14:34.191393+0000","last_became_active":"2024-04-25T01:13:31.518183+0000","last_became_peered":"2024-04-25T01:13:31.518183+0000","last_unstale":"2024-04-25T01:14:34.191393+0000","last_undegraded":"2024-04-25T01:14:34.191393+0000","last_fullsized":"2024-04-25T01:14:34.191393+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:06:17.534019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.805680+0000","last_change":"2024-04-25T01:13:31.522281+0000","last_active":"2024-04-25T01:14:33.805680+0000","last_peered":"2024-04-25T01:14:33.805680+0000","last_clean":"2024-04-25T01:14:33.805680+0000","last_became_active":"2024-04-25T01:13:31.521906+0000","last_became_peered":"2024-04-25T01:13:31.521906+0000","last_unstale":"2024-04-25T01:14:33.805680+0000","last_undegraded":"2024-04-25T01:14:33.805680+0000","last_fullsized":"2024-04-25T01:14:33.805680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:58.761284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802897+0000","last_change":"2024-04-25T01:13:31.514766+0000","last_active":"2024-04-25T01:14:33.802897+0000","last_peered":"2024-04-25T01:14:33.802897+0000","last_clean":"2024-04-25T01:14:33.802897+0000","last_became_active":"2024-04-25T01:13:31.514391+0000","last_became_peered":"2024-04-25T01:13:31.514391+0000","last_unstale":"2024-04-25T01:14:33.802897+0000","last_undegraded":"2024-04-25T01:14:33.802897+0000","last_fullsized":"2024-04-25T01:14:33.802897+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:07.398446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.380208+0000","last_change":"2024-04-25T01:13:31.518976+0000","last_active":"2024-04-25T01:14:34.380208+0000","last_peered":"2024-04-25T01:14:34.380208+0000","last_clean":"2024-04-25T01:14:34.380208+0000","last_became_active":"2024-04-25T01:13:31.518566+0000","last_became_peered":"2024-04-25T01:13:31.518566+0000","last_unstale":"2024-04-25T01:14:34.380208+0000","last_undegraded":"2024-04-25T01:14:34.380208+0000","last_fullsized":"2024-04-25T01:14:34.380208+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:59.477996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.814514+0000","last_change":"2024-04-25T01:13:31.532880+0000","last_active":"2024-04-25T01:14:33.814514+0000","last_peered":"2024-04-25T01:14:33.814514+0000","last_clean":"2024-04-25T01:14:33.814514+0000","last_became_active":"2024-04-25T01:13:31.531470+0000","last_became_peered":"2024-04-25T01:13:31.531470+0000","last_unstale":"2024-04-25T01:14:33.814514+0000","last_undegraded":"2024-04-25T01:14:33.814514+0000","last_fullsized":"2024-04-25T01:14:33.814514+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:55.431431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.383926+0000","last_change":"2024-04-25T01:13:31.532472+0000","last_active":"2024-04-25T01:14:34.383926+0000","last_peered":"2024-04-25T01:14:34.383926+0000","last_clean":"2024-04-25T01:14:34.383926+0000","last_became_active":"2024-04-25T01:13:31.530850+0000","last_became_peered":"2024-04-25T01:13:31.530850+0000","last_unstale":"2024-04-25T01:14:34.383926+0000","last_undegraded":"2024-04-25T01:14:34.383926+0000","last_fullsized":"2024-04-25T01:14:34.383926+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:50:26.890858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.382279+0000","last_change":"2024-04-25T01:13:31.539930+0000","last_active":"2024-04-25T01:14:34.382279+0000","last_peered":"2024-04-25T01:14:34.382279+0000","last_clean":"2024-04-25T01:14:34.382279+0000","last_became_active":"2024-04-25T01:13:31.539523+0000","last_became_peered":"2024-04-25T01:13:31.539523+0000","last_unstale":"2024-04-25T01:14:34.382279+0000","last_undegraded":"2024-04-25T01:14:34.382279+0000","last_fullsized":"2024-04-25T01:14:34.382279+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:07.983893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.190895+0000","last_change":"2024-04-25T01:13:31.536437+0000","last_active":"2024-04-25T01:14:34.190895+0000","last_peered":"2024-04-25T01:14:34.190895+0000","last_clean":"2024-04-25T01:14:34.190895+0000","last_became_active":"2024-04-25T01:13:31.536066+0000","last_became_peered":"2024-04-25T01:13:31.536066+0000","last_unstale":"2024-04-25T01:14:34.190895+0000","last_undegraded":"2024-04-25T01:14:34.190895+0000","last_fullsized":"2024-04-25T01:14:34.190895+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:50.986971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.800646+0000","last_change":"2024-04-25T01:13:31.536706+0000","last_active":"2024-04-25T01:14:33.800646+0000","last_peered":"2024-04-25T01:14:33.800646+0000","last_clean":"2024-04-25T01:14:33.800646+0000","last_became_active":"2024-04-25T01:13:31.536318+0000","last_became_peered":"2024-04-25T01:13:31.536318+0000","last_unstale":"2024-04-25T01:14:33.800646+0000","last_undegraded":"2024-04-25T01:14:33.800646+0000","last_fullsized":"2024-04-25T01:14:33.800646+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:35.526515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.805837+0000","last_change":"2024-04-25T01:13:31.524462+0000","last_active":"2024-04-25T01:14:33.805837+0000","last_peered":"2024-04-25T01:14:33.805837+0000","last_clean":"2024-04-25T01:14:33.805837+0000","last_became_active":"2024-04-25T01:13:31.524018+0000","last_became_peered":"2024-04-25T01:13:31.524018+0000","last_unstale":"2024-04-25T01:14:33.805837+0000","last_undegraded":"2024-04-25T01:14:33.805837+0000","last_fullsized":"2024-04-25T01:14:33.805837+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:43.767053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.798963+0000","last_change":"2024-04-25T01:13:31.531553+0000","last_active":"2024-04-25T01:14:33.798963+0000","last_peered":"2024-04-25T01:14:33.798963+0000","last_clean":"2024-04-25T01:14:33.798963+0000","last_became_active":"2024-04-25T01:13:31.531237+0000","last_became_peered":"2024-04-25T01:13:31.531237+0000","last_unstale":"2024-04-25T01:14:33.798963+0000","last_undegraded":"2024-04-25T01:14:33.798963+0000","last_fullsized":"2024-04-25T01:14:33.798963+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:41.839370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.805877+0000","last_change":"2024-04-25T01:13:30.516847+0000","last_active":"2024-04-25T01:14:33.805877+0000","last_peered":"2024-04-25T01:14:33.805877+0000","last_clean":"2024-04-25T01:14:33.805877+0000","last_became_active":"2024-04-25T01:13:30.516500+0000","last_became_peered":"2024-04-25T01:13:30.516500+0000","last_unstale":"2024-04-25T01:14:33.805877+0000","last_undegraded":"2024-04-25T01:14:33.805877+0000","last_fullsized":"2024-04-25T01:14:33.805877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:37.703075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.801626+0000","last_change":"2024-04-25T01:13:31.518822+0000","last_active":"2024-04-25T01:14:33.801626+0000","last_peered":"2024-04-25T01:14:33.801626+0000","last_clean":"2024-04-25T01:14:33.801626+0000","last_became_active":"2024-04-25T01:13:31.516726+0000","last_became_peered":"2024-04-25T01:13:31.516726+0000","last_unstale":"2024-04-25T01:14:33.801626+0000","last_undegraded":"2024-04-25T01:14:33.801626+0000","last_fullsized":"2024-04-25T01:14:33.801626+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:02.360104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.801965+0000","last_change":"2024-04-25T01:13:30.531875+0000","last_active":"2024-04-25T01:14:33.801965+0000","last_peered":"2024-04-25T01:14:33.801965+0000","last_clean":"2024-04-25T01:14:33.801965+0000","last_became_active":"2024-04-25T01:13:30.531532+0000","last_became_peered":"2024-04-25T01:13:30.531532+0000","last_unstale":"2024-04-25T01:14:33.801965+0000","last_undegraded":"2024-04-25T01:14:33.801965+0000","last_fullsized":"2024-04-25T01:14:33.801965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:30.820946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.385143+0000","last_change":"2024-04-25T01:13:31.534211+0000","last_active":"2024-04-25T01:14:34.385143+0000","last_peered":"2024-04-25T01:14:34.385143+0000","last_clean":"2024-04-25T01:14:34.385143+0000","last_became_active":"2024-04-25T01:13:31.533828+0000","last_became_peered":"2024-04-25T01:13:31.533828+0000","last_unstale":"2024-04-25T01:14:34.385143+0000","last_undegraded":"2024-04-25T01:14:34.385143+0000","last_fullsized":"2024-04-25T01:14:34.385143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:39.488524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802646+0000","last_change":"2024-04-25T01:13:30.517922+0000","last_active":"2024-04-25T01:14:33.802646+0000","last_peered":"2024-04-25T01:14:33.802646+0000","last_clean":"2024-04-25T01:14:33.802646+0000","last_became_active":"2024-04-25T01:13:30.517571+0000","last_became_peered":"2024-04-25T01:13:30.517571+0000","last_unstale":"2024-04-25T01:14:33.802646+0000","last_undegraded":"2024-04-25T01:14:33.802646+0000","last_fullsized":"2024-04-25T01:14:33.802646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:16:34.901210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.191023+0000","last_change":"2024-04-25T01:13:31.530460+0000","last_active":"2024-04-25T01:14:34.191023+0000","last_peered":"2024-04-25T01:14:34.191023+0000","last_clean":"2024-04-25T01:14:34.191023+0000","last_became_active":"2024-04-25T01:13:31.530084+0000","last_became_peered":"2024-04-25T01:13:31.530084+0000","last_unstale":"2024-04-25T01:14:34.191023+0000","last_undegraded":"2024-04-25T01:14:34.191023+0000","last_fullsized":"2024-04-25T01:14:34.191023+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:36.810301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802481+0000","last_change":"2024-04-25T01:13:30.514320+0000","last_active":"2024-04-25T01:14:33.802481+0000","last_peered":"2024-04-25T01:14:33.802481+0000","last_clean":"2024-04-25T01:14:33.802481+0000","last_became_active":"2024-04-25T01:13:30.513752+0000","last_became_peered":"2024-04-25T01:13:30.513752+0000","last_unstale":"2024-04-25T01:14:33.802481+0000","last_undegraded":"2024-04-25T01:14:33.802481+0000","last_fullsized":"2024-04-25T01:14:33.802481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:19.814454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.379840+0000","last_change":"2024-04-25T01:13:31.514516+0000","last_active":"2024-04-25T01:14:34.379840+0000","last_peered":"2024-04-25T01:14:34.379840+0000","last_clean":"2024-04-25T01:14:34.379840+0000","last_became_active":"2024-04-25T01:13:31.514208+0000","last_became_peered":"2024-04-25T01:13:31.514208+0000","last_unstale":"2024-04-25T01:14:34.379840+0000","last_undegraded":"2024-04-25T01:14:34.379840+0000","last_fullsized":"2024-04-25T01:14:34.379840+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:10:11.204770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":110,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:57.654082+0000","last_change":"2024-04-25T01:13:30.519701+0000","last_active":"2024-04-25T01:14:57.654082+0000","last_peered":"2024-04-25T01:14:57.654082+0000","last_clean":"2024-04-25T01:14:57.654082+0000","last_became_active":"2024-04-25T01:13:30.519315+0000","last_became_peered":"2024-04-25T01:13:30.519315+0000","last_unstale":"2024-04-25T01:14:57.654082+0000","last_undegraded":"2024-04-25T01:14:57.654082+0000","last_fullsized":"2024-04-25T01:14:57.654082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:34.613473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802442+0000","last_change":"2024-04-25T01:13:31.528376+0000","last_active":"2024-04-25T01:14:33.802442+0000","last_peered":"2024-04-25T01:14:33.802442+0000","last_clean":"2024-04-25T01:14:33.802442+0000","last_became_active":"2024-04-25T01:13:31.527622+0000","last_became_peered":"2024-04-25T01:13:31.527622+0000","last_unstale":"2024-04-25T01:14:33.802442+0000","last_undegraded":"2024-04-25T01:14:33.802442+0000","last_fullsized":"2024-04-25T01:14:33.802442+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:17.901898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.805174+0000","last_change":"2024-04-25T01:13:30.514798+0000","last_active":"2024-04-25T01:14:33.805174+0000","last_peered":"2024-04-25T01:14:33.805174+0000","last_clean":"2024-04-25T01:14:33.805174+0000","last_became_active":"2024-04-25T01:13:30.514422+0000","last_became_peered":"2024-04-25T01:13:30.514422+0000","last_unstale":"2024-04-25T01:14:33.805174+0000","last_undegraded":"2024-04-25T01:14:33.805174+0000","last_fullsized":"2024-04-25T01:14:33.805174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:27.584578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.379763+0000","last_change":"2024-04-25T01:13:31.510479+0000","last_active":"2024-04-25T01:14:34.379763+0000","last_peered":"2024-04-25T01:14:34.379763+0000","last_clean":"2024-04-25T01:14:34.379763+0000","last_became_active":"2024-04-25T01:13:31.509166+0000","last_became_peered":"2024-04-25T01:13:31.509166+0000","last_unstale":"2024-04-25T01:14:34.379763+0000","last_undegraded":"2024-04-25T01:14:34.379763+0000","last_fullsized":"2024-04-25T01:14:34.379763+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:42.604266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.809618+0000","last_change":"2024-04-25T01:13:30.520435+0000","last_active":"2024-04-25T01:14:33.809618+0000","last_peered":"2024-04-25T01:14:33.809618+0000","last_clean":"2024-04-25T01:14:33.809618+0000","last_became_active":"2024-04-25T01:13:30.520042+0000","last_became_peered":"2024-04-25T01:13:30.520042+0000","last_unstale":"2024-04-25T01:14:33.809618+0000","last_undegraded":"2024-04-25T01:14:33.809618+0000","last_fullsized":"2024-04-25T01:14:33.809618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:37.818149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.800866+0000","last_change":"2024-04-25T01:13:31.524799+0000","last_active":"2024-04-25T01:14:33.800866+0000","last_peered":"2024-04-25T01:14:33.800866+0000","last_clean":"2024-04-25T01:14:33.800866+0000","last_became_active":"2024-04-25T01:13:31.524454+0000","last_became_peered":"2024-04-25T01:13:31.524454+0000","last_unstale":"2024-04-25T01:14:33.800866+0000","last_undegraded":"2024-04-25T01:14:33.800866+0000","last_fullsized":"2024-04-25T01:14:33.800866+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:15.930922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.379905+0000","last_change":"2024-04-25T01:13:30.514104+0000","last_active":"2024-04-25T01:14:34.379905+0000","last_peered":"2024-04-25T01:14:34.379905+0000","last_clean":"2024-04-25T01:14:34.379905+0000","last_became_active":"2024-04-25T01:13:30.513742+0000","last_became_peered":"2024-04-25T01:13:30.513742+0000","last_unstale":"2024-04-25T01:14:34.379905+0000","last_undegraded":"2024-04-25T01:14:34.379905+0000","last_fullsized":"2024-04-25T01:14:34.379905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:42.189997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.805403+0000","last_change":"2024-04-25T01:13:31.528223+0000","last_active":"2024-04-25T01:14:33.805403+0000","last_peered":"2024-04-25T01:14:33.805403+0000","last_clean":"2024-04-25T01:14:33.805403+0000","last_became_active":"2024-04-25T01:13:31.526906+0000","last_became_peered":"2024-04-25T01:13:31.526906+0000","last_unstale":"2024-04-25T01:14:33.805403+0000","last_undegraded":"2024-04-25T01:14:33.805403+0000","last_fullsized":"2024-04-25T01:14:33.805403+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:12:16.276974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":106,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:57.653341+0000","last_change":"2024-04-25T01:13:30.519174+0000","last_active":"2024-04-25T01:14:57.653341+0000","last_peered":"2024-04-25T01:14:57.653341+0000","last_clean":"2024-04-25T01:14:57.653341+0000","last_became_active":"2024-04-25T01:13:30.518792+0000","last_became_peered":"2024-04-25T01:13:30.518792+0000","last_unstale":"2024-04-25T01:14:57.653341+0000","last_undegraded":"2024-04-25T01:14:57.653341+0000","last_fullsized":"2024-04-25T01:14:57.653341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:27.746435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.191331+0000","last_change":"2024-04-25T01:13:31.521482+0000","last_active":"2024-04-25T01:14:34.191331+0000","last_peered":"2024-04-25T01:14:34.191331+0000","last_clean":"2024-04-25T01:14:34.191331+0000","last_became_active":"2024-04-25T01:13:31.521114+0000","last_became_peered":"2024-04-25T01:13:31.521114+0000","last_unstale":"2024-04-25T01:14:34.191331+0000","last_undegraded":"2024-04-25T01:14:34.191331+0000","last_fullsized":"2024-04-25T01:14:34.191331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:48.748691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802983+0000","last_change":"2024-04-25T01:13:30.518427+0000","last_active":"2024-04-25T01:14:33.802983+0000","last_peered":"2024-04-25T01:14:33.802983+0000","last_clean":"2024-04-25T01:14:33.802983+0000","last_became_active":"2024-04-25T01:13:30.518005+0000","last_became_peered":"2024-04-25T01:13:30.518005+0000","last_unstale":"2024-04-25T01:14:33.802983+0000","last_undegraded":"2024-04-25T01:14:33.802983+0000","last_fullsized":"2024-04-25T01:14:33.802983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:29:21.087411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.384870+0000","last_change":"2024-04-25T01:13:31.532598+0000","last_active":"2024-04-25T01:14:34.384870+0000","last_peered":"2024-04-25T01:14:34.384870+0000","last_clean":"2024-04-25T01:14:34.384870+0000","last_became_active":"2024-04-25T01:13:31.530468+0000","last_became_peered":"2024-04-25T01:13:31.530468+0000","last_unstale":"2024-04-25T01:14:34.384870+0000","last_undegraded":"2024-04-25T01:14:34.384870+0000","last_fullsized":"2024-04-25T01:14:34.384870+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:28.254924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.379914+0000","last_change":"2024-04-25T01:13:30.520507+0000","last_active":"2024-04-25T01:14:34.379914+0000","last_peered":"2024-04-25T01:14:34.379914+0000","last_clean":"2024-04-25T01:14:34.379914+0000","last_became_active":"2024-04-25T01:13:30.520165+0000","last_became_peered":"2024-04-25T01:13:30.520165+0000","last_unstale":"2024-04-25T01:14:34.379914+0000","last_undegraded":"2024-04-25T01:14:34.379914+0000","last_fullsized":"2024-04-25T01:14:34.379914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:26.173090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.191217+0000","last_change":"2024-04-25T01:13:31.517021+0000","last_active":"2024-04-25T01:14:34.191217+0000","last_peered":"2024-04-25T01:14:34.191217+0000","last_clean":"2024-04-25T01:14:34.191217+0000","last_became_active":"2024-04-25T01:13:31.516676+0000","last_became_peered":"2024-04-25T01:13:31.516676+0000","last_unstale":"2024-04-25T01:14:34.191217+0000","last_undegraded":"2024-04-25T01:14:34.191217+0000","last_fullsized":"2024-04-25T01:14:34.191217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:25.314544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'28","reported_seq":223,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:58.654738+0000","last_change":"2024-04-25T01:13:30.511963+0000","last_active":"2024-04-25T01:14:58.654738+0000","last_peered":"2024-04-25T01:14:58.654738+0000","last_clean":"2024-04-25T01:14:58.654738+0000","last_became_active":"2024-04-25T01:13:30.511142+0000","last_became_peered":"2024-04-25T01:13:30.511142+0000","last_unstale":"2024-04-25T01:14:58.654738+0000","last_undegraded":"2024-04-25T01:14:58.654738+0000","last_fullsized":"2024-04-25T01:14:58.654738+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:53.545598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58880,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":176,"num_read_kb":145,"num_write":29,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802468+0000","last_change":"2024-04-25T01:13:31.516236+0000","last_active":"2024-04-25T01:14:33.802468+0000","last_peered":"2024-04-25T01:14:33.802468+0000","last_clean":"2024-04-25T01:14:33.802468+0000","last_became_active":"2024-04-25T01:13:31.515871+0000","last_became_peered":"2024-04-25T01:13:31.515871+0000","last_unstale":"2024-04-25T01:14:33.802468+0000","last_undegraded":"2024-04-25T01:14:33.802468+0000","last_fullsized":"2024-04-25T01:14:33.802468+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:24.464246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.798961+0000","last_change":"2024-04-25T01:13:30.513070+0000","last_active":"2024-04-25T01:14:33.798961+0000","last_peered":"2024-04-25T01:14:33.798961+0000","last_clean":"2024-04-25T01:14:33.798961+0000","last_became_active":"2024-04-25T01:13:30.512701+0000","last_became_peered":"2024-04-25T01:13:30.512701+0000","last_unstale":"2024-04-25T01:14:33.798961+0000","last_undegraded":"2024-04-25T01:14:33.798961+0000","last_fullsized":"2024-04-25T01:14:33.798961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:33.482272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.811735+0000","last_change":"2024-04-25T01:13:31.538313+0000","last_active":"2024-04-25T01:14:33.811735+0000","last_peered":"2024-04-25T01:14:33.811735+0000","last_clean":"2024-04-25T01:14:33.811735+0000","last_became_active":"2024-04-25T01:13:31.537907+0000","last_became_peered":"2024-04-25T01:13:31.537907+0000","last_unstale":"2024-04-25T01:14:33.811735+0000","last_undegraded":"2024-04-25T01:14:33.811735+0000","last_fullsized":"2024-04-25T01:14:33.811735+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:19.484311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.379592+0000","last_change":"2024-04-25T01:13:30.520170+0000","last_active":"2024-04-25T01:14:34.379592+0000","last_peered":"2024-04-25T01:14:34.379592+0000","last_clean":"2024-04-25T01:14:34.379592+0000","last_became_active":"2024-04-25T01:13:30.519636+0000","last_became_peered":"2024-04-25T01:13:30.519636+0000","last_unstale":"2024-04-25T01:14:34.379592+0000","last_undegraded":"2024-04-25T01:14:34.379592+0000","last_fullsized":"2024-04-25T01:14:34.379592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:00:32.670127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802918+0000","last_change":"2024-04-25T01:13:31.537630+0000","last_active":"2024-04-25T01:14:33.802918+0000","last_peered":"2024-04-25T01:14:33.802918+0000","last_clean":"2024-04-25T01:14:33.802918+0000","last_became_active":"2024-04-25T01:13:31.537164+0000","last_became_peered":"2024-04-25T01:13:31.537164+0000","last_unstale":"2024-04-25T01:14:33.802918+0000","last_undegraded":"2024-04-25T01:14:33.802918+0000","last_fullsized":"2024-04-25T01:14:33.802918+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:04.091194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":107,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:55.703804+0000","last_change":"2024-04-25T01:13:30.514322+0000","last_active":"2024-04-25T01:14:55.703804+0000","last_peered":"2024-04-25T01:14:55.703804+0000","last_clean":"2024-04-25T01:14:55.703804+0000","last_became_active":"2024-04-25T01:13:30.513990+0000","last_became_peered":"2024-04-25T01:13:30.513990+0000","last_unstale":"2024-04-25T01:14:55.703804+0000","last_undegraded":"2024-04-25T01:14:55.703804+0000","last_fullsized":"2024-04-25T01:14:55.703804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:59.995489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802516+0000","last_change":"2024-04-25T01:13:31.519730+0000","last_active":"2024-04-25T01:14:33.802516+0000","last_peered":"2024-04-25T01:14:33.802516+0000","last_clean":"2024-04-25T01:14:33.802516+0000","last_became_active":"2024-04-25T01:13:31.519363+0000","last_became_peered":"2024-04-25T01:13:31.519363+0000","last_unstale":"2024-04-25T01:14:33.802516+0000","last_undegraded":"2024-04-25T01:14:33.802516+0000","last_fullsized":"2024-04-25T01:14:33.802516+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:26.400754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802325+0000","last_change":"2024-04-25T01:13:30.517220+0000","last_active":"2024-04-25T01:14:33.802325+0000","last_peered":"2024-04-25T01:14:33.802325+0000","last_clean":"2024-04-25T01:14:33.802325+0000","last_became_active":"2024-04-25T01:13:30.516825+0000","last_became_peered":"2024-04-25T01:13:30.516825+0000","last_unstale":"2024-04-25T01:14:33.802325+0000","last_undegraded":"2024-04-25T01:14:33.802325+0000","last_fullsized":"2024-04-25T01:14:33.802325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:04.452894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.379547+0000","last_change":"2024-04-25T01:13:31.517424+0000","last_active":"2024-04-25T01:14:34.379547+0000","last_peered":"2024-04-25T01:14:34.379547+0000","last_clean":"2024-04-25T01:14:34.379547+0000","last_became_active":"2024-04-25T01:13:31.517017+0000","last_became_peered":"2024-04-25T01:13:31.517017+0000","last_unstale":"2024-04-25T01:14:34.379547+0000","last_undegraded":"2024-04-25T01:14:34.379547+0000","last_fullsized":"2024-04-25T01:14:34.379547+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:16.516450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802727+0000","last_change":"2024-04-25T01:13:30.515249+0000","last_active":"2024-04-25T01:14:33.802727+0000","last_peered":"2024-04-25T01:14:33.802727+0000","last_clean":"2024-04-25T01:14:33.802727+0000","last_became_active":"2024-04-25T01:13:30.514819+0000","last_became_peered":"2024-04-25T01:13:30.514819+0000","last_unstale":"2024-04-25T01:14:33.802727+0000","last_undegraded":"2024-04-25T01:14:33.802727+0000","last_fullsized":"2024-04-25T01:14:33.802727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:52.503564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.800361+0000","last_change":"2024-04-25T01:13:31.519143+0000","last_active":"2024-04-25T01:14:33.800361+0000","last_peered":"2024-04-25T01:14:33.800361+0000","last_clean":"2024-04-25T01:14:33.800361+0000","last_became_active":"2024-04-25T01:13:31.518755+0000","last_became_peered":"2024-04-25T01:13:31.518755+0000","last_unstale":"2024-04-25T01:14:33.800361+0000","last_undegraded":"2024-04-25T01:14:33.800361+0000","last_fullsized":"2024-04-25T01:14:33.800361+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:02.870725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.804624+0000","last_change":"2024-04-25T01:13:30.522489+0000","last_active":"2024-04-25T01:14:33.804624+0000","last_peered":"2024-04-25T01:14:33.804624+0000","last_clean":"2024-04-25T01:14:33.804624+0000","last_became_active":"2024-04-25T01:13:30.522128+0000","last_became_peered":"2024-04-25T01:13:30.522128+0000","last_unstale":"2024-04-25T01:14:33.804624+0000","last_undegraded":"2024-04-25T01:14:33.804624+0000","last_fullsized":"2024-04-25T01:14:33.804624+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:12.875627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.805510+0000","last_change":"2024-04-25T01:13:31.537536+0000","last_active":"2024-04-25T01:14:33.805510+0000","last_peered":"2024-04-25T01:14:33.805510+0000","last_clean":"2024-04-25T01:14:33.805510+0000","last_became_active":"2024-04-25T01:13:31.537169+0000","last_became_peered":"2024-04-25T01:13:31.537169+0000","last_unstale":"2024-04-25T01:14:33.805510+0000","last_undegraded":"2024-04-25T01:14:33.805510+0000","last_fullsized":"2024-04-25T01:14:33.805510+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:55.147719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.803001+0000","last_change":"2024-04-25T01:13:30.523870+0000","last_active":"2024-04-25T01:14:33.803001+0000","last_peered":"2024-04-25T01:14:33.803001+0000","last_clean":"2024-04-25T01:14:33.803001+0000","last_became_active":"2024-04-25T01:13:30.523504+0000","last_became_peered":"2024-04-25T01:13:30.523504+0000","last_unstale":"2024-04-25T01:14:33.803001+0000","last_undegraded":"2024-04-25T01:14:33.803001+0000","last_fullsized":"2024-04-25T01:14:33.803001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:42.498898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.805373+0000","last_change":"2024-04-25T01:13:31.539691+0000","last_active":"2024-04-25T01:14:33.805373+0000","last_peered":"2024-04-25T01:14:33.805373+0000","last_clean":"2024-04-25T01:14:33.805373+0000","last_became_active":"2024-04-25T01:13:31.539287+0000","last_became_peered":"2024-04-25T01:13:31.539287+0000","last_unstale":"2024-04-25T01:14:33.805373+0000","last_undegraded":"2024-04-25T01:14:33.805373+0000","last_fullsized":"2024-04-25T01:14:33.805373+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:47.985167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.380057+0000","last_change":"2024-04-25T01:13:31.513438+0000","last_active":"2024-04-25T01:14:34.380057+0000","last_peered":"2024-04-25T01:14:34.380057+0000","last_clean":"2024-04-25T01:14:34.380057+0000","last_became_active":"2024-04-25T01:13:31.513087+0000","last_became_peered":"2024-04-25T01:13:31.513087+0000","last_unstale":"2024-04-25T01:14:34.380057+0000","last_undegraded":"2024-04-25T01:14:34.380057+0000","last_fullsized":"2024-04-25T01:14:34.380057+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:12.999714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":103,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:54.700988+0000","last_change":"2024-04-25T01:13:30.527528+0000","last_active":"2024-04-25T01:14:54.700988+0000","last_peered":"2024-04-25T01:14:54.700988+0000","last_clean":"2024-04-25T01:14:54.700988+0000","last_became_active":"2024-04-25T01:13:30.526040+0000","last_became_peered":"2024-04-25T01:13:30.526040+0000","last_unstale":"2024-04-25T01:14:54.700988+0000","last_undegraded":"2024-04-25T01:14:54.700988+0000","last_fullsized":"2024-04-25T01:14:54.700988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:31.943727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.801323+0000","last_change":"2024-04-25T01:13:31.523311+0000","last_active":"2024-04-25T01:14:33.801323+0000","last_peered":"2024-04-25T01:14:33.801323+0000","last_clean":"2024-04-25T01:14:33.801323+0000","last_became_active":"2024-04-25T01:13:31.518099+0000","last_became_peered":"2024-04-25T01:13:31.518099+0000","last_unstale":"2024-04-25T01:14:33.801323+0000","last_undegraded":"2024-04-25T01:14:33.801323+0000","last_fullsized":"2024-04-25T01:14:33.801323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:02.947202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.380270+0000","last_change":"2024-04-25T01:13:30.512804+0000","last_active":"2024-04-25T01:14:34.380270+0000","last_peered":"2024-04-25T01:14:34.380270+0000","last_clean":"2024-04-25T01:14:34.380270+0000","last_became_active":"2024-04-25T01:13:30.512406+0000","last_became_peered":"2024-04-25T01:13:30.512406+0000","last_unstale":"2024-04-25T01:14:34.380270+0000","last_undegraded":"2024-04-25T01:14:34.380270+0000","last_fullsized":"2024-04-25T01:14:34.380270+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:00.042244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":345,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802699+0000","last_change":"2024-04-25T01:12:07.906198+0000","last_active":"2024-04-25T01:14:33.802699+0000","last_peered":"2024-04-25T01:14:33.802699+0000","last_clean":"2024-04-25T01:14:33.802699+0000","last_became_active":"2024-04-25T01:12:07.905724+0000","last_became_peered":"2024-04-25T01:12:07.905724+0000","last_unstale":"2024-04-25T01:14:33.802699+0000","last_undegraded":"2024-04-25T01:14:33.802699+0000","last_fullsized":"2024-04-25T01:14:33.802699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:18.712681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:18.712681+0000","last_clean_scrub_stamp":"2024-04-25T01:08:18.712681+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:13.899974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.809254+0000","last_change":"2024-04-25T01:13:31.534664+0000","last_active":"2024-04-25T01:14:33.809254+0000","last_peered":"2024-04-25T01:14:33.809254+0000","last_clean":"2024-04-25T01:14:33.809254+0000","last_became_active":"2024-04-25T01:13:31.530082+0000","last_became_peered":"2024-04-25T01:13:31.530082+0000","last_unstale":"2024-04-25T01:14:33.809254+0000","last_undegraded":"2024-04-25T01:14:33.809254+0000","last_fullsized":"2024-04-25T01:14:33.809254+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:08:32.717997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802763+0000","last_change":"2024-04-25T01:13:30.514295+0000","last_active":"2024-04-25T01:14:33.802763+0000","last_peered":"2024-04-25T01:14:33.802763+0000","last_clean":"2024-04-25T01:14:33.802763+0000","last_became_active":"2024-04-25T01:13:30.513844+0000","last_became_peered":"2024-04-25T01:13:30.513844+0000","last_unstale":"2024-04-25T01:14:33.802763+0000","last_undegraded":"2024-04-25T01:14:33.802763+0000","last_fullsized":"2024-04-25T01:14:33.802763+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:36.652771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802546+0000","last_change":"2024-04-25T01:13:30.523088+0000","last_active":"2024-04-25T01:14:33.802546+0000","last_peered":"2024-04-25T01:14:33.802546+0000","last_clean":"2024-04-25T01:14:33.802546+0000","last_became_active":"2024-04-25T01:13:30.522642+0000","last_became_peered":"2024-04-25T01:13:30.522642+0000","last_unstale":"2024-04-25T01:14:33.802546+0000","last_undegraded":"2024-04-25T01:14:33.802546+0000","last_fullsized":"2024-04-25T01:14:33.802546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:55.501983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.812271+0000","last_change":"2024-04-25T01:13:31.531703+0000","last_active":"2024-04-25T01:14:33.812271+0000","last_peered":"2024-04-25T01:14:33.812271+0000","last_clean":"2024-04-25T01:14:33.812271+0000","last_became_active":"2024-04-25T01:13:31.531160+0000","last_became_peered":"2024-04-25T01:13:31.531160+0000","last_unstale":"2024-04-25T01:14:33.812271+0000","last_undegraded":"2024-04-25T01:14:33.812271+0000","last_fullsized":"2024-04-25T01:14:33.812271+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:21.993998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.805803+0000","last_change":"2024-04-25T01:13:30.517908+0000","last_active":"2024-04-25T01:14:33.805803+0000","last_peered":"2024-04-25T01:14:33.805803+0000","last_clean":"2024-04-25T01:14:33.805803+0000","last_became_active":"2024-04-25T01:13:30.517559+0000","last_became_peered":"2024-04-25T01:13:30.517559+0000","last_unstale":"2024-04-25T01:14:33.805803+0000","last_undegraded":"2024-04-25T01:14:33.805803+0000","last_fullsized":"2024-04-25T01:14:33.805803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:44.357328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.801632+0000","last_change":"2024-04-25T01:13:31.515760+0000","last_active":"2024-04-25T01:14:33.801632+0000","last_peered":"2024-04-25T01:14:33.801632+0000","last_clean":"2024-04-25T01:14:33.801632+0000","last_became_active":"2024-04-25T01:13:31.514957+0000","last_became_peered":"2024-04-25T01:13:31.514957+0000","last_unstale":"2024-04-25T01:14:33.801632+0000","last_undegraded":"2024-04-25T01:14:33.801632+0000","last_fullsized":"2024-04-25T01:14:33.801632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:46.169171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.380150+0000","last_change":"2024-04-25T01:13:30.523933+0000","last_active":"2024-04-25T01:14:34.380150+0000","last_peered":"2024-04-25T01:14:34.380150+0000","last_clean":"2024-04-25T01:14:34.380150+0000","last_became_active":"2024-04-25T01:13:30.523497+0000","last_became_peered":"2024-04-25T01:13:30.523497+0000","last_unstale":"2024-04-25T01:14:34.380150+0000","last_undegraded":"2024-04-25T01:14:34.380150+0000","last_fullsized":"2024-04-25T01:14:34.380150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:17:59.336812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.384595+0000","last_change":"2024-04-25T01:13:31.540188+0000","last_active":"2024-04-25T01:14:34.384595+0000","last_peered":"2024-04-25T01:14:34.384595+0000","last_clean":"2024-04-25T01:14:34.384595+0000","last_became_active":"2024-04-25T01:13:31.537121+0000","last_became_peered":"2024-04-25T01:13:31.537121+0000","last_unstale":"2024-04-25T01:14:34.384595+0000","last_undegraded":"2024-04-25T01:14:34.384595+0000","last_fullsized":"2024-04-25T01:14:34.384595+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:57.162639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.190993+0000","last_change":"2024-04-25T01:13:30.516975+0000","last_active":"2024-04-25T01:14:34.190993+0000","last_peered":"2024-04-25T01:14:34.190993+0000","last_clean":"2024-04-25T01:14:34.190993+0000","last_became_active":"2024-04-25T01:13:30.516391+0000","last_became_peered":"2024-04-25T01:13:30.516391+0000","last_unstale":"2024-04-25T01:14:34.190993+0000","last_undegraded":"2024-04-25T01:14:34.190993+0000","last_fullsized":"2024-04-25T01:14:34.190993+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:57:53.486835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802216+0000","last_change":"2024-04-25T01:13:31.514491+0000","last_active":"2024-04-25T01:14:33.802216+0000","last_peered":"2024-04-25T01:14:33.802216+0000","last_clean":"2024-04-25T01:14:33.802216+0000","last_became_active":"2024-04-25T01:13:31.514156+0000","last_became_peered":"2024-04-25T01:13:31.514156+0000","last_unstale":"2024-04-25T01:14:33.802216+0000","last_undegraded":"2024-04-25T01:14:33.802216+0000","last_fullsized":"2024-04-25T01:14:33.802216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:18.588291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.801853+0000","last_change":"2024-04-25T01:13:30.521258+0000","last_active":"2024-04-25T01:14:33.801853+0000","last_peered":"2024-04-25T01:14:33.801853+0000","last_clean":"2024-04-25T01:14:33.801853+0000","last_became_active":"2024-04-25T01:13:30.518359+0000","last_became_peered":"2024-04-25T01:13:30.518359+0000","last_unstale":"2024-04-25T01:14:33.801853+0000","last_undegraded":"2024-04-25T01:14:33.801853+0000","last_fullsized":"2024-04-25T01:14:33.801853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:35.790798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.800260+0000","last_change":"2024-04-25T01:13:31.536755+0000","last_active":"2024-04-25T01:14:33.800260+0000","last_peered":"2024-04-25T01:14:33.800260+0000","last_clean":"2024-04-25T01:14:33.800260+0000","last_became_active":"2024-04-25T01:13:31.536303+0000","last_became_peered":"2024-04-25T01:13:31.536303+0000","last_unstale":"2024-04-25T01:14:33.800260+0000","last_undegraded":"2024-04-25T01:14:33.800260+0000","last_fullsized":"2024-04-25T01:14:33.800260+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:44.793993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.811323+0000","last_change":"2024-04-25T01:14:33.810882+0000","last_active":"2024-04-25T01:14:33.811323+0000","last_peered":"2024-04-25T01:14:33.811323+0000","last_clean":"2024-04-25T01:14:33.811323+0000","last_became_active":"2024-04-25T01:14:33.804758+0000","last_became_peered":"2024-04-25T01:14:33.804758+0000","last_unstale":"2024-04-25T01:14:33.811323+0000","last_undegraded":"2024-04-25T01:14:33.811323+0000","last_fullsized":"2024-04-25T01:14:33.811323+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:53.146681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802467+0000","last_change":"2024-04-25T01:13:31.543234+0000","last_active":"2024-04-25T01:14:33.802467+0000","last_peered":"2024-04-25T01:14:33.802467+0000","last_clean":"2024-04-25T01:14:33.802467+0000","last_became_active":"2024-04-25T01:13:31.542419+0000","last_became_peered":"2024-04-25T01:13:31.542419+0000","last_unstale":"2024-04-25T01:14:33.802467+0000","last_undegraded":"2024-04-25T01:14:33.802467+0000","last_fullsized":"2024-04-25T01:14:33.802467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:57.226505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.388517+0000","last_change":"2024-04-25T01:14:34.388045+0000","last_active":"2024-04-25T01:14:34.388517+0000","last_peered":"2024-04-25T01:14:34.388517+0000","last_clean":"2024-04-25T01:14:34.388517+0000","last_became_active":"2024-04-25T01:14:34.381047+0000","last_became_peered":"2024-04-25T01:14:34.381047+0000","last_unstale":"2024-04-25T01:14:34.388517+0000","last_undegraded":"2024-04-25T01:14:34.388517+0000","last_fullsized":"2024-04-25T01:14:34.388517+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:08.243523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.803146+0000","last_change":"2024-04-25T01:13:31.532194+0000","last_active":"2024-04-25T01:14:33.803146+0000","last_peered":"2024-04-25T01:14:33.803146+0000","last_clean":"2024-04-25T01:14:33.803146+0000","last_became_active":"2024-04-25T01:13:31.531798+0000","last_became_peered":"2024-04-25T01:13:31.531798+0000","last_unstale":"2024-04-25T01:14:33.803146+0000","last_undegraded":"2024-04-25T01:14:33.803146+0000","last_fullsized":"2024-04-25T01:14:33.803146+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:36.872827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.799123+0000","last_change":"2024-04-25T01:13:30.522565+0000","last_active":"2024-04-25T01:14:33.799123+0000","last_peered":"2024-04-25T01:14:33.799123+0000","last_clean":"2024-04-25T01:14:33.799123+0000","last_became_active":"2024-04-25T01:13:30.522207+0000","last_became_peered":"2024-04-25T01:13:30.522207+0000","last_unstale":"2024-04-25T01:14:33.799123+0000","last_undegraded":"2024-04-25T01:14:33.799123+0000","last_fullsized":"2024-04-25T01:14:33.799123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:48.263665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.803043+0000","last_change":"2024-04-25T01:13:31.514330+0000","last_active":"2024-04-25T01:14:33.803043+0000","last_peered":"2024-04-25T01:14:33.803043+0000","last_clean":"2024-04-25T01:14:33.803043+0000","last_became_active":"2024-04-25T01:13:31.513948+0000","last_became_peered":"2024-04-25T01:13:31.513948+0000","last_unstale":"2024-04-25T01:14:33.803043+0000","last_undegraded":"2024-04-25T01:14:33.803043+0000","last_fullsized":"2024-04-25T01:14:33.803043+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:57.544298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.379703+0000","last_change":"2024-04-25T01:13:30.515951+0000","last_active":"2024-04-25T01:14:34.379703+0000","last_peered":"2024-04-25T01:14:34.379703+0000","last_clean":"2024-04-25T01:14:34.379703+0000","last_became_active":"2024-04-25T01:13:30.515388+0000","last_became_peered":"2024-04-25T01:13:30.515388+0000","last_unstale":"2024-04-25T01:14:34.379703+0000","last_undegraded":"2024-04-25T01:14:34.379703+0000","last_fullsized":"2024-04-25T01:14:34.379703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:11.041466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802802+0000","last_change":"2024-04-25T01:13:31.503386+0000","last_active":"2024-04-25T01:14:33.802802+0000","last_peered":"2024-04-25T01:14:33.802802+0000","last_clean":"2024-04-25T01:14:33.802802+0000","last_became_active":"2024-04-25T01:13:31.502966+0000","last_became_peered":"2024-04-25T01:13:31.502966+0000","last_unstale":"2024-04-25T01:14:33.802802+0000","last_undegraded":"2024-04-25T01:14:33.802802+0000","last_fullsized":"2024-04-25T01:14:33.802802+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:07.520811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.812829+0000","last_change":"2024-04-25T01:13:30.528033+0000","last_active":"2024-04-25T01:14:33.812829+0000","last_peered":"2024-04-25T01:14:33.812829+0000","last_clean":"2024-04-25T01:14:33.812829+0000","last_became_active":"2024-04-25T01:13:30.527609+0000","last_became_peered":"2024-04-25T01:13:30.527609+0000","last_unstale":"2024-04-25T01:14:33.812829+0000","last_undegraded":"2024-04-25T01:14:33.812829+0000","last_fullsized":"2024-04-25T01:14:33.812829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:08.182355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802984+0000","last_change":"2024-04-25T01:13:31.516569+0000","last_active":"2024-04-25T01:14:33.802984+0000","last_peered":"2024-04-25T01:14:33.802984+0000","last_clean":"2024-04-25T01:14:33.802984+0000","last_became_active":"2024-04-25T01:13:31.515746+0000","last_became_peered":"2024-04-25T01:13:31.515746+0000","last_unstale":"2024-04-25T01:14:33.802984+0000","last_undegraded":"2024-04-25T01:14:33.802984+0000","last_fullsized":"2024-04-25T01:14:33.802984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:27.860296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.809971+0000","last_change":"2024-04-25T01:14:33.809971+0000","last_active":"2024-04-25T01:14:33.809971+0000","last_peered":"2024-04-25T01:14:33.809971+0000","last_clean":"2024-04-25T01:14:33.809971+0000","last_became_active":"2024-04-25T01:14:33.809588+0000","last_became_peered":"2024-04-25T01:14:33.809588+0000","last_unstale":"2024-04-25T01:14:33.809971+0000","last_undegraded":"2024-04-25T01:14:33.809971+0000","last_fullsized":"2024-04-25T01:14:33.809971+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:24.492239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.798763+0000","last_change":"2024-04-25T01:13:31.514663+0000","last_active":"2024-04-25T01:14:33.798763+0000","last_peered":"2024-04-25T01:14:33.798763+0000","last_clean":"2024-04-25T01:14:33.798763+0000","last_became_active":"2024-04-25T01:13:31.514246+0000","last_became_peered":"2024-04-25T01:13:31.514246+0000","last_unstale":"2024-04-25T01:14:33.798763+0000","last_undegraded":"2024-04-25T01:14:33.798763+0000","last_fullsized":"2024-04-25T01:14:33.798763+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:27.235598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.798864+0000","last_change":"2024-04-25T01:13:31.533374+0000","last_active":"2024-04-25T01:14:33.798864+0000","last_peered":"2024-04-25T01:14:33.798864+0000","last_clean":"2024-04-25T01:14:33.798864+0000","last_became_active":"2024-04-25T01:13:31.533145+0000","last_became_peered":"2024-04-25T01:13:31.533145+0000","last_unstale":"2024-04-25T01:14:33.798864+0000","last_undegraded":"2024-04-25T01:14:33.798864+0000","last_fullsized":"2024-04-25T01:14:33.798864+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:50.987823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.383168+0000","last_change":"2024-04-25T01:13:31.541682+0000","last_active":"2024-04-25T01:14:34.383168+0000","last_peered":"2024-04-25T01:14:34.383168+0000","last_clean":"2024-04-25T01:14:34.383168+0000","last_became_active":"2024-04-25T01:13:31.541257+0000","last_became_peered":"2024-04-25T01:13:31.541257+0000","last_unstale":"2024-04-25T01:14:34.383168+0000","last_undegraded":"2024-04-25T01:14:34.383168+0000","last_fullsized":"2024-04-25T01:14:34.383168+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:08.033895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.387542+0000","last_change":"2024-04-25T01:14:34.387542+0000","last_active":"2024-04-25T01:14:34.387542+0000","last_peered":"2024-04-25T01:14:34.387542+0000","last_clean":"2024-04-25T01:14:34.387542+0000","last_became_active":"2024-04-25T01:14:34.385838+0000","last_became_peered":"2024-04-25T01:14:34.385838+0000","last_unstale":"2024-04-25T01:14:34.387542+0000","last_undegraded":"2024-04-25T01:14:34.387542+0000","last_fullsized":"2024-04-25T01:14:34.387542+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:59.361382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.800447+0000","last_change":"2024-04-25T01:13:31.519068+0000","last_active":"2024-04-25T01:14:33.800447+0000","last_peered":"2024-04-25T01:14:33.800447+0000","last_clean":"2024-04-25T01:14:33.800447+0000","last_became_active":"2024-04-25T01:13:31.518705+0000","last_became_peered":"2024-04-25T01:13:31.518705+0000","last_unstale":"2024-04-25T01:14:33.800447+0000","last_undegraded":"2024-04-25T01:14:33.800447+0000","last_fullsized":"2024-04-25T01:14:33.800447+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:11.186149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.799066+0000","last_change":"2024-04-25T01:13:31.514572+0000","last_active":"2024-04-25T01:14:33.799066+0000","last_peered":"2024-04-25T01:14:33.799066+0000","last_clean":"2024-04-25T01:14:33.799066+0000","last_became_active":"2024-04-25T01:13:31.514178+0000","last_became_peered":"2024-04-25T01:13:31.514178+0000","last_unstale":"2024-04-25T01:14:33.799066+0000","last_undegraded":"2024-04-25T01:14:33.799066+0000","last_fullsized":"2024-04-25T01:14:33.799066+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:17.856802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.801510+0000","last_change":"2024-04-25T01:13:31.520837+0000","last_active":"2024-04-25T01:14:33.801510+0000","last_peered":"2024-04-25T01:14:33.801510+0000","last_clean":"2024-04-25T01:14:33.801510+0000","last_became_active":"2024-04-25T01:13:31.515601+0000","last_became_peered":"2024-04-25T01:13:31.515601+0000","last_unstale":"2024-04-25T01:14:33.801510+0000","last_undegraded":"2024-04-25T01:14:33.801510+0000","last_fullsized":"2024-04-25T01:14:33.801510+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:37.583156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.379883+0000","last_change":"2024-04-25T01:13:31.527420+0000","last_active":"2024-04-25T01:14:34.379883+0000","last_peered":"2024-04-25T01:14:34.379883+0000","last_clean":"2024-04-25T01:14:34.379883+0000","last_became_active":"2024-04-25T01:13:31.527160+0000","last_became_peered":"2024-04-25T01:13:31.527160+0000","last_unstale":"2024-04-25T01:14:34.379883+0000","last_undegraded":"2024-04-25T01:14:34.379883+0000","last_fullsized":"2024-04-25T01:14:34.379883+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:17:46.201486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802154+0000","last_change":"2024-04-25T01:13:31.518356+0000","last_active":"2024-04-25T01:14:33.802154+0000","last_peered":"2024-04-25T01:14:33.802154+0000","last_clean":"2024-04-25T01:14:33.802154+0000","last_became_active":"2024-04-25T01:13:31.517654+0000","last_became_peered":"2024-04-25T01:13:31.517654+0000","last_unstale":"2024-04-25T01:14:33.802154+0000","last_undegraded":"2024-04-25T01:14:33.802154+0000","last_fullsized":"2024-04-25T01:14:33.802154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:17.254963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.191030+0000","last_change":"2024-04-25T01:13:31.526217+0000","last_active":"2024-04-25T01:14:34.191030+0000","last_peered":"2024-04-25T01:14:34.191030+0000","last_clean":"2024-04-25T01:14:34.191030+0000","last_became_active":"2024-04-25T01:13:31.525801+0000","last_became_peered":"2024-04-25T01:13:31.525801+0000","last_unstale":"2024-04-25T01:14:34.191030+0000","last_undegraded":"2024-04-25T01:14:34.191030+0000","last_fullsized":"2024-04-25T01:14:34.191030+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:48:30.244205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.814205+0000","last_change":"2024-04-25T01:13:31.543596+0000","last_active":"2024-04-25T01:14:33.814205+0000","last_peered":"2024-04-25T01:14:33.814205+0000","last_clean":"2024-04-25T01:14:33.814205+0000","last_became_active":"2024-04-25T01:13:31.542149+0000","last_became_peered":"2024-04-25T01:13:31.542149+0000","last_unstale":"2024-04-25T01:14:33.814205+0000","last_undegraded":"2024-04-25T01:14:33.814205+0000","last_fullsized":"2024-04-25T01:14:33.814205+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:05:50.392151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.800265+0000","last_change":"2024-04-25T01:13:31.513575+0000","last_active":"2024-04-25T01:14:33.800265+0000","last_peered":"2024-04-25T01:14:33.800265+0000","last_clean":"2024-04-25T01:14:33.800265+0000","last_became_active":"2024-04-25T01:13:31.513085+0000","last_became_peered":"2024-04-25T01:13:31.513085+0000","last_unstale":"2024-04-25T01:14:33.800265+0000","last_undegraded":"2024-04-25T01:14:33.800265+0000","last_fullsized":"2024-04-25T01:14:33.800265+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:36.853033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62881,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":522,"num_read_kb":326,"num_write":158,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":188643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318306,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":462720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579942,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32796,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712356,"statfs":{"total":95995035648,"available":95961452544,"internally_reserved":0,"allocated":1654784,"data_stored":1113472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743467,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":462956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005106,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":462876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266742,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32716,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712436,"statfs":{"total":95995035648,"available":95961534464,"internally_reserved":0,"allocated":1568768,"data_stored":1054806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430268,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":462345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1024000,"data_stored":521018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":463616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":522368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117067,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":978944,"data_stored":462656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855431,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":463173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691906,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1581056,"data_stored":1054099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:15:02.464 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T01:15:02.975 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:15:03.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:02 smithi169 ceph-mon[32759]: from='client.24922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:15:03.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:02 smithi040 ceph-mon[32518]: from='client.24922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:15:03.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:02 smithi031 ceph-mon[28053]: from='client.24922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:15:04.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:03 smithi169 ceph-mon[32759]: pgmap v364: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:04.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:03 smithi169 ceph-mon[32759]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-25T01:15:04.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:03 smithi040 ceph-mon[32518]: pgmap v364: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:04.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:03 smithi040 ceph-mon[32518]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-25T01:15:04.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:03 smithi031 ceph-mon[28053]: pgmap v364: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:04.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:03 smithi031 ceph-mon[28053]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-25T01:15:05.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.221+0000 7f9eee600700 1 -- 172.21.15.31:0/4013165622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee8102e90 msgr2=0x7f9ee8106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:05.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.221+0000 7f9eee600700 1 --2- 172.21.15.31:0/4013165622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee8102e90 0x7f9ee8106e90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0009990 tx=0x7f9ed0009ca0 comp rx=0 tx=0).stop 2024-04-25T01:15:05.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.222+0000 7f9eee600700 1 -- 172.21.15.31:0/4013165622 shutdown_connections 2024-04-25T01:15:05.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.222+0000 7f9eee600700 1 --2- 172.21.15.31:0/4013165622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ee8107570 0x7f9ee810f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:05.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.222+0000 7f9eee600700 1 --2- 172.21.15.31:0/4013165622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee81024e0 0x7f9ee81028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:05.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.222+0000 7f9eee600700 1 --2- 172.21.15.31:0/4013165622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee8102e90 0x7f9ee8106e90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:05.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.222+0000 7f9eee600700 1 -- 172.21.15.31:0/4013165622 >> 172.21.15.31:0/4013165622 conn(0x7f9ee8079390 msgr2=0x7f9ee80797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:05.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.222+0000 7f9eee600700 1 -- 172.21.15.31:0/4013165622 shutdown_connections 2024-04-25T01:15:05.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.222+0000 7f9eee600700 1 -- 172.21.15.31:0/4013165622 wait complete. 2024-04-25T01:15:05.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.223+0000 7f9eee600700 1 Processor -- start 2024-04-25T01:15:05.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.223+0000 7f9eee600700 1 -- start start 2024-04-25T01:15:05.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.223+0000 7f9eee600700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee81024e0 0x7f9ee8074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:05.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.223+0000 7f9eee600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ee8102e90 0x7f9ee8074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:05.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.223+0000 7f9eee600700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee8107570 0x7f9ee8074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:05.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.223+0000 7f9eee600700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee8112530 con 0x7f9ee81024e0 2024-04-25T01:15:05.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.223+0000 7f9eee600700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee81123b0 con 0x7f9ee8107570 2024-04-25T01:15:05.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.223+0000 7f9eee600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee81126b0 con 0x7f9ee8102e90 2024-04-25T01:15:05.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.224+0000 7f9ee7e00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee8107570 0x7f9ee8074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:05.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.224+0000 7f9ee7400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee81024e0 0x7f9ee8074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:05.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.224+0000 7f9ee7e00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee8107570 0x7f9ee8074fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:40606/0 (socket says 172.21.15.31:40606) 2024-04-25T01:15:05.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.224+0000 7f9ee7e00700 1 -- 172.21.15.31:0/3315937389 learned_addr learned my addr 172.21.15.31:0/3315937389 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:05.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.224+0000 7f9edfe00700 1 --2- 172.21.15.31:0/3315937389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ee8102e90 0x7f9ee8074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:05.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.224+0000 7f9ee7e00700 1 -- 172.21.15.31:0/3315937389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ee8102e90 msgr2=0x7f9ee8074a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:05.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.224+0000 7f9ee7e00700 1 --2- 172.21.15.31:0/3315937389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ee8102e90 0x7f9ee8074a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:05.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.224+0000 7f9ee7e00700 1 -- 172.21.15.31:0/3315937389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee81024e0 msgr2=0x7f9ee8074530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:05.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.224+0000 7f9ee7e00700 1 --2- 172.21.15.31:0/3315937389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee81024e0 0x7f9ee8074530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:05.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.224+0000 7f9ee7e00700 1 -- 172.21.15.31:0/3315937389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee81a6f50 con 0x7f9ee8107570 2024-04-25T01:15:05.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.224+0000 7f9edfe00700 1 --2- 172.21.15.31:0/3315937389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ee8102e90 0x7f9ee8074a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:05.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.225+0000 7f9ee7e00700 1 --2- 172.21.15.31:0/3315937389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee8107570 0x7f9ee8074fb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000ef30 tx=0x7f9ee000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:05.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.225+0000 7f9ee4c00700 1 -- 172.21.15.31:0/3315937389 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee00092e0 con 0x7f9ee8107570 2024-04-25T01:15:05.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.225+0000 7f9eee600700 1 -- 172.21.15.31:0/3315937389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee81a71e0 con 0x7f9ee8107570 2024-04-25T01:15:05.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.225+0000 7f9ee4c00700 1 -- 172.21.15.31:0/3315937389 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee0004bb0 con 0x7f9ee8107570 2024-04-25T01:15:05.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.225+0000 7f9ee4c00700 1 -- 172.21.15.31:0/3315937389 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0004280 con 0x7f9ee8107570 2024-04-25T01:15:05.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.225+0000 7f9eee600700 1 -- 172.21.15.31:0/3315937389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee81a76f0 con 0x7f9ee8107570 2024-04-25T01:15:05.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.227+0000 7f9eee600700 1 -- 172.21.15.31:0/3315937389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ee8068f20 con 0x7f9ee8107570 2024-04-25T01:15:05.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.228+0000 7f9ee4c00700 1 -- 172.21.15.31:0/3315937389 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9ee001b030 con 0x7f9ee8107570 2024-04-25T01:15:05.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.230+0000 7f9ee4c00700 1 --2- 172.21.15.31:0/3315937389 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f9ecc0a2ca0 0x7f9ecc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:05.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.230+0000 7f9ee4c00700 1 -- 172.21.15.31:0/3315937389 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f9ee00cea60 con 0x7f9ee8107570 2024-04-25T01:15:05.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.230+0000 7f9ee7400700 1 --2- 172.21.15.31:0/3315937389 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f9ecc0a2ca0 0x7f9ecc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:05.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.231+0000 7f9ee7400700 1 --2- 172.21.15.31:0/3315937389 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f9ecc0a2ca0 0x7f9ecc0a5160 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009680 tx=0x7f9ed8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:05.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.234+0000 7f9ee4c00700 1 -- 172.21.15.31:0/3315937389 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9ee0022020 con 0x7f9ee8107570 2024-04-25T01:15:05.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.465+0000 7f9eee600700 1 -- 172.21.15.31:0/3315937389 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9ee8062670 con 0x7f9ecc0a2ca0 2024-04-25T01:15:05.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.489+0000 7f9ee4c00700 1 -- 172.21.15.31:0/3315937389 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258831 (secure 0 0 0) 0x7f9ee8062670 con 0x7f9ecc0a2ca0 2024-04-25T01:15:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:15:05.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.496+0000 7f9eee600700 1 -- 172.21.15.31:0/3315937389 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f9ecc0a2ca0 msgr2=0x7f9ecc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:05.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.496+0000 7f9eee600700 1 --2- 172.21.15.31:0/3315937389 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f9ecc0a2ca0 0x7f9ecc0a5160 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009680 tx=0x7f9ed8009260 comp rx=0 tx=0).stop 2024-04-25T01:15:05.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.496+0000 7f9eee600700 1 -- 172.21.15.31:0/3315937389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee8107570 msgr2=0x7f9ee8074fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:05.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.496+0000 7f9eee600700 1 --2- 172.21.15.31:0/3315937389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee8107570 0x7f9ee8074fb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000ef30 tx=0x7f9ee000c3a0 comp rx=0 tx=0).stop 2024-04-25T01:15:05.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.497+0000 7f9eee600700 1 -- 172.21.15.31:0/3315937389 shutdown_connections 2024-04-25T01:15:05.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.497+0000 7f9eee600700 1 --2- 172.21.15.31:0/3315937389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee81024e0 0x7f9ee8074530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:05.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.497+0000 7f9eee600700 1 --2- 172.21.15.31:0/3315937389 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f9ecc0a2ca0 0x7f9ecc0a5160 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:05.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.497+0000 7f9eee600700 1 --2- 172.21.15.31:0/3315937389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee8107570 0x7f9ee8074fb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:05.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.497+0000 7f9eee600700 1 --2- 172.21.15.31:0/3315937389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ee8102e90 0x7f9ee8074a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:05.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.497+0000 7f9eee600700 1 -- 172.21.15.31:0/3315937389 >> 172.21.15.31:0/3315937389 conn(0x7f9ee8079390 msgr2=0x7f9ee8101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:05.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.497+0000 7f9eee600700 1 -- 172.21.15.31:0/3315937389 shutdown_connections 2024-04-25T01:15:05.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:05.497+0000 7f9eee600700 1 -- 172.21.15.31:0/3315937389 wait complete. 2024-04-25T01:15:05.485 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-04-25T01:15:06.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:05 smithi031 ceph-mon[28053]: pgmap v365: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:15:06.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:05 smithi169 ceph-mon[32759]: pgmap v365: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:15:06.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:05 smithi040 ceph-mon[32518]: pgmap v365: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:15:06.444 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-04-25T01:15:04.107788+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653732,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":674,"num_read_kb":451,"num_write":346,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":444,"ondisk_log_size":444,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375052,"kb_used_data":13356,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566772,"statfs":{"total":1151940427776,"available":1151556374528,"internally_reserved":0,"allocated":13676544,"data_stored":7506744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":41,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005376"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.800410+0000","last_change":"2024-04-25T01:13:31.536782+0000","last_active":"2024-04-25T01:14:33.800410+0000","last_peered":"2024-04-25T01:14:33.800410+0000","last_clean":"2024-04-25T01:14:33.800410+0000","last_became_active":"2024-04-25T01:13:31.536426+0000","last_became_peered":"2024-04-25T01:13:31.536426+0000","last_unstale":"2024-04-25T01:14:33.800410+0000","last_undegraded":"2024-04-25T01:14:33.800410+0000","last_fullsized":"2024-04-25T01:14:33.800410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:07.489179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.384567+0000","last_change":"2024-04-25T01:13:31.543521+0000","last_active":"2024-04-25T01:14:34.384567+0000","last_peered":"2024-04-25T01:14:34.384567+0000","last_clean":"2024-04-25T01:14:34.384567+0000","last_became_active":"2024-04-25T01:13:31.542767+0000","last_became_peered":"2024-04-25T01:13:31.542767+0000","last_unstale":"2024-04-25T01:14:34.384567+0000","last_undegraded":"2024-04-25T01:14:34.384567+0000","last_fullsized":"2024-04-25T01:14:34.384567+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:33.287564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.383601+0000","last_change":"2024-04-25T01:13:31.537297+0000","last_active":"2024-04-25T01:14:34.383601+0000","last_peered":"2024-04-25T01:14:34.383601+0000","last_clean":"2024-04-25T01:14:34.383601+0000","last_became_active":"2024-04-25T01:13:31.536930+0000","last_became_peered":"2024-04-25T01:13:31.536930+0000","last_unstale":"2024-04-25T01:14:34.383601+0000","last_undegraded":"2024-04-25T01:14:34.383601+0000","last_fullsized":"2024-04-25T01:14:34.383601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:58:35.465287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.380366+0000","last_change":"2024-04-25T01:13:31.516514+0000","last_active":"2024-04-25T01:14:34.380366+0000","last_peered":"2024-04-25T01:14:34.380366+0000","last_clean":"2024-04-25T01:14:34.380366+0000","last_became_active":"2024-04-25T01:13:31.516192+0000","last_became_peered":"2024-04-25T01:13:31.516192+0000","last_unstale":"2024-04-25T01:14:34.380366+0000","last_undegraded":"2024-04-25T01:14:34.380366+0000","last_fullsized":"2024-04-25T01:14:34.380366+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:27.416626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802262+0000","last_change":"2024-04-25T01:13:31.538662+0000","last_active":"2024-04-25T01:14:33.802262+0000","last_peered":"2024-04-25T01:14:33.802262+0000","last_clean":"2024-04-25T01:14:33.802262+0000","last_became_active":"2024-04-25T01:13:31.538362+0000","last_became_peered":"2024-04-25T01:13:31.538362+0000","last_unstale":"2024-04-25T01:14:33.802262+0000","last_undegraded":"2024-04-25T01:14:33.802262+0000","last_fullsized":"2024-04-25T01:14:33.802262+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:10.744891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:15:02.088336+0000","last_change":"2024-04-25T01:14:33.811985+0000","last_active":"2024-04-25T01:15:02.088336+0000","last_peered":"2024-04-25T01:15:02.088336+0000","last_clean":"2024-04-25T01:15:02.088336+0000","last_became_active":"2024-04-25T01:14:33.811680+0000","last_became_peered":"2024-04-25T01:14:33.811680+0000","last_unstale":"2024-04-25T01:15:02.088336+0000","last_undegraded":"2024-04-25T01:15:02.088336+0000","last_fullsized":"2024-04-25T01:15:02.088336+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:25.004679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802564+0000","last_change":"2024-04-25T01:13:31.530362+0000","last_active":"2024-04-25T01:14:33.802564+0000","last_peered":"2024-04-25T01:14:33.802564+0000","last_clean":"2024-04-25T01:14:33.802564+0000","last_became_active":"2024-04-25T01:13:31.529869+0000","last_became_peered":"2024-04-25T01:13:31.529869+0000","last_unstale":"2024-04-25T01:14:33.802564+0000","last_undegraded":"2024-04-25T01:14:33.802564+0000","last_fullsized":"2024-04-25T01:14:33.802564+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:54.499230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:15:02.101004+0000","last_change":"2024-04-25T01:14:34.383073+0000","last_active":"2024-04-25T01:15:02.101004+0000","last_peered":"2024-04-25T01:15:02.101004+0000","last_clean":"2024-04-25T01:15:02.101004+0000","last_became_active":"2024-04-25T01:14:34.381510+0000","last_became_peered":"2024-04-25T01:14:34.381510+0000","last_unstale":"2024-04-25T01:15:02.101004+0000","last_undegraded":"2024-04-25T01:15:02.101004+0000","last_fullsized":"2024-04-25T01:15:02.101004+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:10.130178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:15:02.104729+0000","last_change":"2024-04-25T01:13:31.518589+0000","last_active":"2024-04-25T01:15:02.104729+0000","last_peered":"2024-04-25T01:15:02.104729+0000","last_clean":"2024-04-25T01:15:02.104729+0000","last_became_active":"2024-04-25T01:13:31.518183+0000","last_became_peered":"2024-04-25T01:13:31.518183+0000","last_unstale":"2024-04-25T01:15:02.104729+0000","last_undegraded":"2024-04-25T01:15:02.104729+0000","last_fullsized":"2024-04-25T01:15:02.104729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:06:17.534019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:15:02.095730+0000","last_change":"2024-04-25T01:13:31.522281+0000","last_active":"2024-04-25T01:15:02.095730+0000","last_peered":"2024-04-25T01:15:02.095730+0000","last_clean":"2024-04-25T01:15:02.095730+0000","last_became_active":"2024-04-25T01:13:31.521906+0000","last_became_peered":"2024-04-25T01:13:31.521906+0000","last_unstale":"2024-04-25T01:15:02.095730+0000","last_undegraded":"2024-04-25T01:15:02.095730+0000","last_fullsized":"2024-04-25T01:15:02.095730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:58.761284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802897+0000","last_change":"2024-04-25T01:13:31.514766+0000","last_active":"2024-04-25T01:14:33.802897+0000","last_peered":"2024-04-25T01:14:33.802897+0000","last_clean":"2024-04-25T01:14:33.802897+0000","last_became_active":"2024-04-25T01:13:31.514391+0000","last_became_peered":"2024-04-25T01:13:31.514391+0000","last_unstale":"2024-04-25T01:14:33.802897+0000","last_undegraded":"2024-04-25T01:14:33.802897+0000","last_fullsized":"2024-04-25T01:14:33.802897+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:07.398446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.380208+0000","last_change":"2024-04-25T01:13:31.518976+0000","last_active":"2024-04-25T01:14:34.380208+0000","last_peered":"2024-04-25T01:14:34.380208+0000","last_clean":"2024-04-25T01:14:34.380208+0000","last_became_active":"2024-04-25T01:13:31.518566+0000","last_became_peered":"2024-04-25T01:13:31.518566+0000","last_unstale":"2024-04-25T01:14:34.380208+0000","last_undegraded":"2024-04-25T01:14:34.380208+0000","last_fullsized":"2024-04-25T01:14:34.380208+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:59.477996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.814514+0000","last_change":"2024-04-25T01:13:31.532880+0000","last_active":"2024-04-25T01:14:33.814514+0000","last_peered":"2024-04-25T01:14:33.814514+0000","last_clean":"2024-04-25T01:14:33.814514+0000","last_became_active":"2024-04-25T01:13:31.531470+0000","last_became_peered":"2024-04-25T01:13:31.531470+0000","last_unstale":"2024-04-25T01:14:33.814514+0000","last_undegraded":"2024-04-25T01:14:33.814514+0000","last_fullsized":"2024-04-25T01:14:33.814514+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:55.431431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.383926+0000","last_change":"2024-04-25T01:13:31.532472+0000","last_active":"2024-04-25T01:14:34.383926+0000","last_peered":"2024-04-25T01:14:34.383926+0000","last_clean":"2024-04-25T01:14:34.383926+0000","last_became_active":"2024-04-25T01:13:31.530850+0000","last_became_peered":"2024-04-25T01:13:31.530850+0000","last_unstale":"2024-04-25T01:14:34.383926+0000","last_undegraded":"2024-04-25T01:14:34.383926+0000","last_fullsized":"2024-04-25T01:14:34.383926+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:50:26.890858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.382279+0000","last_change":"2024-04-25T01:13:31.539930+0000","last_active":"2024-04-25T01:14:34.382279+0000","last_peered":"2024-04-25T01:14:34.382279+0000","last_clean":"2024-04-25T01:14:34.382279+0000","last_became_active":"2024-04-25T01:13:31.539523+0000","last_became_peered":"2024-04-25T01:13:31.539523+0000","last_unstale":"2024-04-25T01:14:34.382279+0000","last_undegraded":"2024-04-25T01:14:34.382279+0000","last_fullsized":"2024-04-25T01:14:34.382279+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:07.983893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:15:02.109110+0000","last_change":"2024-04-25T01:13:31.536437+0000","last_active":"2024-04-25T01:15:02.109110+0000","last_peered":"2024-04-25T01:15:02.109110+0000","last_clean":"2024-04-25T01:15:02.109110+0000","last_became_active":"2024-04-25T01:13:31.536066+0000","last_became_peered":"2024-04-25T01:13:31.536066+0000","last_unstale":"2024-04-25T01:15:02.109110+0000","last_undegraded":"2024-04-25T01:15:02.109110+0000","last_fullsized":"2024-04-25T01:15:02.109110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:50.986971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.800646+0000","last_change":"2024-04-25T01:13:31.536706+0000","last_active":"2024-04-25T01:14:33.800646+0000","last_peered":"2024-04-25T01:14:33.800646+0000","last_clean":"2024-04-25T01:14:33.800646+0000","last_became_active":"2024-04-25T01:13:31.536318+0000","last_became_peered":"2024-04-25T01:13:31.536318+0000","last_unstale":"2024-04-25T01:14:33.800646+0000","last_undegraded":"2024-04-25T01:14:33.800646+0000","last_fullsized":"2024-04-25T01:14:33.800646+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:35.526515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:15:02.109748+0000","last_change":"2024-04-25T01:13:31.524462+0000","last_active":"2024-04-25T01:15:02.109748+0000","last_peered":"2024-04-25T01:15:02.109748+0000","last_clean":"2024-04-25T01:15:02.109748+0000","last_became_active":"2024-04-25T01:13:31.524018+0000","last_became_peered":"2024-04-25T01:13:31.524018+0000","last_unstale":"2024-04-25T01:15:02.109748+0000","last_undegraded":"2024-04-25T01:15:02.109748+0000","last_fullsized":"2024-04-25T01:15:02.109748+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:43.767053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:15:02.092307+0000","last_change":"2024-04-25T01:13:31.531553+0000","last_active":"2024-04-25T01:15:02.092307+0000","last_peered":"2024-04-25T01:15:02.092307+0000","last_clean":"2024-04-25T01:15:02.092307+0000","last_became_active":"2024-04-25T01:13:31.531237+0000","last_became_peered":"2024-04-25T01:13:31.531237+0000","last_unstale":"2024-04-25T01:15:02.092307+0000","last_undegraded":"2024-04-25T01:15:02.092307+0000","last_fullsized":"2024-04-25T01:15:02.092307+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:41.839370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.805877+0000","last_change":"2024-04-25T01:13:30.516847+0000","last_active":"2024-04-25T01:14:33.805877+0000","last_peered":"2024-04-25T01:14:33.805877+0000","last_clean":"2024-04-25T01:14:33.805877+0000","last_became_active":"2024-04-25T01:13:30.516500+0000","last_became_peered":"2024-04-25T01:13:30.516500+0000","last_unstale":"2024-04-25T01:14:33.805877+0000","last_undegraded":"2024-04-25T01:14:33.805877+0000","last_fullsized":"2024-04-25T01:14:33.805877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:37.703075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.801626+0000","last_change":"2024-04-25T01:13:31.518822+0000","last_active":"2024-04-25T01:14:33.801626+0000","last_peered":"2024-04-25T01:14:33.801626+0000","last_clean":"2024-04-25T01:14:33.801626+0000","last_became_active":"2024-04-25T01:13:31.516726+0000","last_became_peered":"2024-04-25T01:13:31.516726+0000","last_unstale":"2024-04-25T01:14:33.801626+0000","last_undegraded":"2024-04-25T01:14:33.801626+0000","last_fullsized":"2024-04-25T01:14:33.801626+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:02.360104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.801965+0000","last_change":"2024-04-25T01:13:30.531875+0000","last_active":"2024-04-25T01:14:33.801965+0000","last_peered":"2024-04-25T01:14:33.801965+0000","last_clean":"2024-04-25T01:14:33.801965+0000","last_became_active":"2024-04-25T01:13:30.531532+0000","last_became_peered":"2024-04-25T01:13:30.531532+0000","last_unstale":"2024-04-25T01:14:33.801965+0000","last_undegraded":"2024-04-25T01:14:33.801965+0000","last_fullsized":"2024-04-25T01:14:33.801965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:30.820946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.385143+0000","last_change":"2024-04-25T01:13:31.534211+0000","last_active":"2024-04-25T01:14:34.385143+0000","last_peered":"2024-04-25T01:14:34.385143+0000","last_clean":"2024-04-25T01:14:34.385143+0000","last_became_active":"2024-04-25T01:13:31.533828+0000","last_became_peered":"2024-04-25T01:13:31.533828+0000","last_unstale":"2024-04-25T01:14:34.385143+0000","last_undegraded":"2024-04-25T01:14:34.385143+0000","last_fullsized":"2024-04-25T01:14:34.385143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:39.488524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802646+0000","last_change":"2024-04-25T01:13:30.517922+0000","last_active":"2024-04-25T01:14:33.802646+0000","last_peered":"2024-04-25T01:14:33.802646+0000","last_clean":"2024-04-25T01:14:33.802646+0000","last_became_active":"2024-04-25T01:13:30.517571+0000","last_became_peered":"2024-04-25T01:13:30.517571+0000","last_unstale":"2024-04-25T01:14:33.802646+0000","last_undegraded":"2024-04-25T01:14:33.802646+0000","last_fullsized":"2024-04-25T01:14:33.802646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:16:34.901210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:15:02.108898+0000","last_change":"2024-04-25T01:13:31.530460+0000","last_active":"2024-04-25T01:15:02.108898+0000","last_peered":"2024-04-25T01:15:02.108898+0000","last_clean":"2024-04-25T01:15:02.108898+0000","last_became_active":"2024-04-25T01:13:31.530084+0000","last_became_peered":"2024-04-25T01:13:31.530084+0000","last_unstale":"2024-04-25T01:15:02.108898+0000","last_undegraded":"2024-04-25T01:15:02.108898+0000","last_fullsized":"2024-04-25T01:15:02.108898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:36.810301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802481+0000","last_change":"2024-04-25T01:13:30.514320+0000","last_active":"2024-04-25T01:14:33.802481+0000","last_peered":"2024-04-25T01:14:33.802481+0000","last_clean":"2024-04-25T01:14:33.802481+0000","last_became_active":"2024-04-25T01:13:30.513752+0000","last_became_peered":"2024-04-25T01:13:30.513752+0000","last_unstale":"2024-04-25T01:14:33.802481+0000","last_undegraded":"2024-04-25T01:14:33.802481+0000","last_fullsized":"2024-04-25T01:14:33.802481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:19.814454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.379840+0000","last_change":"2024-04-25T01:13:31.514516+0000","last_active":"2024-04-25T01:14:34.379840+0000","last_peered":"2024-04-25T01:14:34.379840+0000","last_clean":"2024-04-25T01:14:34.379840+0000","last_became_active":"2024-04-25T01:13:31.514208+0000","last_became_peered":"2024-04-25T01:13:31.514208+0000","last_unstale":"2024-04-25T01:14:34.379840+0000","last_undegraded":"2024-04-25T01:14:34.379840+0000","last_fullsized":"2024-04-25T01:14:34.379840+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:10:11.204770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":110,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:57.654082+0000","last_change":"2024-04-25T01:13:30.519701+0000","last_active":"2024-04-25T01:14:57.654082+0000","last_peered":"2024-04-25T01:14:57.654082+0000","last_clean":"2024-04-25T01:14:57.654082+0000","last_became_active":"2024-04-25T01:13:30.519315+0000","last_became_peered":"2024-04-25T01:13:30.519315+0000","last_unstale":"2024-04-25T01:14:57.654082+0000","last_undegraded":"2024-04-25T01:14:57.654082+0000","last_fullsized":"2024-04-25T01:14:57.654082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:34.613473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802442+0000","last_change":"2024-04-25T01:13:31.528376+0000","last_active":"2024-04-25T01:14:33.802442+0000","last_peered":"2024-04-25T01:14:33.802442+0000","last_clean":"2024-04-25T01:14:33.802442+0000","last_became_active":"2024-04-25T01:13:31.527622+0000","last_became_peered":"2024-04-25T01:13:31.527622+0000","last_unstale":"2024-04-25T01:14:33.802442+0000","last_undegraded":"2024-04-25T01:14:33.802442+0000","last_fullsized":"2024-04-25T01:14:33.802442+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:17.901898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.805174+0000","last_change":"2024-04-25T01:13:30.514798+0000","last_active":"2024-04-25T01:14:33.805174+0000","last_peered":"2024-04-25T01:14:33.805174+0000","last_clean":"2024-04-25T01:14:33.805174+0000","last_became_active":"2024-04-25T01:13:30.514422+0000","last_became_peered":"2024-04-25T01:13:30.514422+0000","last_unstale":"2024-04-25T01:14:33.805174+0000","last_undegraded":"2024-04-25T01:14:33.805174+0000","last_fullsized":"2024-04-25T01:14:33.805174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:27.584578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.379763+0000","last_change":"2024-04-25T01:13:31.510479+0000","last_active":"2024-04-25T01:14:34.379763+0000","last_peered":"2024-04-25T01:14:34.379763+0000","last_clean":"2024-04-25T01:14:34.379763+0000","last_became_active":"2024-04-25T01:13:31.509166+0000","last_became_peered":"2024-04-25T01:13:31.509166+0000","last_unstale":"2024-04-25T01:14:34.379763+0000","last_undegraded":"2024-04-25T01:14:34.379763+0000","last_fullsized":"2024-04-25T01:14:34.379763+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:42.604266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.809618+0000","last_change":"2024-04-25T01:13:30.520435+0000","last_active":"2024-04-25T01:14:33.809618+0000","last_peered":"2024-04-25T01:14:33.809618+0000","last_clean":"2024-04-25T01:14:33.809618+0000","last_became_active":"2024-04-25T01:13:30.520042+0000","last_became_peered":"2024-04-25T01:13:30.520042+0000","last_unstale":"2024-04-25T01:14:33.809618+0000","last_undegraded":"2024-04-25T01:14:33.809618+0000","last_fullsized":"2024-04-25T01:14:33.809618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:37.818149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.800866+0000","last_change":"2024-04-25T01:13:31.524799+0000","last_active":"2024-04-25T01:14:33.800866+0000","last_peered":"2024-04-25T01:14:33.800866+0000","last_clean":"2024-04-25T01:14:33.800866+0000","last_became_active":"2024-04-25T01:13:31.524454+0000","last_became_peered":"2024-04-25T01:13:31.524454+0000","last_unstale":"2024-04-25T01:14:33.800866+0000","last_undegraded":"2024-04-25T01:14:33.800866+0000","last_fullsized":"2024-04-25T01:14:33.800866+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:15.930922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.379905+0000","last_change":"2024-04-25T01:13:30.514104+0000","last_active":"2024-04-25T01:14:34.379905+0000","last_peered":"2024-04-25T01:14:34.379905+0000","last_clean":"2024-04-25T01:14:34.379905+0000","last_became_active":"2024-04-25T01:13:30.513742+0000","last_became_peered":"2024-04-25T01:13:30.513742+0000","last_unstale":"2024-04-25T01:14:34.379905+0000","last_undegraded":"2024-04-25T01:14:34.379905+0000","last_fullsized":"2024-04-25T01:14:34.379905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:04:42.189997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:15:02.102789+0000","last_change":"2024-04-25T01:13:31.528223+0000","last_active":"2024-04-25T01:15:02.102789+0000","last_peered":"2024-04-25T01:15:02.102789+0000","last_clean":"2024-04-25T01:15:02.102789+0000","last_became_active":"2024-04-25T01:13:31.526906+0000","last_became_peered":"2024-04-25T01:13:31.526906+0000","last_unstale":"2024-04-25T01:15:02.102789+0000","last_undegraded":"2024-04-25T01:15:02.102789+0000","last_fullsized":"2024-04-25T01:15:02.102789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:12:16.276974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":111,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:15:02.660329+0000","last_change":"2024-04-25T01:13:30.519174+0000","last_active":"2024-04-25T01:15:02.660329+0000","last_peered":"2024-04-25T01:15:02.660329+0000","last_clean":"2024-04-25T01:15:02.660329+0000","last_became_active":"2024-04-25T01:13:30.518792+0000","last_became_peered":"2024-04-25T01:13:30.518792+0000","last_unstale":"2024-04-25T01:15:02.660329+0000","last_undegraded":"2024-04-25T01:15:02.660329+0000","last_fullsized":"2024-04-25T01:15:02.660329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:27.746435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":89,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:15:02.108724+0000","last_change":"2024-04-25T01:13:31.521482+0000","last_active":"2024-04-25T01:15:02.108724+0000","last_peered":"2024-04-25T01:15:02.108724+0000","last_clean":"2024-04-25T01:15:02.108724+0000","last_became_active":"2024-04-25T01:13:31.521114+0000","last_became_peered":"2024-04-25T01:13:31.521114+0000","last_unstale":"2024-04-25T01:15:02.108724+0000","last_undegraded":"2024-04-25T01:15:02.108724+0000","last_fullsized":"2024-04-25T01:15:02.108724+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:48.748691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802983+0000","last_change":"2024-04-25T01:13:30.518427+0000","last_active":"2024-04-25T01:14:33.802983+0000","last_peered":"2024-04-25T01:14:33.802983+0000","last_clean":"2024-04-25T01:14:33.802983+0000","last_became_active":"2024-04-25T01:13:30.518005+0000","last_became_peered":"2024-04-25T01:13:30.518005+0000","last_unstale":"2024-04-25T01:14:33.802983+0000","last_undegraded":"2024-04-25T01:14:33.802983+0000","last_fullsized":"2024-04-25T01:14:33.802983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:29:21.087411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.384870+0000","last_change":"2024-04-25T01:13:31.532598+0000","last_active":"2024-04-25T01:14:34.384870+0000","last_peered":"2024-04-25T01:14:34.384870+0000","last_clean":"2024-04-25T01:14:34.384870+0000","last_became_active":"2024-04-25T01:13:31.530468+0000","last_became_peered":"2024-04-25T01:13:31.530468+0000","last_unstale":"2024-04-25T01:14:34.384870+0000","last_undegraded":"2024-04-25T01:14:34.384870+0000","last_fullsized":"2024-04-25T01:14:34.384870+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:28.254924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.379914+0000","last_change":"2024-04-25T01:13:30.520507+0000","last_active":"2024-04-25T01:14:34.379914+0000","last_peered":"2024-04-25T01:14:34.379914+0000","last_clean":"2024-04-25T01:14:34.379914+0000","last_became_active":"2024-04-25T01:13:30.520165+0000","last_became_peered":"2024-04-25T01:13:30.520165+0000","last_unstale":"2024-04-25T01:14:34.379914+0000","last_undegraded":"2024-04-25T01:14:34.379914+0000","last_fullsized":"2024-04-25T01:14:34.379914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:26.173090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:15:02.107507+0000","last_change":"2024-04-25T01:13:31.517021+0000","last_active":"2024-04-25T01:15:02.107507+0000","last_peered":"2024-04-25T01:15:02.107507+0000","last_clean":"2024-04-25T01:15:02.107507+0000","last_became_active":"2024-04-25T01:13:31.516676+0000","last_became_peered":"2024-04-25T01:13:31.516676+0000","last_unstale":"2024-04-25T01:15:02.107507+0000","last_undegraded":"2024-04-25T01:15:02.107507+0000","last_fullsized":"2024-04-25T01:15:02.107507+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:58:25.314544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'31","reported_seq":237,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:15:03.662103+0000","last_change":"2024-04-25T01:13:30.511963+0000","last_active":"2024-04-25T01:15:03.662103+0000","last_peered":"2024-04-25T01:15:03.662103+0000","last_clean":"2024-04-25T01:15:03.662103+0000","last_became_active":"2024-04-25T01:13:30.511142+0000","last_became_peered":"2024-04-25T01:13:30.511142+0000","last_unstale":"2024-04-25T01:15:03.662103+0000","last_undegraded":"2024-04-25T01:15:03.662103+0000","last_fullsized":"2024-04-25T01:15:03.662103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:19:53.545598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59129,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":187,"num_read_kb":151,"num_write":32,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802468+0000","last_change":"2024-04-25T01:13:31.516236+0000","last_active":"2024-04-25T01:14:33.802468+0000","last_peered":"2024-04-25T01:14:33.802468+0000","last_clean":"2024-04-25T01:14:33.802468+0000","last_became_active":"2024-04-25T01:13:31.515871+0000","last_became_peered":"2024-04-25T01:13:31.515871+0000","last_unstale":"2024-04-25T01:14:33.802468+0000","last_undegraded":"2024-04-25T01:14:33.802468+0000","last_fullsized":"2024-04-25T01:14:33.802468+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:24.464246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.798961+0000","last_change":"2024-04-25T01:13:30.513070+0000","last_active":"2024-04-25T01:14:33.798961+0000","last_peered":"2024-04-25T01:14:33.798961+0000","last_clean":"2024-04-25T01:14:33.798961+0000","last_became_active":"2024-04-25T01:13:30.512701+0000","last_became_peered":"2024-04-25T01:13:30.512701+0000","last_unstale":"2024-04-25T01:14:33.798961+0000","last_undegraded":"2024-04-25T01:14:33.798961+0000","last_fullsized":"2024-04-25T01:14:33.798961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:40:33.482272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.811735+0000","last_change":"2024-04-25T01:13:31.538313+0000","last_active":"2024-04-25T01:14:33.811735+0000","last_peered":"2024-04-25T01:14:33.811735+0000","last_clean":"2024-04-25T01:14:33.811735+0000","last_became_active":"2024-04-25T01:13:31.537907+0000","last_became_peered":"2024-04-25T01:13:31.537907+0000","last_unstale":"2024-04-25T01:14:33.811735+0000","last_undegraded":"2024-04-25T01:14:33.811735+0000","last_fullsized":"2024-04-25T01:14:33.811735+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:19.484311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.379592+0000","last_change":"2024-04-25T01:13:30.520170+0000","last_active":"2024-04-25T01:14:34.379592+0000","last_peered":"2024-04-25T01:14:34.379592+0000","last_clean":"2024-04-25T01:14:34.379592+0000","last_became_active":"2024-04-25T01:13:30.519636+0000","last_became_peered":"2024-04-25T01:13:30.519636+0000","last_unstale":"2024-04-25T01:14:34.379592+0000","last_undegraded":"2024-04-25T01:14:34.379592+0000","last_fullsized":"2024-04-25T01:14:34.379592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:00:32.670127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802918+0000","last_change":"2024-04-25T01:13:31.537630+0000","last_active":"2024-04-25T01:14:33.802918+0000","last_peered":"2024-04-25T01:14:33.802918+0000","last_clean":"2024-04-25T01:14:33.802918+0000","last_became_active":"2024-04-25T01:13:31.537164+0000","last_became_peered":"2024-04-25T01:13:31.537164+0000","last_unstale":"2024-04-25T01:14:33.802918+0000","last_undegraded":"2024-04-25T01:14:33.802918+0000","last_fullsized":"2024-04-25T01:14:33.802918+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:04.091194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":112,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:15:00.713289+0000","last_change":"2024-04-25T01:13:30.514322+0000","last_active":"2024-04-25T01:15:00.713289+0000","last_peered":"2024-04-25T01:15:00.713289+0000","last_clean":"2024-04-25T01:15:00.713289+0000","last_became_active":"2024-04-25T01:13:30.513990+0000","last_became_peered":"2024-04-25T01:13:30.513990+0000","last_unstale":"2024-04-25T01:15:00.713289+0000","last_undegraded":"2024-04-25T01:15:00.713289+0000","last_fullsized":"2024-04-25T01:15:00.713289+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:59.995489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":90,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802516+0000","last_change":"2024-04-25T01:13:31.519730+0000","last_active":"2024-04-25T01:14:33.802516+0000","last_peered":"2024-04-25T01:14:33.802516+0000","last_clean":"2024-04-25T01:14:33.802516+0000","last_became_active":"2024-04-25T01:13:31.519363+0000","last_became_peered":"2024-04-25T01:13:31.519363+0000","last_unstale":"2024-04-25T01:14:33.802516+0000","last_undegraded":"2024-04-25T01:14:33.802516+0000","last_fullsized":"2024-04-25T01:14:33.802516+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:26.400754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802325+0000","last_change":"2024-04-25T01:13:30.517220+0000","last_active":"2024-04-25T01:14:33.802325+0000","last_peered":"2024-04-25T01:14:33.802325+0000","last_clean":"2024-04-25T01:14:33.802325+0000","last_became_active":"2024-04-25T01:13:30.516825+0000","last_became_peered":"2024-04-25T01:13:30.516825+0000","last_unstale":"2024-04-25T01:14:33.802325+0000","last_undegraded":"2024-04-25T01:14:33.802325+0000","last_fullsized":"2024-04-25T01:14:33.802325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:04.452894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.379547+0000","last_change":"2024-04-25T01:13:31.517424+0000","last_active":"2024-04-25T01:14:34.379547+0000","last_peered":"2024-04-25T01:14:34.379547+0000","last_clean":"2024-04-25T01:14:34.379547+0000","last_became_active":"2024-04-25T01:13:31.517017+0000","last_became_peered":"2024-04-25T01:13:31.517017+0000","last_unstale":"2024-04-25T01:14:34.379547+0000","last_undegraded":"2024-04-25T01:14:34.379547+0000","last_fullsized":"2024-04-25T01:14:34.379547+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:16.516450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802727+0000","last_change":"2024-04-25T01:13:30.515249+0000","last_active":"2024-04-25T01:14:33.802727+0000","last_peered":"2024-04-25T01:14:33.802727+0000","last_clean":"2024-04-25T01:14:33.802727+0000","last_became_active":"2024-04-25T01:13:30.514819+0000","last_became_peered":"2024-04-25T01:13:30.514819+0000","last_unstale":"2024-04-25T01:14:33.802727+0000","last_undegraded":"2024-04-25T01:14:33.802727+0000","last_fullsized":"2024-04-25T01:14:33.802727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:57:52.503564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.800361+0000","last_change":"2024-04-25T01:13:31.519143+0000","last_active":"2024-04-25T01:14:33.800361+0000","last_peered":"2024-04-25T01:14:33.800361+0000","last_clean":"2024-04-25T01:14:33.800361+0000","last_became_active":"2024-04-25T01:13:31.518755+0000","last_became_peered":"2024-04-25T01:13:31.518755+0000","last_unstale":"2024-04-25T01:14:33.800361+0000","last_undegraded":"2024-04-25T01:14:33.800361+0000","last_fullsized":"2024-04-25T01:14:33.800361+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:02.870725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.804624+0000","last_change":"2024-04-25T01:13:30.522489+0000","last_active":"2024-04-25T01:14:33.804624+0000","last_peered":"2024-04-25T01:14:33.804624+0000","last_clean":"2024-04-25T01:14:33.804624+0000","last_became_active":"2024-04-25T01:13:30.522128+0000","last_became_peered":"2024-04-25T01:13:30.522128+0000","last_unstale":"2024-04-25T01:14:33.804624+0000","last_undegraded":"2024-04-25T01:14:33.804624+0000","last_fullsized":"2024-04-25T01:14:33.804624+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:12.875627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'5","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:15:02.101849+0000","last_change":"2024-04-25T01:13:31.537536+0000","last_active":"2024-04-25T01:15:02.101849+0000","last_peered":"2024-04-25T01:15:02.101849+0000","last_clean":"2024-04-25T01:15:02.101849+0000","last_became_active":"2024-04-25T01:13:31.537169+0000","last_became_peered":"2024-04-25T01:13:31.537169+0000","last_unstale":"2024-04-25T01:15:02.101849+0000","last_undegraded":"2024-04-25T01:15:02.101849+0000","last_fullsized":"2024-04-25T01:15:02.101849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:55.147719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.803001+0000","last_change":"2024-04-25T01:13:30.523870+0000","last_active":"2024-04-25T01:14:33.803001+0000","last_peered":"2024-04-25T01:14:33.803001+0000","last_clean":"2024-04-25T01:14:33.803001+0000","last_became_active":"2024-04-25T01:13:30.523504+0000","last_became_peered":"2024-04-25T01:13:30.523504+0000","last_unstale":"2024-04-25T01:14:33.803001+0000","last_undegraded":"2024-04-25T01:14:33.803001+0000","last_fullsized":"2024-04-25T01:14:33.803001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:42.498898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:15:02.107431+0000","last_change":"2024-04-25T01:13:31.539691+0000","last_active":"2024-04-25T01:15:02.107431+0000","last_peered":"2024-04-25T01:15:02.107431+0000","last_clean":"2024-04-25T01:15:02.107431+0000","last_became_active":"2024-04-25T01:13:31.539287+0000","last_became_peered":"2024-04-25T01:13:31.539287+0000","last_unstale":"2024-04-25T01:15:02.107431+0000","last_undegraded":"2024-04-25T01:15:02.107431+0000","last_fullsized":"2024-04-25T01:15:02.107431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:47.985167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.380057+0000","last_change":"2024-04-25T01:13:31.513438+0000","last_active":"2024-04-25T01:14:34.380057+0000","last_peered":"2024-04-25T01:14:34.380057+0000","last_clean":"2024-04-25T01:14:34.380057+0000","last_became_active":"2024-04-25T01:13:31.513087+0000","last_became_peered":"2024-04-25T01:13:31.513087+0000","last_unstale":"2024-04-25T01:14:34.380057+0000","last_undegraded":"2024-04-25T01:14:34.380057+0000","last_fullsized":"2024-04-25T01:14:34.380057+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:12.999714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":108,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:59.710750+0000","last_change":"2024-04-25T01:13:30.527528+0000","last_active":"2024-04-25T01:14:59.710750+0000","last_peered":"2024-04-25T01:14:59.710750+0000","last_clean":"2024-04-25T01:14:59.710750+0000","last_became_active":"2024-04-25T01:13:30.526040+0000","last_became_peered":"2024-04-25T01:13:30.526040+0000","last_unstale":"2024-04-25T01:14:59.710750+0000","last_undegraded":"2024-04-25T01:14:59.710750+0000","last_fullsized":"2024-04-25T01:14:59.710750+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:31.943727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.801323+0000","last_change":"2024-04-25T01:13:31.523311+0000","last_active":"2024-04-25T01:14:33.801323+0000","last_peered":"2024-04-25T01:14:33.801323+0000","last_clean":"2024-04-25T01:14:33.801323+0000","last_became_active":"2024-04-25T01:13:31.518099+0000","last_became_peered":"2024-04-25T01:13:31.518099+0000","last_unstale":"2024-04-25T01:14:33.801323+0000","last_undegraded":"2024-04-25T01:14:33.801323+0000","last_fullsized":"2024-04-25T01:14:33.801323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:02.947202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.380270+0000","last_change":"2024-04-25T01:13:30.512804+0000","last_active":"2024-04-25T01:14:34.380270+0000","last_peered":"2024-04-25T01:14:34.380270+0000","last_clean":"2024-04-25T01:14:34.380270+0000","last_became_active":"2024-04-25T01:13:30.512406+0000","last_became_peered":"2024-04-25T01:13:30.512406+0000","last_unstale":"2024-04-25T01:14:34.380270+0000","last_undegraded":"2024-04-25T01:14:34.380270+0000","last_fullsized":"2024-04-25T01:14:34.380270+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:00.042244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":345,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802699+0000","last_change":"2024-04-25T01:12:07.906198+0000","last_active":"2024-04-25T01:14:33.802699+0000","last_peered":"2024-04-25T01:14:33.802699+0000","last_clean":"2024-04-25T01:14:33.802699+0000","last_became_active":"2024-04-25T01:12:07.905724+0000","last_became_peered":"2024-04-25T01:12:07.905724+0000","last_unstale":"2024-04-25T01:14:33.802699+0000","last_undegraded":"2024-04-25T01:14:33.802699+0000","last_fullsized":"2024-04-25T01:14:33.802699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:18.712681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:18.712681+0000","last_clean_scrub_stamp":"2024-04-25T01:08:18.712681+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:13.899974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.809254+0000","last_change":"2024-04-25T01:13:31.534664+0000","last_active":"2024-04-25T01:14:33.809254+0000","last_peered":"2024-04-25T01:14:33.809254+0000","last_clean":"2024-04-25T01:14:33.809254+0000","last_became_active":"2024-04-25T01:13:31.530082+0000","last_became_peered":"2024-04-25T01:13:31.530082+0000","last_unstale":"2024-04-25T01:14:33.809254+0000","last_undegraded":"2024-04-25T01:14:33.809254+0000","last_fullsized":"2024-04-25T01:14:33.809254+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:08:32.717997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802763+0000","last_change":"2024-04-25T01:13:30.514295+0000","last_active":"2024-04-25T01:14:33.802763+0000","last_peered":"2024-04-25T01:14:33.802763+0000","last_clean":"2024-04-25T01:14:33.802763+0000","last_became_active":"2024-04-25T01:13:30.513844+0000","last_became_peered":"2024-04-25T01:13:30.513844+0000","last_unstale":"2024-04-25T01:14:33.802763+0000","last_undegraded":"2024-04-25T01:14:33.802763+0000","last_fullsized":"2024-04-25T01:14:33.802763+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:36.652771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802546+0000","last_change":"2024-04-25T01:13:30.523088+0000","last_active":"2024-04-25T01:14:33.802546+0000","last_peered":"2024-04-25T01:14:33.802546+0000","last_clean":"2024-04-25T01:14:33.802546+0000","last_became_active":"2024-04-25T01:13:30.522642+0000","last_became_peered":"2024-04-25T01:13:30.522642+0000","last_unstale":"2024-04-25T01:14:33.802546+0000","last_undegraded":"2024-04-25T01:14:33.802546+0000","last_fullsized":"2024-04-25T01:14:33.802546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:55.501983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.812271+0000","last_change":"2024-04-25T01:13:31.531703+0000","last_active":"2024-04-25T01:14:33.812271+0000","last_peered":"2024-04-25T01:14:33.812271+0000","last_clean":"2024-04-25T01:14:33.812271+0000","last_became_active":"2024-04-25T01:13:31.531160+0000","last_became_peered":"2024-04-25T01:13:31.531160+0000","last_unstale":"2024-04-25T01:14:33.812271+0000","last_undegraded":"2024-04-25T01:14:33.812271+0000","last_fullsized":"2024-04-25T01:14:33.812271+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:21.993998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.805895+0000","last_change":"2024-04-25T01:13:30.517908+0000","last_active":"2024-04-25T01:14:33.805895+0000","last_peered":"2024-04-25T01:14:33.805895+0000","last_clean":"2024-04-25T01:14:33.805895+0000","last_became_active":"2024-04-25T01:13:30.517559+0000","last_became_peered":"2024-04-25T01:13:30.517559+0000","last_unstale":"2024-04-25T01:14:33.805895+0000","last_undegraded":"2024-04-25T01:14:33.805895+0000","last_fullsized":"2024-04-25T01:14:33.805895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:44.357328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.801632+0000","last_change":"2024-04-25T01:13:31.515760+0000","last_active":"2024-04-25T01:14:33.801632+0000","last_peered":"2024-04-25T01:14:33.801632+0000","last_clean":"2024-04-25T01:14:33.801632+0000","last_became_active":"2024-04-25T01:13:31.514957+0000","last_became_peered":"2024-04-25T01:13:31.514957+0000","last_unstale":"2024-04-25T01:14:33.801632+0000","last_undegraded":"2024-04-25T01:14:33.801632+0000","last_fullsized":"2024-04-25T01:14:33.801632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:46.169171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.380150+0000","last_change":"2024-04-25T01:13:30.523933+0000","last_active":"2024-04-25T01:14:34.380150+0000","last_peered":"2024-04-25T01:14:34.380150+0000","last_clean":"2024-04-25T01:14:34.380150+0000","last_became_active":"2024-04-25T01:13:30.523497+0000","last_became_peered":"2024-04-25T01:13:30.523497+0000","last_unstale":"2024-04-25T01:14:34.380150+0000","last_undegraded":"2024-04-25T01:14:34.380150+0000","last_fullsized":"2024-04-25T01:14:34.380150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:17:59.336812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.384595+0000","last_change":"2024-04-25T01:13:31.540188+0000","last_active":"2024-04-25T01:14:34.384595+0000","last_peered":"2024-04-25T01:14:34.384595+0000","last_clean":"2024-04-25T01:14:34.384595+0000","last_became_active":"2024-04-25T01:13:31.537121+0000","last_became_peered":"2024-04-25T01:13:31.537121+0000","last_unstale":"2024-04-25T01:14:34.384595+0000","last_undegraded":"2024-04-25T01:14:34.384595+0000","last_fullsized":"2024-04-25T01:14:34.384595+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:57.162639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.190993+0000","last_change":"2024-04-25T01:13:30.516975+0000","last_active":"2024-04-25T01:14:34.190993+0000","last_peered":"2024-04-25T01:14:34.190993+0000","last_clean":"2024-04-25T01:14:34.190993+0000","last_became_active":"2024-04-25T01:13:30.516391+0000","last_became_peered":"2024-04-25T01:13:30.516391+0000","last_unstale":"2024-04-25T01:14:34.190993+0000","last_undegraded":"2024-04-25T01:14:34.190993+0000","last_fullsized":"2024-04-25T01:14:34.190993+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:57:53.486835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802216+0000","last_change":"2024-04-25T01:13:31.514491+0000","last_active":"2024-04-25T01:14:33.802216+0000","last_peered":"2024-04-25T01:14:33.802216+0000","last_clean":"2024-04-25T01:14:33.802216+0000","last_became_active":"2024-04-25T01:13:31.514156+0000","last_became_peered":"2024-04-25T01:13:31.514156+0000","last_unstale":"2024-04-25T01:14:33.802216+0000","last_undegraded":"2024-04-25T01:14:33.802216+0000","last_fullsized":"2024-04-25T01:14:33.802216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:18.588291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.801853+0000","last_change":"2024-04-25T01:13:30.521258+0000","last_active":"2024-04-25T01:14:33.801853+0000","last_peered":"2024-04-25T01:14:33.801853+0000","last_clean":"2024-04-25T01:14:33.801853+0000","last_became_active":"2024-04-25T01:13:30.518359+0000","last_became_peered":"2024-04-25T01:13:30.518359+0000","last_unstale":"2024-04-25T01:14:33.801853+0000","last_undegraded":"2024-04-25T01:14:33.801853+0000","last_fullsized":"2024-04-25T01:14:33.801853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:35.790798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.800260+0000","last_change":"2024-04-25T01:13:31.536755+0000","last_active":"2024-04-25T01:14:33.800260+0000","last_peered":"2024-04-25T01:14:33.800260+0000","last_clean":"2024-04-25T01:14:33.800260+0000","last_became_active":"2024-04-25T01:13:31.536303+0000","last_became_peered":"2024-04-25T01:13:31.536303+0000","last_unstale":"2024-04-25T01:14:33.800260+0000","last_undegraded":"2024-04-25T01:14:33.800260+0000","last_fullsized":"2024-04-25T01:14:33.800260+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:44.793993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.811323+0000","last_change":"2024-04-25T01:14:33.810882+0000","last_active":"2024-04-25T01:14:33.811323+0000","last_peered":"2024-04-25T01:14:33.811323+0000","last_clean":"2024-04-25T01:14:33.811323+0000","last_became_active":"2024-04-25T01:14:33.804758+0000","last_became_peered":"2024-04-25T01:14:33.804758+0000","last_unstale":"2024-04-25T01:14:33.811323+0000","last_undegraded":"2024-04-25T01:14:33.811323+0000","last_fullsized":"2024-04-25T01:14:33.811323+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:53.146681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802467+0000","last_change":"2024-04-25T01:13:31.543234+0000","last_active":"2024-04-25T01:14:33.802467+0000","last_peered":"2024-04-25T01:14:33.802467+0000","last_clean":"2024-04-25T01:14:33.802467+0000","last_became_active":"2024-04-25T01:13:31.542419+0000","last_became_peered":"2024-04-25T01:13:31.542419+0000","last_unstale":"2024-04-25T01:14:33.802467+0000","last_undegraded":"2024-04-25T01:14:33.802467+0000","last_fullsized":"2024-04-25T01:14:33.802467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:57.226505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.388517+0000","last_change":"2024-04-25T01:14:34.388045+0000","last_active":"2024-04-25T01:14:34.388517+0000","last_peered":"2024-04-25T01:14:34.388517+0000","last_clean":"2024-04-25T01:14:34.388517+0000","last_became_active":"2024-04-25T01:14:34.381047+0000","last_became_peered":"2024-04-25T01:14:34.381047+0000","last_unstale":"2024-04-25T01:14:34.388517+0000","last_undegraded":"2024-04-25T01:14:34.388517+0000","last_fullsized":"2024-04-25T01:14:34.388517+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:08.243523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.803146+0000","last_change":"2024-04-25T01:13:31.532194+0000","last_active":"2024-04-25T01:14:33.803146+0000","last_peered":"2024-04-25T01:14:33.803146+0000","last_clean":"2024-04-25T01:14:33.803146+0000","last_became_active":"2024-04-25T01:13:31.531798+0000","last_became_peered":"2024-04-25T01:13:31.531798+0000","last_unstale":"2024-04-25T01:14:33.803146+0000","last_undegraded":"2024-04-25T01:14:33.803146+0000","last_fullsized":"2024-04-25T01:14:33.803146+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:36.872827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.799123+0000","last_change":"2024-04-25T01:13:30.522565+0000","last_active":"2024-04-25T01:14:33.799123+0000","last_peered":"2024-04-25T01:14:33.799123+0000","last_clean":"2024-04-25T01:14:33.799123+0000","last_became_active":"2024-04-25T01:13:30.522207+0000","last_became_peered":"2024-04-25T01:13:30.522207+0000","last_unstale":"2024-04-25T01:14:33.799123+0000","last_undegraded":"2024-04-25T01:14:33.799123+0000","last_fullsized":"2024-04-25T01:14:33.799123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:48.263665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.803043+0000","last_change":"2024-04-25T01:13:31.514330+0000","last_active":"2024-04-25T01:14:33.803043+0000","last_peered":"2024-04-25T01:14:33.803043+0000","last_clean":"2024-04-25T01:14:33.803043+0000","last_became_active":"2024-04-25T01:13:31.513948+0000","last_became_peered":"2024-04-25T01:13:31.513948+0000","last_unstale":"2024-04-25T01:14:33.803043+0000","last_undegraded":"2024-04-25T01:14:33.803043+0000","last_fullsized":"2024-04-25T01:14:33.803043+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:57.544298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.379703+0000","last_change":"2024-04-25T01:13:30.515951+0000","last_active":"2024-04-25T01:14:34.379703+0000","last_peered":"2024-04-25T01:14:34.379703+0000","last_clean":"2024-04-25T01:14:34.379703+0000","last_became_active":"2024-04-25T01:13:30.515388+0000","last_became_peered":"2024-04-25T01:13:30.515388+0000","last_unstale":"2024-04-25T01:14:34.379703+0000","last_undegraded":"2024-04-25T01:14:34.379703+0000","last_fullsized":"2024-04-25T01:14:34.379703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:11.041466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802802+0000","last_change":"2024-04-25T01:13:31.503386+0000","last_active":"2024-04-25T01:14:33.802802+0000","last_peered":"2024-04-25T01:14:33.802802+0000","last_clean":"2024-04-25T01:14:33.802802+0000","last_became_active":"2024-04-25T01:13:31.502966+0000","last_became_peered":"2024-04-25T01:13:31.502966+0000","last_unstale":"2024-04-25T01:14:33.802802+0000","last_undegraded":"2024-04-25T01:14:33.802802+0000","last_fullsized":"2024-04-25T01:14:33.802802+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:07.520811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.812829+0000","last_change":"2024-04-25T01:13:30.528033+0000","last_active":"2024-04-25T01:14:33.812829+0000","last_peered":"2024-04-25T01:14:33.812829+0000","last_clean":"2024-04-25T01:14:33.812829+0000","last_became_active":"2024-04-25T01:13:30.527609+0000","last_became_peered":"2024-04-25T01:13:30.527609+0000","last_unstale":"2024-04-25T01:14:33.812829+0000","last_undegraded":"2024-04-25T01:14:33.812829+0000","last_fullsized":"2024-04-25T01:14:33.812829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:29.438458+0000","last_clean_scrub_stamp":"2024-04-25T01:13:29.438458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:08.182355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802984+0000","last_change":"2024-04-25T01:13:31.516569+0000","last_active":"2024-04-25T01:14:33.802984+0000","last_peered":"2024-04-25T01:14:33.802984+0000","last_clean":"2024-04-25T01:14:33.802984+0000","last_became_active":"2024-04-25T01:13:31.515746+0000","last_became_peered":"2024-04-25T01:13:31.515746+0000","last_unstale":"2024-04-25T01:14:33.802984+0000","last_undegraded":"2024-04-25T01:14:33.802984+0000","last_fullsized":"2024-04-25T01:14:33.802984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:27.860296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.809971+0000","last_change":"2024-04-25T01:14:33.809971+0000","last_active":"2024-04-25T01:14:33.809971+0000","last_peered":"2024-04-25T01:14:33.809971+0000","last_clean":"2024-04-25T01:14:33.809971+0000","last_became_active":"2024-04-25T01:14:33.809588+0000","last_became_peered":"2024-04-25T01:14:33.809588+0000","last_unstale":"2024-04-25T01:14:33.809971+0000","last_undegraded":"2024-04-25T01:14:33.809971+0000","last_fullsized":"2024-04-25T01:14:33.809971+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:49:24.492239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:15:02.093775+0000","last_change":"2024-04-25T01:13:31.514663+0000","last_active":"2024-04-25T01:15:02.093775+0000","last_peered":"2024-04-25T01:15:02.093775+0000","last_clean":"2024-04-25T01:15:02.093775+0000","last_became_active":"2024-04-25T01:13:31.514246+0000","last_became_peered":"2024-04-25T01:13:31.514246+0000","last_unstale":"2024-04-25T01:15:02.093775+0000","last_undegraded":"2024-04-25T01:15:02.093775+0000","last_fullsized":"2024-04-25T01:15:02.093775+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:27.235598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:15:02.104116+0000","last_change":"2024-04-25T01:13:31.533374+0000","last_active":"2024-04-25T01:15:02.104116+0000","last_peered":"2024-04-25T01:15:02.104116+0000","last_clean":"2024-04-25T01:15:02.104116+0000","last_became_active":"2024-04-25T01:13:31.533145+0000","last_became_peered":"2024-04-25T01:13:31.533145+0000","last_unstale":"2024-04-25T01:15:02.104116+0000","last_undegraded":"2024-04-25T01:15:02.104116+0000","last_fullsized":"2024-04-25T01:15:02.104116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:50.987823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.383168+0000","last_change":"2024-04-25T01:13:31.541682+0000","last_active":"2024-04-25T01:14:34.383168+0000","last_peered":"2024-04-25T01:14:34.383168+0000","last_clean":"2024-04-25T01:14:34.383168+0000","last_became_active":"2024-04-25T01:13:31.541257+0000","last_became_peered":"2024-04-25T01:13:31.541257+0000","last_unstale":"2024-04-25T01:14:34.383168+0000","last_undegraded":"2024-04-25T01:14:34.383168+0000","last_fullsized":"2024-04-25T01:14:34.383168+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:08.033895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.387542+0000","last_change":"2024-04-25T01:14:34.387542+0000","last_active":"2024-04-25T01:14:34.387542+0000","last_peered":"2024-04-25T01:14:34.387542+0000","last_clean":"2024-04-25T01:14:34.387542+0000","last_became_active":"2024-04-25T01:14:34.385838+0000","last_became_peered":"2024-04-25T01:14:34.385838+0000","last_unstale":"2024-04-25T01:14:34.387542+0000","last_undegraded":"2024-04-25T01:14:34.387542+0000","last_fullsized":"2024-04-25T01:14:34.387542+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:59.361382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.800447+0000","last_change":"2024-04-25T01:13:31.519068+0000","last_active":"2024-04-25T01:14:33.800447+0000","last_peered":"2024-04-25T01:14:33.800447+0000","last_clean":"2024-04-25T01:14:33.800447+0000","last_became_active":"2024-04-25T01:13:31.518705+0000","last_became_peered":"2024-04-25T01:13:31.518705+0000","last_unstale":"2024-04-25T01:14:33.800447+0000","last_undegraded":"2024-04-25T01:14:33.800447+0000","last_fullsized":"2024-04-25T01:14:33.800447+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:11.186149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:15:02.081929+0000","last_change":"2024-04-25T01:13:31.514572+0000","last_active":"2024-04-25T01:15:02.081929+0000","last_peered":"2024-04-25T01:15:02.081929+0000","last_clean":"2024-04-25T01:15:02.081929+0000","last_became_active":"2024-04-25T01:13:31.514178+0000","last_became_peered":"2024-04-25T01:13:31.514178+0000","last_unstale":"2024-04-25T01:15:02.081929+0000","last_undegraded":"2024-04-25T01:15:02.081929+0000","last_fullsized":"2024-04-25T01:15:02.081929+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:17.856802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.801510+0000","last_change":"2024-04-25T01:13:31.520837+0000","last_active":"2024-04-25T01:14:33.801510+0000","last_peered":"2024-04-25T01:14:33.801510+0000","last_clean":"2024-04-25T01:14:33.801510+0000","last_became_active":"2024-04-25T01:13:31.515601+0000","last_became_peered":"2024-04-25T01:13:31.515601+0000","last_unstale":"2024-04-25T01:14:33.801510+0000","last_undegraded":"2024-04-25T01:14:33.801510+0000","last_fullsized":"2024-04-25T01:14:33.801510+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:37.583156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:34.379883+0000","last_change":"2024-04-25T01:13:31.527420+0000","last_active":"2024-04-25T01:14:34.379883+0000","last_peered":"2024-04-25T01:14:34.379883+0000","last_clean":"2024-04-25T01:14:34.379883+0000","last_became_active":"2024-04-25T01:13:31.527160+0000","last_became_peered":"2024-04-25T01:13:31.527160+0000","last_unstale":"2024-04-25T01:14:34.379883+0000","last_undegraded":"2024-04-25T01:14:34.379883+0000","last_fullsized":"2024-04-25T01:14:34.379883+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:17:46.201486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.802154+0000","last_change":"2024-04-25T01:13:31.518356+0000","last_active":"2024-04-25T01:14:33.802154+0000","last_peered":"2024-04-25T01:14:33.802154+0000","last_clean":"2024-04-25T01:14:33.802154+0000","last_became_active":"2024-04-25T01:13:31.517654+0000","last_became_peered":"2024-04-25T01:13:31.517654+0000","last_unstale":"2024-04-25T01:14:33.802154+0000","last_undegraded":"2024-04-25T01:14:33.802154+0000","last_fullsized":"2024-04-25T01:14:33.802154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:17.254963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:15:02.106970+0000","last_change":"2024-04-25T01:13:31.526217+0000","last_active":"2024-04-25T01:15:02.106970+0000","last_peered":"2024-04-25T01:15:02.106970+0000","last_clean":"2024-04-25T01:15:02.106970+0000","last_became_active":"2024-04-25T01:13:31.525801+0000","last_became_peered":"2024-04-25T01:13:31.525801+0000","last_unstale":"2024-04-25T01:15:02.106970+0000","last_undegraded":"2024-04-25T01:15:02.106970+0000","last_fullsized":"2024-04-25T01:15:02.106970+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:48:30.244205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.814205+0000","last_change":"2024-04-25T01:13:31.543596+0000","last_active":"2024-04-25T01:14:33.814205+0000","last_peered":"2024-04-25T01:14:33.814205+0000","last_clean":"2024-04-25T01:14:33.814205+0000","last_became_active":"2024-04-25T01:13:31.542149+0000","last_became_peered":"2024-04-25T01:13:31.542149+0000","last_unstale":"2024-04-25T01:14:33.814205+0000","last_undegraded":"2024-04-25T01:14:33.814205+0000","last_fullsized":"2024-04-25T01:14:33.814205+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:05:50.392151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T01:14:33.800265+0000","last_change":"2024-04-25T01:13:31.513575+0000","last_active":"2024-04-25T01:14:33.800265+0000","last_peered":"2024-04-25T01:14:33.800265+0000","last_clean":"2024-04-25T01:14:33.800265+0000","last_became_active":"2024-04-25T01:13:31.513085+0000","last_became_peered":"2024-04-25T01:13:31.513085+0000","last_unstale":"2024-04-25T01:14:33.800265+0000","last_undegraded":"2024-04-25T01:14:33.800265+0000","last_fullsized":"2024-04-25T01:14:33.800265+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:13:30.482131+0000","last_clean_scrub_stamp":"2024-04-25T01:13:30.482131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:36.853033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63130,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":548,"num_read_kb":342,"num_write":163,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":189531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318306,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":462720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579943,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32792,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712360,"statfs":{"total":95995035648,"available":95961456640,"internally_reserved":0,"allocated":1654784,"data_stored":1113862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743468,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":462956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005106,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":462876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266743,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32716,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712436,"statfs":{"total":95995035648,"available":95961534464,"internally_reserved":0,"allocated":1568768,"data_stored":1054806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430269,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":462345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":521267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":463616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":522368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117068,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":978944,"data_stored":462656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855432,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":463173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691906,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1581056,"data_stored":1054099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:15:06.448 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T01:15:06.448 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T01:15:06.449 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T01:15:06.449 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T01:15:06.969 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:15:07.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:06 smithi169 ceph-mon[32759]: from='client.24917 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:15:07.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:06 smithi040 ceph-mon[32518]: from='client.24917 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:15:07.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:06 smithi031 ceph-mon[28053]: from='client.24917 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:15:08.197 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:07 smithi040 ceph-mon[32518]: pgmap v366: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T01:15:08.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:07 smithi031 ceph-mon[28053]: pgmap v366: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T01:15:08.444 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:07 smithi169 ceph-mon[32759]: pgmap v366: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T01:15:09.247 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:09 smithi031 ceph-mon[28053]: pgmap v367: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T01:15:09.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.434+0000 7f8010800700 1 -- 172.21.15.31:0/1352697245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f800810a540 msgr2=0x7f800810a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:09.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.434+0000 7f8010800700 1 --2- 172.21.15.31:0/1352697245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f800810a540 0x7f800810a920 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0098f0 tx=0x7f7ffc009c00 comp rx=0 tx=0).stop 2024-04-25T01:15:09.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.434+0000 7f8010800700 1 -- 172.21.15.31:0/1352697245 shutdown_connections 2024-04-25T01:15:09.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.434+0000 7f8010800700 1 --2- 172.21.15.31:0/1352697245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8008104e50 0x7f8008111310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:09.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.434+0000 7f8010800700 1 --2- 172.21.15.31:0/1352697245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f800810a540 0x7f800810a920 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:09.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.434+0000 7f8010800700 1 --2- 172.21.15.31:0/1352697245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8008104490 0x7f8008104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:09.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.434+0000 7f8010800700 1 -- 172.21.15.31:0/1352697245 >> 172.21.15.31:0/1352697245 conn(0x7f80080fff60 msgr2=0x7f8008102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:09.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.434+0000 7f8010800700 1 -- 172.21.15.31:0/1352697245 shutdown_connections 2024-04-25T01:15:09.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.434+0000 7f8010800700 1 -- 172.21.15.31:0/1352697245 wait complete. 2024-04-25T01:15:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.435+0000 7f8010800700 1 Processor -- start 2024-04-25T01:15:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.435+0000 7f8010800700 1 -- start start 2024-04-25T01:15:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.436+0000 7f8010800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8008104490 0x7f80081a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.436+0000 7f8010800700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8008104e50 0x7f80081a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.436+0000 7f8010800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800810a540 0x7f800819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.436+0000 7f8010800700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f80081147c0 con 0x7f800810a540 2024-04-25T01:15:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.436+0000 7f8010800700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8008114640 con 0x7f8008104e50 2024-04-25T01:15:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.436+0000 7f8010800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8008114940 con 0x7f8008104490 2024-04-25T01:15:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.436+0000 7f800e800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800810a540 0x7f800819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.436+0000 7f800e800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800810a540 0x7f800819aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45952/0 (socket says 172.21.15.31:45952) 2024-04-25T01:15:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.436+0000 7f800e800700 1 -- 172.21.15.31:0/229623706 learned_addr learned my addr 172.21.15.31:0/229623706 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.436+0000 7f800d400700 1 --2- 172.21.15.31:0/229623706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8008104e50 0x7f80081a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:09.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.436+0000 7f800de00700 1 --2- 172.21.15.31:0/229623706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8008104490 0x7f80081a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:09.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.436+0000 7f800e800700 1 -- 172.21.15.31:0/229623706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8008104490 msgr2=0x7f80081a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:09.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.436+0000 7f800e800700 1 --2- 172.21.15.31:0/229623706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8008104490 0x7f80081a08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:09.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.436+0000 7f800e800700 1 -- 172.21.15.31:0/229623706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8008104e50 msgr2=0x7f80081a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:09.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.436+0000 7f800e800700 1 --2- 172.21.15.31:0/229623706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8008104e50 0x7f80081a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:09.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.437+0000 7f800e800700 1 -- 172.21.15.31:0/229623706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800819b240 con 0x7f800810a540 2024-04-25T01:15:09.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.437+0000 7f800d400700 1 --2- 172.21.15.31:0/229623706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8008104e50 0x7f80081a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:09.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.437+0000 7f800e800700 1 --2- 172.21.15.31:0/229623706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800810a540 0x7f800819aae0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f800400c110 tx=0x7f800400c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:09.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.437+0000 7f8002a00700 1 -- 172.21.15.31:0/229623706 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80040092e0 con 0x7f800810a540 2024-04-25T01:15:09.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.437+0000 7f8002a00700 1 -- 172.21.15.31:0/229623706 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8004004bb0 con 0x7f800810a540 2024-04-25T01:15:09.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.437+0000 7f8010800700 1 -- 172.21.15.31:0/229623706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f800819b4d0 con 0x7f800810a540 2024-04-25T01:15:09.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.437+0000 7f8002a00700 1 -- 172.21.15.31:0/229623706 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004007d10 con 0x7f800810a540 2024-04-25T01:15:09.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.437+0000 7f8010800700 1 -- 172.21.15.31:0/229623706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80081a77a0 con 0x7f800810a540 2024-04-25T01:15:09.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.439+0000 7f8010800700 1 -- 172.21.15.31:0/229623706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f800810c9d0 con 0x7f800810a540 2024-04-25T01:15:09.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.441+0000 7f8002a00700 1 -- 172.21.15.31:0/229623706 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f800401a020 con 0x7f800810a540 2024-04-25T01:15:09.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.443+0000 7f8002a00700 1 --2- 172.21.15.31:0/229623706 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7ff00a2c20 0x7f7ff00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:09.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.443+0000 7f8002a00700 1 -- 172.21.15.31:0/229623706 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f80040ceac0 con 0x7f800810a540 2024-04-25T01:15:09.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.443+0000 7f800de00700 1 --2- 172.21.15.31:0/229623706 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7ff00a2c20 0x7f7ff00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:09.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.444+0000 7f800de00700 1 --2- 172.21.15.31:0/229623706 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7ff00a2c20 0x7f7ff00a50e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8008109670 tx=0x7f7ffc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:09.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.447+0000 7f8002a00700 1 -- 172.21.15.31:0/229623706 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8004010040 con 0x7f800810a540 2024-04-25T01:15:09.444 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:09 smithi169 ceph-mon[32759]: pgmap v367: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T01:15:09.446 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:09 smithi040 ceph-mon[32518]: pgmap v367: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T01:15:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.724+0000 7f8010800700 1 -- 172.21.15.31:0/229623706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f800819bfc0 con 0x7f800810a540 2024-04-25T01:15:09.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.725+0000 7f8002a00700 1 -- 172.21.15.31:0/229623706 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f800409ba10 con 0x7f800810a540 2024-04-25T01:15:09.711 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:15:09.711 INFO:teuthology.orchestra.run.smithi031.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T01:15:09.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.728+0000 7f8010800700 1 -- 172.21.15.31:0/229623706 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7ff00a2c20 msgr2=0x7f7ff00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:09.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.728+0000 7f8010800700 1 --2- 172.21.15.31:0/229623706 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7ff00a2c20 0x7f7ff00a50e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8008109670 tx=0x7f7ffc01a040 comp rx=0 tx=0).stop 2024-04-25T01:15:09.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.728+0000 7f8010800700 1 -- 172.21.15.31:0/229623706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800810a540 msgr2=0x7f800819aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:09.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.728+0000 7f8010800700 1 --2- 172.21.15.31:0/229623706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800810a540 0x7f800819aae0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f800400c110 tx=0x7f800400c420 comp rx=0 tx=0).stop 2024-04-25T01:15:09.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.728+0000 7f8010800700 1 -- 172.21.15.31:0/229623706 shutdown_connections 2024-04-25T01:15:09.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.728+0000 7f8010800700 1 --2- 172.21.15.31:0/229623706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800810a540 0x7f800819aae0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:09.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.728+0000 7f8010800700 1 --2- 172.21.15.31:0/229623706 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f7ff00a2c20 0x7f7ff00a50e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:09.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.728+0000 7f8010800700 1 --2- 172.21.15.31:0/229623706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8008104490 0x7f80081a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:09.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.728+0000 7f8010800700 1 --2- 172.21.15.31:0/229623706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8008104e50 0x7f80081a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:09.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.728+0000 7f8010800700 1 -- 172.21.15.31:0/229623706 >> 172.21.15.31:0/229623706 conn(0x7f80080fff60 msgr2=0x7f80081016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:09.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.728+0000 7f8010800700 1 -- 172.21.15.31:0/229623706 shutdown_connections 2024-04-25T01:15:09.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:09.728+0000 7f8010800700 1 -- 172.21.15.31:0/229623706 wait complete. 2024-04-25T01:15:10.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:10 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/229623706' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T01:15:10.443 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:10 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/229623706' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T01:15:10.447 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:10 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/229623706' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T01:15:10.566 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T01:15:10.567 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:15:10.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.805+0000 7f3755a00640 1 -- 172.21.15.31:0/2434993177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37501049b0 msgr2=0x7f3750104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:10.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.805+0000 7f3755a00640 1 --2- 172.21.15.31:0/2434993177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37501049b0 0x7f3750104e30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3738009900 tx=0x7f373802a3f0 comp rx=0 tx=0).stop 2024-04-25T01:15:10.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.805+0000 7f3755a00640 1 -- 172.21.15.31:0/2434993177 shutdown_connections 2024-04-25T01:15:10.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.805+0000 7f3755a00640 1 --2- 172.21.15.31:0/2434993177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3750105370 0x7f375010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:10.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.805+0000 7f3755a00640 1 --2- 172.21.15.31:0/2434993177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37501049b0 0x7f3750104e30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:10.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.805+0000 7f3755a00640 1 --2- 172.21.15.31:0/2434993177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37501037c0 0x7f3750103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:10.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.805+0000 7f3755a00640 1 -- 172.21.15.31:0/2434993177 >> 172.21.15.31:0/2434993177 conn(0x7f37500fed50 msgr2=0x7f3750101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:10.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.805+0000 7f3755a00640 1 -- 172.21.15.31:0/2434993177 shutdown_connections 2024-04-25T01:15:10.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.805+0000 7f3755a00640 1 -- 172.21.15.31:0/2434993177 wait complete. 2024-04-25T01:15:10.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.806+0000 7f3755a00640 1 Processor -- start 2024-04-25T01:15:10.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.806+0000 7f3755a00640 1 -- start start 2024-04-25T01:15:10.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.807+0000 7f3755a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37501037c0 0x7f3750199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.807+0000 7f3755a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37501049b0 0x7f375019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.807+0000 7f3755a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3750105370 0x7f37501a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.807+0000 7f3755a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f375010f470 con 0x7f37501037c0 2024-04-25T01:15:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.807+0000 7f3755a00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f375010f2f0 con 0x7f3750105370 2024-04-25T01:15:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.807+0000 7f3755a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f375010f5f0 con 0x7f37501049b0 2024-04-25T01:15:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.807+0000 7f374fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37501037c0 0x7f3750199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.807+0000 7f374fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37501037c0 0x7f3750199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45960/0 (socket says 172.21.15.31:45960) 2024-04-25T01:15:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.807+0000 7f374fe00640 1 -- 172.21.15.31:0/1585130760 learned_addr learned my addr 172.21.15.31:0/1585130760 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.807+0000 7f3747400640 1 --2- 172.21.15.31:0/1585130760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37501049b0 0x7f375019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.807+0000 7f3755000640 1 --2- 172.21.15.31:0/1585130760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3750105370 0x7f37501a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.808+0000 7f374fe00640 1 -- 172.21.15.31:0/1585130760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37501049b0 msgr2=0x7f375019a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.808+0000 7f374fe00640 1 --2- 172.21.15.31:0/1585130760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37501049b0 0x7f375019a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:10.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.808+0000 7f374fe00640 1 -- 172.21.15.31:0/1585130760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3750105370 msgr2=0x7f37501a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:10.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.808+0000 7f374fe00640 1 --2- 172.21.15.31:0/1585130760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3750105370 0x7f37501a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:10.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.808+0000 7f374fe00640 1 -- 172.21.15.31:0/1585130760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37501a1700 con 0x7f37501037c0 2024-04-25T01:15:10.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.808+0000 7f374fe00640 1 --2- 172.21.15.31:0/1585130760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37501037c0 0x7f3750199ae0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3740009d20 tx=0x7f374000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:10.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.808+0000 7f374d600640 1 -- 172.21.15.31:0/1585130760 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740010630 con 0x7f37501037c0 2024-04-25T01:15:10.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.808+0000 7f374d600640 1 -- 172.21.15.31:0/1585130760 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3740010cb0 con 0x7f37501037c0 2024-04-25T01:15:10.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.808+0000 7f3755a00640 1 -- 172.21.15.31:0/1585130760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37501a19f0 con 0x7f37501037c0 2024-04-25T01:15:10.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.808+0000 7f374d600640 1 -- 172.21.15.31:0/1585130760 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740018ce0 con 0x7f37501037c0 2024-04-25T01:15:10.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.808+0000 7f3755a00640 1 -- 172.21.15.31:0/1585130760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37501a1f60 con 0x7f37501037c0 2024-04-25T01:15:10.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.809+0000 7f3755a00640 1 -- 172.21.15.31:0/1585130760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f375010aae0 con 0x7f37501037c0 2024-04-25T01:15:10.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.812+0000 7f374d600640 1 -- 172.21.15.31:0/1585130760 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f374001f020 con 0x7f37501037c0 2024-04-25T01:15:10.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.813+0000 7f374d600640 1 --2- 172.21.15.31:0/1585130760 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f37340a2c10 0x7f37340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:10.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.813+0000 7f374d600640 1 -- 172.21.15.31:0/1585130760 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f37400cf380 con 0x7f37501037c0 2024-04-25T01:15:10.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.813+0000 7f3747400640 1 --2- 172.21.15.31:0/1585130760 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f37340a2c10 0x7f37340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:10.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.814+0000 7f3747400640 1 --2- 172.21.15.31:0/1585130760 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f37340a2c10 0x7f37340a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3738031440 tx=0x7f37380315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:10.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:10.817+0000 7f374d600640 1 -- 172.21.15.31:0/1585130760 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f374009c2d0 con 0x7f37501037c0 2024-04-25T01:15:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.026+0000 7f3755a00640 1 -- 172.21.15.31:0/1585130760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f375002d170 con 0x7f37501037c0 2024-04-25T01:15:11.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.033+0000 7f374d600640 1 -- 172.21.15.31:0/1585130760 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22530 (secure 0 0 0) 0x7f374009bbd0 con 0x7f37501037c0 2024-04-25T01:15:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:15:11.019 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":89,"fsid":"95849b94-029f-11ef-bc93-c7b262605968","created":"2024-04-25T01:04:23.310856+0000","modified":"2024-04-25T01:14:33.793638+0000","last_up_change":"2024-04-25T01:12:29.393907+0000","last_in_change":"2024-04-25T01:12:08.978148+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:18.662391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:13:29.258771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:13:30.480996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa08e4c-fe6b-4116-bdf5-1adf5b465d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6803","nonce":788174091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6805","nonce":788174091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6809","nonce":788174091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6807","nonce":788174091}]},"public_addr":"172.21.15.31:6803/788174091","cluster_addr":"172.21.15.31:6805/788174091","heartbeat_back_addr":"172.21.15.31:6809/788174091","heartbeat_front_addr":"172.21.15.31:6807/788174091","state":["exists","up"]},{"osd":1,"uuid":"85339fe7-0dbc-4485-9334-2b039993c1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6801","nonce":3196394509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6803","nonce":3196394509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6807","nonce":3196394509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6805","nonce":3196394509}]},"public_addr":"172.21.15.40:6801/3196394509","cluster_addr":"172.21.15.40:6803/3196394509","heartbeat_back_addr":"172.21.15.40:6807/3196394509","heartbeat_front_addr":"172.21.15.40:6805/3196394509","state":["exists","up"]},{"osd":2,"uuid":"ab62ecb2-8aae-4987-9e2e-589e0f2fee5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6801","nonce":1955211197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6803","nonce":1955211197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6807","nonce":1955211197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6805","nonce":1955211197}]},"public_addr":"172.21.15.169:6801/1955211197","cluster_addr":"172.21.15.169:6803/1955211197","heartbeat_back_addr":"172.21.15.169:6807/1955211197","heartbeat_front_addr":"172.21.15.169:6805/1955211197","state":["exists","up"]},{"osd":3,"uuid":"8b5a7881-3401-495c-8ca4-c90685919d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6811","nonce":1228688643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6813","nonce":1228688643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6817","nonce":1228688643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6815","nonce":1228688643}]},"public_addr":"172.21.15.31:6811/1228688643","cluster_addr":"172.21.15.31:6813/1228688643","heartbeat_back_addr":"172.21.15.31:6817/1228688643","heartbeat_front_addr":"172.21.15.31:6815/1228688643","state":["exists","up"]},{"osd":4,"uuid":"56deb61d-a334-4cf2-8ac7-fbb6e6947092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6809","nonce":1913779091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6811","nonce":1913779091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6815","nonce":1913779091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6813","nonce":1913779091}]},"public_addr":"172.21.15.40:6809/1913779091","cluster_addr":"172.21.15.40:6811/1913779091","heartbeat_back_addr":"172.21.15.40:6815/1913779091","heartbeat_front_addr":"172.21.15.40:6813/1913779091","state":["exists","up"]},{"osd":5,"uuid":"8da55693-f9c7-49a7-99e0-165078f8001b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6809","nonce":2549650781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6811","nonce":2549650781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6815","nonce":2549650781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6813","nonce":2549650781}]},"public_addr":"172.21.15.169:6809/2549650781","cluster_addr":"172.21.15.169:6811/2549650781","heartbeat_back_addr":"172.21.15.169:6815/2549650781","heartbeat_front_addr":"172.21.15.169:6813/2549650781","state":["exists","up"]},{"osd":6,"uuid":"fc8350d2-1ad6-43cb-89f4-d336afc99ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6819","nonce":611507624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6821","nonce":611507624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6825","nonce":611507624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6823","nonce":611507624}]},"public_addr":"172.21.15.31:6819/611507624","cluster_addr":"172.21.15.31:6821/611507624","heartbeat_back_addr":"172.21.15.31:6825/611507624","heartbeat_front_addr":"172.21.15.31:6823/611507624","state":["exists","up"]},{"osd":7,"uuid":"3121b58b-8c94-40b5-9d48-f56552d7c39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6817","nonce":3776085849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6819","nonce":3776085849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6823","nonce":3776085849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6821","nonce":3776085849}]},"public_addr":"172.21.15.40:6817/3776085849","cluster_addr":"172.21.15.40:6819/3776085849","heartbeat_back_addr":"172.21.15.40:6823/3776085849","heartbeat_front_addr":"172.21.15.40:6821/3776085849","state":["exists","up"]},{"osd":8,"uuid":"f60407df-91f5-477a-9702-ddaef24faf82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6817","nonce":2736546122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6819","nonce":2736546122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6823","nonce":2736546122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6821","nonce":2736546122}]},"public_addr":"172.21.15.169:6817/2736546122","cluster_addr":"172.21.15.169:6819/2736546122","heartbeat_back_addr":"172.21.15.169:6823/2736546122","heartbeat_front_addr":"172.21.15.169:6821/2736546122","state":["exists","up"]},{"osd":9,"uuid":"7506a407-1960-49bc-9e38-7f2a8bf996ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6827","nonce":1187207880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6829","nonce":1187207880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6833","nonce":1187207880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6831","nonce":1187207880}]},"public_addr":"172.21.15.31:6827/1187207880","cluster_addr":"172.21.15.31:6829/1187207880","heartbeat_back_addr":"172.21.15.31:6833/1187207880","heartbeat_front_addr":"172.21.15.31:6831/1187207880","state":["exists","up"]},{"osd":10,"uuid":"831c48e2-b0d0-451f-959b-228e927542cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6825","nonce":3050148001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6827","nonce":3050148001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6831","nonce":3050148001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6829","nonce":3050148001}]},"public_addr":"172.21.15.40:6825/3050148001","cluster_addr":"172.21.15.40:6827/3050148001","heartbeat_back_addr":"172.21.15.40:6831/3050148001","heartbeat_front_addr":"172.21.15.40:6829/3050148001","state":["exists","up"]},{"osd":11,"uuid":"144fe0aa-f757-4b3c-90ad-99775e17d57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6825","nonce":262401944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6827","nonce":262401944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6831","nonce":262401944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6829","nonce":262401944}]},"public_addr":"172.21.15.169:6825/262401944","cluster_addr":"172.21.15.169:6827/262401944","heartbeat_back_addr":"172.21.15.169:6831/262401944","heartbeat_front_addr":"172.21.15.169:6829/262401944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:25.640995+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:51.036909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.635633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:48.467912+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:14.454971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:37.609315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:12.441807+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:38.239215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:02.230758+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:35.762553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:02.051405+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:26.569593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6801/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4004014066":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/1930101646":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/2638309002":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2238638505":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2902841144":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4190696323":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/480558521":"2024-04-26T01:04:57.149272+0000","172.21.15.31:6801/2638309002":"2024-04-26T01:04:57.149272+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:15:11.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.035+0000 7f3755a00640 1 -- 172.21.15.31:0/1585130760 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f37340a2c10 msgr2=0x7f37340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:11.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.035+0000 7f3755a00640 1 --2- 172.21.15.31:0/1585130760 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f37340a2c10 0x7f37340a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3738031440 tx=0x7f37380315b0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.036+0000 7f3755a00640 1 -- 172.21.15.31:0/1585130760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37501037c0 msgr2=0x7f3750199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:11.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.036+0000 7f3755a00640 1 --2- 172.21.15.31:0/1585130760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37501037c0 0x7f3750199ae0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3740009d20 tx=0x7f374000c6a0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.036+0000 7f3755a00640 1 -- 172.21.15.31:0/1585130760 shutdown_connections 2024-04-25T01:15:11.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.036+0000 7f3755a00640 1 --2- 172.21.15.31:0/1585130760 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f37340a2c10 0x7f37340a50d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.036+0000 7f3755a00640 1 --2- 172.21.15.31:0/1585130760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3750105370 0x7f37501a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.036+0000 7f3755a00640 1 --2- 172.21.15.31:0/1585130760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37501049b0 0x7f375019a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.036+0000 7f3755a00640 1 --2- 172.21.15.31:0/1585130760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37501037c0 0x7f3750199ae0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.036+0000 7f3755a00640 1 -- 172.21.15.31:0/1585130760 >> 172.21.15.31:0/1585130760 conn(0x7f37500fed50 msgr2=0x7f3750107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:11.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.036+0000 7f3755a00640 1 -- 172.21.15.31:0/1585130760 shutdown_connections 2024-04-25T01:15:11.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.036+0000 7f3755a00640 1 -- 172.21.15.31:0/1585130760 wait complete. 2024-04-25T01:15:11.043 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:18.662391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:13:29.258771+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:13:30.480996+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:15:11.044 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:15:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.278+0000 7fd558e00640 1 -- 172.21.15.31:0/594033751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd554103280 msgr2=0x7fd554109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.278+0000 7fd558e00640 1 --2- 172.21.15.31:0/594033751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd554103280 0x7fd554109b10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009900 tx=0x7fd54c02a3f0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.278+0000 7fd558e00640 1 -- 172.21.15.31:0/594033751 shutdown_connections 2024-04-25T01:15:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.278+0000 7fd558e00640 1 --2- 172.21.15.31:0/594033751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd554103280 0x7fd554109b10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.278+0000 7fd558e00640 1 --2- 172.21.15.31:0/594033751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5541028c0 0x7fd554102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.278+0000 7fd558e00640 1 --2- 172.21.15.31:0/594033751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5541016d0 0x7fd554101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.278+0000 7fd558e00640 1 -- 172.21.15.31:0/594033751 >> 172.21.15.31:0/594033751 conn(0x7fd5540fcc40 msgr2=0x7fd5540ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.278+0000 7fd558e00640 1 -- 172.21.15.31:0/594033751 shutdown_connections 2024-04-25T01:15:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.278+0000 7fd558e00640 1 -- 172.21.15.31:0/594033751 wait complete. 2024-04-25T01:15:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.279+0000 7fd558e00640 1 Processor -- start 2024-04-25T01:15:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.280+0000 7fd558e00640 1 -- start start 2024-04-25T01:15:11.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.280+0000 7fd558e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5541016d0 0x7fd554199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:11.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.280+0000 7fd558e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5541028c0 0x7fd55419a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.280+0000 7fd558e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd554103280 0x7fd5541a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.280+0000 7fd558e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd55410d2c0 con 0x7fd554103280 2024-04-25T01:15:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.280+0000 7fd558e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd55410d140 con 0x7fd5541028c0 2024-04-25T01:15:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.280+0000 7fd558e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd55410d440 con 0x7fd5541016d0 2024-04-25T01:15:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.280+0000 7fd553e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd554103280 0x7fd5541a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.280+0000 7fd553e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd554103280 0x7fd5541a10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45978/0 (socket says 172.21.15.31:45978) 2024-04-25T01:15:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.280+0000 7fd553e00640 1 -- 172.21.15.31:0/2139344585 learned_addr learned my addr 172.21.15.31:0/2139344585 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.280+0000 7fd552a00640 1 --2- 172.21.15.31:0/2139344585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5541028c0 0x7fd55419a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.280+0000 7fd553400640 1 --2- 172.21.15.31:0/2139344585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5541016d0 0x7fd554199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.281+0000 7fd553e00640 1 -- 172.21.15.31:0/2139344585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5541016d0 msgr2=0x7fd554199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.281+0000 7fd553e00640 1 --2- 172.21.15.31:0/2139344585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5541016d0 0x7fd554199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.281+0000 7fd553e00640 1 -- 172.21.15.31:0/2139344585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5541028c0 msgr2=0x7fd55419a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.281+0000 7fd553e00640 1 --2- 172.21.15.31:0/2139344585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5541028c0 0x7fd55419a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.281+0000 7fd553e00640 1 -- 172.21.15.31:0/2139344585 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5541a1730 con 0x7fd554103280 2024-04-25T01:15:11.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.281+0000 7fd553400640 1 --2- 172.21.15.31:0/2139344585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5541016d0 0x7fd554199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:11.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.281+0000 7fd553e00640 1 --2- 172.21.15.31:0/2139344585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd554103280 0x7fd5541a10c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0314e0 tx=0x7fd54c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:11.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.281+0000 7fd54be00640 1 -- 172.21.15.31:0/2139344585 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c03f030 con 0x7fd554103280 2024-04-25T01:15:11.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.281+0000 7fd54be00640 1 -- 172.21.15.31:0/2139344585 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd54c003ce0 con 0x7fd554103280 2024-04-25T01:15:11.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.281+0000 7fd54be00640 1 -- 172.21.15.31:0/2139344585 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c035070 con 0x7fd554103280 2024-04-25T01:15:11.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.282+0000 7fd558e00640 1 -- 172.21.15.31:0/2139344585 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5541a19c0 con 0x7fd554103280 2024-04-25T01:15:11.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.282+0000 7fd558e00640 1 -- 172.21.15.31:0/2139344585 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5541a1f00 con 0x7fd554103280 2024-04-25T01:15:11.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.283+0000 7fd558e00640 1 -- 172.21.15.31:0/2139344585 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5541a2130 con 0x7fd554103280 2024-04-25T01:15:11.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.285+0000 7fd54be00640 1 -- 172.21.15.31:0/2139344585 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd54c042030 con 0x7fd554103280 2024-04-25T01:15:11.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.287+0000 7fd54be00640 1 --2- 172.21.15.31:0/2139344585 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd5380a2c10 0x7fd5380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:11.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.287+0000 7fd54be00640 1 -- 172.21.15.31:0/2139344585 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fd54c0ef3d0 con 0x7fd554103280 2024-04-25T01:15:11.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.287+0000 7fd553400640 1 --2- 172.21.15.31:0/2139344585 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd5380a2c10 0x7fd5380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:11.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.288+0000 7fd553400640 1 --2- 172.21.15.31:0/2139344585 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd5380a2c10 0x7fd5380a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd554102720 tx=0x7fd544009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:11.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.291+0000 7fd54be00640 1 -- 172.21.15.31:0/2139344585 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd54c0bbcd0 con 0x7fd554103280 2024-04-25T01:15:11.444 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:11 smithi169 ceph-mon[32759]: pgmap v368: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T01:15:11.447 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:11 smithi040 ceph-mon[32518]: pgmap v368: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T01:15:11.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.495+0000 7fd558e00640 1 -- 172.21.15.31:0/2139344585 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd5541a24c0 con 0x7fd554103280 2024-04-25T01:15:11.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.495+0000 7fd54be00640 1 -- 172.21.15.31:0/2139344585 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fd54c0bffd0 con 0x7fd554103280 2024-04-25T01:15:11.481 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T01:15:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.498+0000 7fd558e00640 1 -- 172.21.15.31:0/2139344585 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd5380a2c10 msgr2=0x7fd5380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.498+0000 7fd558e00640 1 --2- 172.21.15.31:0/2139344585 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd5380a2c10 0x7fd5380a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd554102720 tx=0x7fd544009500 comp rx=0 tx=0).stop 2024-04-25T01:15:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.498+0000 7fd558e00640 1 -- 172.21.15.31:0/2139344585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd554103280 msgr2=0x7fd5541a10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.498+0000 7fd558e00640 1 --2- 172.21.15.31:0/2139344585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd554103280 0x7fd5541a10c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0314e0 tx=0x7fd54c0042f0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.499+0000 7fd558e00640 1 -- 172.21.15.31:0/2139344585 shutdown_connections 2024-04-25T01:15:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.499+0000 7fd558e00640 1 --2- 172.21.15.31:0/2139344585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd554103280 0x7fd5541a10c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.499+0000 7fd558e00640 1 --2- 172.21.15.31:0/2139344585 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd5380a2c10 0x7fd5380a50d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.499+0000 7fd558e00640 1 --2- 172.21.15.31:0/2139344585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5541028c0 0x7fd55419a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.499+0000 7fd558e00640 1 --2- 172.21.15.31:0/2139344585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5541016d0 0x7fd554199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.499+0000 7fd558e00640 1 -- 172.21.15.31:0/2139344585 >> 172.21.15.31:0/2139344585 conn(0x7fd5540fcc40 msgr2=0x7fd554073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.499+0000 7fd558e00640 1 -- 172.21.15.31:0/2139344585 shutdown_connections 2024-04-25T01:15:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.499+0000 7fd558e00640 1 -- 172.21.15.31:0/2139344585 wait complete. 2024-04-25T01:15:11.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:11 smithi031 ceph-mon[28053]: pgmap v368: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T01:15:11.505 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:15:11.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.737+0000 7f03baa00640 1 -- 172.21.15.31:0/4229541012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03b4105430 msgr2=0x7f03b410bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:11.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.737+0000 7f03baa00640 1 --2- 172.21.15.31:0/4229541012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03b4105430 0x7f03b410bcc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f03ac009860 tx=0x7f03ac02a340 comp rx=0 tx=0).stop 2024-04-25T01:15:11.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.737+0000 7f03baa00640 1 -- 172.21.15.31:0/4229541012 shutdown_connections 2024-04-25T01:15:11.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.737+0000 7f03baa00640 1 --2- 172.21.15.31:0/4229541012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03b4105430 0x7f03b410bcc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.737+0000 7f03baa00640 1 --2- 172.21.15.31:0/4229541012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03b4104a70 0x7f03b4104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.737+0000 7f03baa00640 1 --2- 172.21.15.31:0/4229541012 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03b4103880 0x7f03b4103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.737+0000 7f03baa00640 1 -- 172.21.15.31:0/4229541012 >> 172.21.15.31:0/4229541012 conn(0x7f03b40fedf0 msgr2=0x7f03b4101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:11.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.737+0000 7f03baa00640 1 -- 172.21.15.31:0/4229541012 shutdown_connections 2024-04-25T01:15:11.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.738+0000 7f03baa00640 1 -- 172.21.15.31:0/4229541012 wait complete. 2024-04-25T01:15:11.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.738+0000 7f03baa00640 1 Processor -- start 2024-04-25T01:15:11.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.739+0000 7f03baa00640 1 -- start start 2024-04-25T01:15:11.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.739+0000 7f03baa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03b4103880 0x7f03b4199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:11.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.739+0000 7f03baa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03b4104a70 0x7f03b419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:11.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.739+0000 7f03baa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03b4105430 0x7f03b41a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:11.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.739+0000 7f03baa00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f03b410f4e0 con 0x7f03b4103880 2024-04-25T01:15:11.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.739+0000 7f03baa00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f03b410f360 con 0x7f03b4104a70 2024-04-25T01:15:11.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.739+0000 7f03baa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f03b410f660 con 0x7f03b4105430 2024-04-25T01:15:11.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.739+0000 7f03b9600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03b4103880 0x7f03b4199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:11.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.739+0000 7f03b9600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03b4103880 0x7f03b4199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45998/0 (socket says 172.21.15.31:45998) 2024-04-25T01:15:11.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.739+0000 7f03b9600640 1 -- 172.21.15.31:0/1761214997 learned_addr learned my addr 172.21.15.31:0/1761214997 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:11.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.739+0000 7f03ba000640 1 --2- 172.21.15.31:0/1761214997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03b4105430 0x7f03b41a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:11.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.739+0000 7f03b8c00640 1 --2- 172.21.15.31:0/1761214997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03b4104a70 0x7f03b419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:11.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.740+0000 7f03ba000640 1 -- 172.21.15.31:0/1761214997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03b4104a70 msgr2=0x7f03b419a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:11.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.740+0000 7f03ba000640 1 --2- 172.21.15.31:0/1761214997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03b4104a70 0x7f03b419a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.740+0000 7f03ba000640 1 -- 172.21.15.31:0/1761214997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03b4103880 msgr2=0x7f03b4199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:11.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.740+0000 7f03ba000640 1 --2- 172.21.15.31:0/1761214997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03b4103880 0x7f03b4199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.740+0000 7f03ba000640 1 -- 172.21.15.31:0/1761214997 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b41a1710 con 0x7f03b4105430 2024-04-25T01:15:11.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.740+0000 7f03b9600640 1 --2- 172.21.15.31:0/1761214997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03b4103880 0x7f03b4199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:11.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.740+0000 7f03b8c00640 1 --2- 172.21.15.31:0/1761214997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03b4104a70 0x7f03b419a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:11.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.740+0000 7f03ba000640 1 --2- 172.21.15.31:0/1761214997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03b4105430 0x7f03b41a10a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f03ac02a860 tx=0x7f03ac033d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:11.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.740+0000 7f03b2000640 1 -- 172.21.15.31:0/1761214997 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac037040 con 0x7f03b4105430 2024-04-25T01:15:11.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.740+0000 7f03baa00640 1 -- 172.21.15.31:0/1761214997 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b41a19a0 con 0x7f03b4105430 2024-04-25T01:15:11.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.740+0000 7f03b2000640 1 -- 172.21.15.31:0/1761214997 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03ac004050 con 0x7f03b4105430 2024-04-25T01:15:11.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.740+0000 7f03b2000640 1 -- 172.21.15.31:0/1761214997 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac003a50 con 0x7f03b4105430 2024-04-25T01:15:11.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.741+0000 7f03baa00640 1 -- 172.21.15.31:0/1761214997 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b41a1eb0 con 0x7f03b4105430 2024-04-25T01:15:11.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.742+0000 7f03baa00640 1 -- 172.21.15.31:0/1761214997 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03b410aba0 con 0x7f03b4105430 2024-04-25T01:15:11.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.744+0000 7f03b2000640 1 -- 172.21.15.31:0/1761214997 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f03ac06bd10 con 0x7f03b4105430 2024-04-25T01:15:11.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.745+0000 7f03b2000640 1 --2- 172.21.15.31:0/1761214997 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f039c0a2c10 0x7f039c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:11.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.745+0000 7f03b2000640 1 -- 172.21.15.31:0/1761214997 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f03ac0ee9d0 con 0x7f03b4105430 2024-04-25T01:15:11.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.745+0000 7f03b9600640 1 --2- 172.21.15.31:0/1761214997 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f039c0a2c10 0x7f039c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:11.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.746+0000 7f03b9600640 1 --2- 172.21.15.31:0/1761214997 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f039c0a2c10 0x7f039c0a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f03a800f7b0 tx=0x7f03a8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:11.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.748+0000 7f03b2000640 1 -- 172.21.15.31:0/1761214997 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f03ac049020 con 0x7f03b4105430 2024-04-25T01:15:11.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.951+0000 7f03baa00640 1 -- 172.21.15.31:0/1761214997 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f03b402d300 con 0x7f03b4105430 2024-04-25T01:15:11.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.952+0000 7f03b2000640 1 -- 172.21.15.31:0/1761214997 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f03ac0bb920 con 0x7f03b4105430 2024-04-25T01:15:11.938 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-25T01:15:11.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.954+0000 7f03baa00640 1 -- 172.21.15.31:0/1761214997 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f039c0a2c10 msgr2=0x7f039c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:11.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.954+0000 7f03baa00640 1 --2- 172.21.15.31:0/1761214997 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f039c0a2c10 0x7f039c0a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f03a800f7b0 tx=0x7f03a8009500 comp rx=0 tx=0).stop 2024-04-25T01:15:11.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.954+0000 7f03baa00640 1 -- 172.21.15.31:0/1761214997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03b4105430 msgr2=0x7f03b41a10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:11.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.954+0000 7f03baa00640 1 --2- 172.21.15.31:0/1761214997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03b4105430 0x7f03b41a10a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f03ac02a860 tx=0x7f03ac033d10 comp rx=0 tx=0).stop 2024-04-25T01:15:11.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.954+0000 7f03baa00640 1 -- 172.21.15.31:0/1761214997 shutdown_connections 2024-04-25T01:15:11.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.954+0000 7f03baa00640 1 --2- 172.21.15.31:0/1761214997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03b4105430 0x7f03b41a10a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.954+0000 7f03baa00640 1 --2- 172.21.15.31:0/1761214997 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f039c0a2c10 0x7f039c0a50d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.954+0000 7f03baa00640 1 --2- 172.21.15.31:0/1761214997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03b4104a70 0x7f03b419a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.954+0000 7f03baa00640 1 --2- 172.21.15.31:0/1761214997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03b4103880 0x7f03b4199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:11.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.954+0000 7f03baa00640 1 -- 172.21.15.31:0/1761214997 >> 172.21.15.31:0/1761214997 conn(0x7f03b40fedf0 msgr2=0x7f03b4107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:11.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.955+0000 7f03baa00640 1 -- 172.21.15.31:0/1761214997 shutdown_connections 2024-04-25T01:15:11.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:11.955+0000 7f03baa00640 1 -- 172.21.15.31:0/1761214997 wait complete. 2024-04-25T01:15:11.960 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:15:12.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.188+0000 7fc56fe00640 1 -- 172.21.15.31:0/3952694515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5701020e0 msgr2=0x7fc5701044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:12.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.188+0000 7fc56fe00640 1 --2- 172.21.15.31:0/3952694515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5701020e0 0x7fc5701044d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc560009900 tx=0x7fc56002a3f0 comp rx=0 tx=0).stop 2024-04-25T01:15:12.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.188+0000 7fc56fe00640 1 -- 172.21.15.31:0/3952694515 shutdown_connections 2024-04-25T01:15:12.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.188+0000 7fc56fe00640 1 --2- 172.21.15.31:0/3952694515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc570107340 0x7fc570109790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:12.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.188+0000 7fc56fe00640 1 --2- 172.21.15.31:0/3952694515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc570104a10 0x7fc570106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:12.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.188+0000 7fc56fe00640 1 --2- 172.21.15.31:0/3952694515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5701020e0 0x7fc5701044d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:12.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.188+0000 7fc56fe00640 1 -- 172.21.15.31:0/3952694515 >> 172.21.15.31:0/3952694515 conn(0x7fc5700fbaf0 msgr2=0x7fc5700fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:12.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.189+0000 7fc56fe00640 1 -- 172.21.15.31:0/3952694515 shutdown_connections 2024-04-25T01:15:12.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.189+0000 7fc56fe00640 1 -- 172.21.15.31:0/3952694515 wait complete. 2024-04-25T01:15:12.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.189+0000 7fc56fe00640 1 Processor -- start 2024-04-25T01:15:12.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.190+0000 7fc56fe00640 1 -- start start 2024-04-25T01:15:12.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.190+0000 7fc56fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5701020e0 0x7fc570199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:12.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.190+0000 7fc56fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc570104a10 0x7fc57019a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:12.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.190+0000 7fc56fe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc570107340 0x7fc5701a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:12.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.190+0000 7fc56fe00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc57010d2f0 con 0x7fc570104a10 2024-04-25T01:15:12.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.190+0000 7fc56fe00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc57010d170 con 0x7fc570107340 2024-04-25T01:15:12.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.190+0000 7fc56fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc57010d470 con 0x7fc5701020e0 2024-04-25T01:15:12.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.190+0000 7fc567e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc570104a10 0x7fc57019a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:12.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.190+0000 7fc567e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc570104a10 0x7fc57019a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46026/0 (socket says 172.21.15.31:46026) 2024-04-25T01:15:12.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.190+0000 7fc567e00640 1 -- 172.21.15.31:0/2745127556 learned_addr learned my addr 172.21.15.31:0/2745127556 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:12.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.190+0000 7fc56f400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc570107340 0x7fc5701a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:12.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.190+0000 7fc56ea00640 1 --2- 172.21.15.31:0/2745127556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5701020e0 0x7fc570199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:12.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.191+0000 7fc567e00640 1 -- 172.21.15.31:0/2745127556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5701020e0 msgr2=0x7fc570199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:12.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.191+0000 7fc567e00640 1 --2- 172.21.15.31:0/2745127556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5701020e0 0x7fc570199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:12.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.191+0000 7fc567e00640 1 -- 172.21.15.31:0/2745127556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc570107340 msgr2=0x7fc5701a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:12.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.191+0000 7fc567e00640 1 --2- 172.21.15.31:0/2745127556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc570107340 0x7fc5701a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:12.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.191+0000 7fc567e00640 1 -- 172.21.15.31:0/2745127556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5701a1800 con 0x7fc570104a10 2024-04-25T01:15:12.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.191+0000 7fc56f400640 1 --2- 172.21.15.31:0/2745127556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc570107340 0x7fc5701a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:12.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.191+0000 7fc567e00640 1 --2- 172.21.15.31:0/2745127556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc570104a10 0x7fc57019a120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc55800e9f0 tx=0x7fc55800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:12.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.191+0000 7fc567400640 1 -- 172.21.15.31:0/2745127556 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55800ccc0 con 0x7fc570104a10 2024-04-25T01:15:12.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.191+0000 7fc56fe00640 1 -- 172.21.15.31:0/2745127556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5701a1af0 con 0x7fc570104a10 2024-04-25T01:15:12.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.191+0000 7fc567400640 1 -- 172.21.15.31:0/2745127556 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc558005500 con 0x7fc570104a10 2024-04-25T01:15:12.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.191+0000 7fc567400640 1 -- 172.21.15.31:0/2745127556 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc558010690 con 0x7fc570104a10 2024-04-25T01:15:12.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.191+0000 7fc56fe00640 1 -- 172.21.15.31:0/2745127556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5701a1ff0 con 0x7fc570104a10 2024-04-25T01:15:12.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.192+0000 7fc56fe00640 1 -- 172.21.15.31:0/2745127556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc570108e90 con 0x7fc570104a10 2024-04-25T01:15:12.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.194+0000 7fc567400640 1 -- 172.21.15.31:0/2745127556 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc5580046c0 con 0x7fc570104a10 2024-04-25T01:15:12.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.196+0000 7fc567400640 1 --2- 172.21.15.31:0/2745127556 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc5540a2b90 0x7fc5540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:12.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.196+0000 7fc567400640 1 -- 172.21.15.31:0/2745127556 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fc5580cf3b0 con 0x7fc570104a10 2024-04-25T01:15:12.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.196+0000 7fc56ea00640 1 --2- 172.21.15.31:0/2745127556 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc5540a2b90 0x7fc5540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:12.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.197+0000 7fc56ea00640 1 --2- 172.21.15.31:0/2745127556 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc5540a2b90 0x7fc5540a5050 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc5600314e0 tx=0x7fc560000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:12.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.200+0000 7fc567400640 1 -- 172.21.15.31:0/2745127556 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc55809c300 con 0x7fc570104a10 2024-04-25T01:15:12.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:12 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1585130760' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:15:12.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:12 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2139344585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:15:12.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:12 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1761214997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:15:12.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.405+0000 7fc56fe00640 1 -- 172.21.15.31:0/2745127556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc570062340 con 0x7fc570104a10 2024-04-25T01:15:12.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.406+0000 7fc567400640 1 -- 172.21.15.31:0/2745127556 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fc55809bc00 con 0x7fc570104a10 2024-04-25T01:15:12.392 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-04-25T01:15:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.409+0000 7fc56fe00640 1 -- 172.21.15.31:0/2745127556 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc5540a2b90 msgr2=0x7fc5540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.409+0000 7fc56fe00640 1 --2- 172.21.15.31:0/2745127556 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc5540a2b90 0x7fc5540a5050 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc5600314e0 tx=0x7fc560000bc0 comp rx=0 tx=0).stop 2024-04-25T01:15:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.409+0000 7fc56fe00640 1 -- 172.21.15.31:0/2745127556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc570104a10 msgr2=0x7fc57019a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.409+0000 7fc56fe00640 1 --2- 172.21.15.31:0/2745127556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc570104a10 0x7fc57019a120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc55800e9f0 tx=0x7fc55800edf0 comp rx=0 tx=0).stop 2024-04-25T01:15:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.409+0000 7fc56fe00640 1 -- 172.21.15.31:0/2745127556 shutdown_connections 2024-04-25T01:15:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.409+0000 7fc56fe00640 1 --2- 172.21.15.31:0/2745127556 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc5540a2b90 0x7fc5540a5050 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.409+0000 7fc56fe00640 1 --2- 172.21.15.31:0/2745127556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc570107340 0x7fc5701a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.409+0000 7fc56fe00640 1 --2- 172.21.15.31:0/2745127556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc570104a10 0x7fc57019a120 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:12.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.409+0000 7fc56fe00640 1 --2- 172.21.15.31:0/2745127556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5701020e0 0x7fc570199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:12.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.409+0000 7fc56fe00640 1 -- 172.21.15.31:0/2745127556 >> 172.21.15.31:0/2745127556 conn(0x7fc5700fbaf0 msgr2=0x7fc570101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:12.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.409+0000 7fc56fe00640 1 -- 172.21.15.31:0/2745127556 shutdown_connections 2024-04-25T01:15:12.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.410+0000 7fc56fe00640 1 -- 172.21.15.31:0/2745127556 wait complete. 2024-04-25T01:15:12.414 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:15:12.444 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:12 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1585130760' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:15:12.444 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:12 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2139344585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:15:12.444 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:12 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1761214997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:15:12.446 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:12 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1585130760' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:15:12.447 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:12 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2139344585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:15:12.447 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:12 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1761214997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:15:12.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.642+0000 7f88dd000640 1 -- 172.21.15.31:0/385223998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88d8105370 msgr2=0x7f88d810bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:12.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.642+0000 7f88dd000640 1 --2- 172.21.15.31:0/385223998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88d8105370 0x7f88d810bc00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009be0 tx=0x7f88d002a6d0 comp rx=0 tx=0).stop 2024-04-25T01:15:12.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.642+0000 7f88dd000640 1 -- 172.21.15.31:0/385223998 shutdown_connections 2024-04-25T01:15:12.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.642+0000 7f88dd000640 1 --2- 172.21.15.31:0/385223998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88d8105370 0x7f88d810bc00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:12.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.642+0000 7f88dd000640 1 --2- 172.21.15.31:0/385223998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88d81049b0 0x7f88d8104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:12.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.642+0000 7f88dd000640 1 --2- 172.21.15.31:0/385223998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88d81037c0 0x7f88d8103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:12.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.642+0000 7f88dd000640 1 -- 172.21.15.31:0/385223998 >> 172.21.15.31:0/385223998 conn(0x7f88d80fed50 msgr2=0x7f88d8101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:12.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.642+0000 7f88dd000640 1 -- 172.21.15.31:0/385223998 shutdown_connections 2024-04-25T01:15:12.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.642+0000 7f88dd000640 1 -- 172.21.15.31:0/385223998 wait complete. 2024-04-25T01:15:12.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.643+0000 7f88dd000640 1 Processor -- start 2024-04-25T01:15:12.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.643+0000 7f88dd000640 1 -- start start 2024-04-25T01:15:12.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.643+0000 7f88dd000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88d81037c0 0x7f88d8199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:12.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.643+0000 7f88dd000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88d81049b0 0x7f88d8199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:12.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.643+0000 7f88dd000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88d8105370 0x7f88d81a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:12.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.643+0000 7f88dd000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f88d810f3f0 con 0x7f88d81049b0 2024-04-25T01:15:12.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.643+0000 7f88dd000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f88d810f270 con 0x7f88d81037c0 2024-04-25T01:15:12.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.643+0000 7f88dd000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f88d810f570 con 0x7f88d8105370 2024-04-25T01:15:12.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.644+0000 7f88cea00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88d81049b0 0x7f88d8199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:12.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.644+0000 7f88cea00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88d81049b0 0x7f88d8199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46050/0 (socket says 172.21.15.31:46050) 2024-04-25T01:15:12.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.644+0000 7f88cea00640 1 -- 172.21.15.31:0/1700740485 learned_addr learned my addr 172.21.15.31:0/1700740485 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:12.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.644+0000 7f88d7400640 1 --2- 172.21.15.31:0/1700740485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88d81037c0 0x7f88d8199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:12.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.644+0000 7f88d7e00640 1 --2- 172.21.15.31:0/1700740485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88d8105370 0x7f88d81a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:12.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.644+0000 7f88cea00640 1 -- 172.21.15.31:0/1700740485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88d8105370 msgr2=0x7f88d81a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:12.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.644+0000 7f88cea00640 1 --2- 172.21.15.31:0/1700740485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88d8105370 0x7f88d81a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:12.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.644+0000 7f88cea00640 1 -- 172.21.15.31:0/1700740485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88d81037c0 msgr2=0x7f88d8199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:12.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.644+0000 7f88cea00640 1 --2- 172.21.15.31:0/1700740485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88d81037c0 0x7f88d8199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:12.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.644+0000 7f88cea00640 1 -- 172.21.15.31:0/1700740485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d81a1680 con 0x7f88d81049b0 2024-04-25T01:15:12.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.644+0000 7f88cea00640 1 --2- 172.21.15.31:0/1700740485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88d81049b0 0x7f88d8199fa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f88c000e9c0 tx=0x7f88c000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:12.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.645+0000 7f88d4c00640 1 -- 172.21.15.31:0/1700740485 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c000cde0 con 0x7f88d81049b0 2024-04-25T01:15:12.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.645+0000 7f88dd000640 1 -- 172.21.15.31:0/1700740485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d81a1970 con 0x7f88d81049b0 2024-04-25T01:15:12.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.645+0000 7f88d4c00640 1 -- 172.21.15.31:0/1700740485 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c00035f0 con 0x7f88d81049b0 2024-04-25T01:15:12.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.645+0000 7f88d4c00640 1 -- 172.21.15.31:0/1700740485 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0010650 con 0x7f88d81049b0 2024-04-25T01:15:12.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.645+0000 7f88dd000640 1 -- 172.21.15.31:0/1700740485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d81a1f10 con 0x7f88d81049b0 2024-04-25T01:15:12.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.646+0000 7f88dd000640 1 -- 172.21.15.31:0/1700740485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88d810ab60 con 0x7f88d81049b0 2024-04-25T01:15:12.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.648+0000 7f88d4c00640 1 -- 172.21.15.31:0/1700740485 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f88c00037a0 con 0x7f88d81049b0 2024-04-25T01:15:12.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.649+0000 7f88d4c00640 1 --2- 172.21.15.31:0/1700740485 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f88bc0a2c10 0x7f88bc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:12.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.650+0000 7f88d4c00640 1 -- 172.21.15.31:0/1700740485 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f88c00cf2f0 con 0x7f88d81049b0 2024-04-25T01:15:12.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.650+0000 7f88d7400640 1 --2- 172.21.15.31:0/1700740485 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f88bc0a2c10 0x7f88bc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:12.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.650+0000 7f88d7400640 1 --2- 172.21.15.31:0/1700740485 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f88bc0a2c10 0x7f88bc0a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f88d8104810 tx=0x7f88c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:12.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.654+0000 7f88d4c00640 1 -- 172.21.15.31:0/1700740485 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f88c009c240 con 0x7f88d81049b0 2024-04-25T01:15:12.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.899+0000 7f88dd000640 1 -- 172.21.15.31:0/1700740485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88d802d170 con 0x7f88d81049b0 2024-04-25T01:15:12.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.900+0000 7f88d4c00640 1 -- 172.21.15.31:0/1700740485 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11854 (secure 0 0 0) 0x7f88c009bb40 con 0x7f88d81049b0 2024-04-25T01:15:12.886 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:15:12.887 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/498218714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":498218714},{"type":"v1","addr":"172.21.15.31:6835","nonce":498218714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2647199475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2647199475},{"type":"v1","addr":"172.21.15.31:6839","nonce":2647199475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/30518311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":30518311},{"type":"v1","addr":"172.21.15.31:6841","nonce":30518311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/420362823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":420362823},{"type":"v1","addr":"172.21.15.169:6833","nonce":420362823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1943710438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1943710438},{"type":"v1","addr":"172.21.15.40:6833","nonce":1943710438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/4191344115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":4191344115},{"type":"v1","addr":"172.21.15.31:6837","nonce":4191344115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:13:30.975848+0000","modified":"2024-04-25T01:13:36.545846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24565,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6835/2916305801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2916305801},{"type":"v1","addr":"172.21.15.40:6835","nonce":2916305801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.169:6835/2253264784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2253264784},{"type":"v1","addr":"172.21.15.169:6835","nonce":2253264784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6837/2883308641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2883308641},{"type":"v1","addr":"172.21.15.169:6837","nonce":2883308641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/590109562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":590109562},{"type":"v1","addr":"172.21.15.40:6837","nonce":590109562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/4185710218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":4185710218},{"type":"v1","addr":"172.21.15.169:6839","nonce":4185710218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6839/927194004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":927194004},{"type":"v1","addr":"172.21.15.40:6839","nonce":927194004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24532,24565,24596]},"id":1}]} 2024-04-25T01:15:12.887 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 23 2024-04-25T01:15:12.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.903+0000 7f88dd000640 1 -- 172.21.15.31:0/1700740485 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f88bc0a2c10 msgr2=0x7f88bc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:12.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.903+0000 7f88dd000640 1 --2- 172.21.15.31:0/1700740485 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f88bc0a2c10 0x7f88bc0a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f88d8104810 tx=0x7f88c8005f90 comp rx=0 tx=0).stop 2024-04-25T01:15:12.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.903+0000 7f88dd000640 1 -- 172.21.15.31:0/1700740485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88d81049b0 msgr2=0x7f88d8199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:12.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.903+0000 7f88dd000640 1 --2- 172.21.15.31:0/1700740485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88d81049b0 0x7f88d8199fa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f88c000e9c0 tx=0x7f88c000ee80 comp rx=0 tx=0).stop 2024-04-25T01:15:12.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.903+0000 7f88dd000640 1 -- 172.21.15.31:0/1700740485 shutdown_connections 2024-04-25T01:15:12.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.903+0000 7f88dd000640 1 --2- 172.21.15.31:0/1700740485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88d8105370 0x7f88d81a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:12.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.903+0000 7f88dd000640 1 --2- 172.21.15.31:0/1700740485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88d81049b0 0x7f88d8199fa0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:12.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.903+0000 7f88dd000640 1 --2- 172.21.15.31:0/1700740485 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f88bc0a2c10 0x7f88bc0a50d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:12.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.903+0000 7f88dd000640 1 --2- 172.21.15.31:0/1700740485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88d81037c0 0x7f88d8199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:12.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.903+0000 7f88dd000640 1 -- 172.21.15.31:0/1700740485 >> 172.21.15.31:0/1700740485 conn(0x7f88d80fed50 msgr2=0x7f88d8107d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:12.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.903+0000 7f88dd000640 1 -- 172.21.15.31:0/1700740485 shutdown_connections 2024-04-25T01:15:12.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:12.904+0000 7f88dd000640 1 -- 172.21.15.31:0/1700740485 wait complete. 2024-04-25T01:15:12.910 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:15:13.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.132+0000 7f12ea000640 1 -- 172.21.15.31:0/3904349854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12e4102040 msgr2=0x7f12e4104430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:13.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.133+0000 7f12ea000640 1 --2- 172.21.15.31:0/3904349854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12e4102040 0x7f12e4104430 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f12d8009860 tx=0x7f12d802a340 comp rx=0 tx=0).stop 2024-04-25T01:15:13.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.133+0000 7f12ea000640 1 -- 172.21.15.31:0/3904349854 shutdown_connections 2024-04-25T01:15:13.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.133+0000 7f12ea000640 1 --2- 172.21.15.31:0/3904349854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12e41072a0 0x7f12e41096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:13.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.133+0000 7f12ea000640 1 --2- 172.21.15.31:0/3904349854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12e4104970 0x7f12e4106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:13.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.133+0000 7f12ea000640 1 --2- 172.21.15.31:0/3904349854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12e4102040 0x7f12e4104430 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:13.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.133+0000 7f12ea000640 1 -- 172.21.15.31:0/3904349854 >> 172.21.15.31:0/3904349854 conn(0x7f12e40fba10 msgr2=0x7f12e40fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:13.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.133+0000 7f12ea000640 1 -- 172.21.15.31:0/3904349854 shutdown_connections 2024-04-25T01:15:13.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.133+0000 7f12ea000640 1 -- 172.21.15.31:0/3904349854 wait complete. 2024-04-25T01:15:13.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.134+0000 7f12ea000640 1 Processor -- start 2024-04-25T01:15:13.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.134+0000 7f12ea000640 1 -- start start 2024-04-25T01:15:13.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.134+0000 7f12ea000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12e4102040 0x7f12e41019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:13.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.134+0000 7f12ea000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12e4104970 0x7f12e40ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:13.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.134+0000 7f12ea000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12e41072a0 0x7f12e4100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:13.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.134+0000 7f12ea000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f12e410d250 con 0x7f12e4102040 2024-04-25T01:15:13.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.134+0000 7f12ea000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f12e410d0d0 con 0x7f12e41072a0 2024-04-25T01:15:13.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.134+0000 7f12ea000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f12e410d3d0 con 0x7f12e4104970 2024-04-25T01:15:13.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.135+0000 7f12e3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12e4104970 0x7f12e40ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:13.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.135+0000 7f12e3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12e4104970 0x7f12e40ffff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:59806/0 (socket says 172.21.15.31:59806) 2024-04-25T01:15:13.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.135+0000 7f12e3e00640 1 -- 172.21.15.31:0/272698156 learned_addr learned my addr 172.21.15.31:0/272698156 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:13.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.135+0000 7f12e9600640 1 --2- 172.21.15.31:0/272698156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12e41072a0 0x7f12e4100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:13.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.135+0000 7f12e3e00640 1 -- 172.21.15.31:0/272698156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12e41072a0 msgr2=0x7f12e4100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:13.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.135+0000 7f12e3e00640 1 --2- 172.21.15.31:0/272698156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12e41072a0 0x7f12e4100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:13.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.135+0000 7f12e3e00640 1 -- 172.21.15.31:0/272698156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12e4102040 msgr2=0x7f12e41019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:13.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.135+0000 7f12e8c00640 1 --2- 172.21.15.31:0/272698156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12e4102040 0x7f12e41019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:13.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.136+0000 7f12e3e00640 1 --2- 172.21.15.31:0/272698156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12e4102040 0x7f12e41019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:13.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.136+0000 7f12e3e00640 1 -- 172.21.15.31:0/272698156 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e4100dc0 con 0x7f12e4104970 2024-04-25T01:15:13.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.136+0000 7f12e8c00640 1 --2- 172.21.15.31:0/272698156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12e4102040 0x7f12e41019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:15:13.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.136+0000 7f12e3e00640 1 --2- 172.21.15.31:0/272698156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12e4104970 0x7f12e40ffff0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f12d000e9c0 tx=0x7f12d000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:13.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.136+0000 7f12e1600640 1 -- 172.21.15.31:0/272698156 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d000cd80 con 0x7f12e4104970 2024-04-25T01:15:13.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.136+0000 7f12e1600640 1 -- 172.21.15.31:0/272698156 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12d0005500 con 0x7f12e4104970 2024-04-25T01:15:13.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.136+0000 7f12ea000640 1 -- 172.21.15.31:0/272698156 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e41a19f0 con 0x7f12e4104970 2024-04-25T01:15:13.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.136+0000 7f12e1600640 1 -- 172.21.15.31:0/272698156 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d0010690 con 0x7f12e4104970 2024-04-25T01:15:13.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.136+0000 7f12ea000640 1 -- 172.21.15.31:0/272698156 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e41a1f90 con 0x7f12e4104970 2024-04-25T01:15:13.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.138+0000 7f12ea000640 1 -- 172.21.15.31:0/272698156 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12e40fd500 con 0x7f12e4104970 2024-04-25T01:15:13.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.140+0000 7f12e1600640 1 -- 172.21.15.31:0/272698156 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f12d0010830 con 0x7f12e4104970 2024-04-25T01:15:13.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.142+0000 7f12e1600640 1 --2- 172.21.15.31:0/272698156 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f12cc0a2c10 0x7f12cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:13.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.142+0000 7f12e1600640 1 -- 172.21.15.31:0/272698156 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f12d00cf480 con 0x7f12e4104970 2024-04-25T01:15:13.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.142+0000 7f12e8c00640 1 --2- 172.21.15.31:0/272698156 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f12cc0a2c10 0x7f12cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:13.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.144+0000 7f12e8c00640 1 --2- 172.21.15.31:0/272698156 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f12cc0a2c10 0x7f12cc0a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f12d8009780 tx=0x7f12d8030df0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:13.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.146+0000 7f12e1600640 1 -- 172.21.15.31:0/272698156 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f12d009c3d0 con 0x7f12e4104970 2024-04-25T01:15:13.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:13 smithi031 ceph-mon[28053]: pgmap v369: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T01:15:13.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:13 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2745127556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:15:13.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:13 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1700740485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:15:13.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.353+0000 7f12ea000640 1 -- 172.21.15.31:0/272698156 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f12e4062340 con 0x7f12e4104970 2024-04-25T01:15:13.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.356+0000 7f12e1600640 1 -- 172.21.15.31:0/272698156 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22530 (secure 0 0 0) 0x7f12d009bcd0 con 0x7f12e4104970 2024-04-25T01:15:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:15:13.342 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":89,"fsid":"95849b94-029f-11ef-bc93-c7b262605968","created":"2024-04-25T01:04:23.310856+0000","modified":"2024-04-25T01:14:33.793638+0000","last_up_change":"2024-04-25T01:12:29.393907+0000","last_in_change":"2024-04-25T01:12:08.978148+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:18.662391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:13:29.258771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:13:30.480996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa08e4c-fe6b-4116-bdf5-1adf5b465d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6803","nonce":788174091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6805","nonce":788174091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6809","nonce":788174091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6807","nonce":788174091}]},"public_addr":"172.21.15.31:6803/788174091","cluster_addr":"172.21.15.31:6805/788174091","heartbeat_back_addr":"172.21.15.31:6809/788174091","heartbeat_front_addr":"172.21.15.31:6807/788174091","state":["exists","up"]},{"osd":1,"uuid":"85339fe7-0dbc-4485-9334-2b039993c1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6801","nonce":3196394509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6803","nonce":3196394509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6807","nonce":3196394509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6805","nonce":3196394509}]},"public_addr":"172.21.15.40:6801/3196394509","cluster_addr":"172.21.15.40:6803/3196394509","heartbeat_back_addr":"172.21.15.40:6807/3196394509","heartbeat_front_addr":"172.21.15.40:6805/3196394509","state":["exists","up"]},{"osd":2,"uuid":"ab62ecb2-8aae-4987-9e2e-589e0f2fee5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6801","nonce":1955211197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6803","nonce":1955211197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6807","nonce":1955211197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6805","nonce":1955211197}]},"public_addr":"172.21.15.169:6801/1955211197","cluster_addr":"172.21.15.169:6803/1955211197","heartbeat_back_addr":"172.21.15.169:6807/1955211197","heartbeat_front_addr":"172.21.15.169:6805/1955211197","state":["exists","up"]},{"osd":3,"uuid":"8b5a7881-3401-495c-8ca4-c90685919d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6811","nonce":1228688643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6813","nonce":1228688643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6817","nonce":1228688643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6815","nonce":1228688643}]},"public_addr":"172.21.15.31:6811/1228688643","cluster_addr":"172.21.15.31:6813/1228688643","heartbeat_back_addr":"172.21.15.31:6817/1228688643","heartbeat_front_addr":"172.21.15.31:6815/1228688643","state":["exists","up"]},{"osd":4,"uuid":"56deb61d-a334-4cf2-8ac7-fbb6e6947092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6809","nonce":1913779091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6811","nonce":1913779091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6815","nonce":1913779091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6813","nonce":1913779091}]},"public_addr":"172.21.15.40:6809/1913779091","cluster_addr":"172.21.15.40:6811/1913779091","heartbeat_back_addr":"172.21.15.40:6815/1913779091","heartbeat_front_addr":"172.21.15.40:6813/1913779091","state":["exists","up"]},{"osd":5,"uuid":"8da55693-f9c7-49a7-99e0-165078f8001b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6809","nonce":2549650781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6811","nonce":2549650781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6815","nonce":2549650781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6813","nonce":2549650781}]},"public_addr":"172.21.15.169:6809/2549650781","cluster_addr":"172.21.15.169:6811/2549650781","heartbeat_back_addr":"172.21.15.169:6815/2549650781","heartbeat_front_addr":"172.21.15.169:6813/2549650781","state":["exists","up"]},{"osd":6,"uuid":"fc8350d2-1ad6-43cb-89f4-d336afc99ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6819","nonce":611507624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6821","nonce":611507624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6825","nonce":611507624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6823","nonce":611507624}]},"public_addr":"172.21.15.31:6819/611507624","cluster_addr":"172.21.15.31:6821/611507624","heartbeat_back_addr":"172.21.15.31:6825/611507624","heartbeat_front_addr":"172.21.15.31:6823/611507624","state":["exists","up"]},{"osd":7,"uuid":"3121b58b-8c94-40b5-9d48-f56552d7c39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6817","nonce":3776085849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6819","nonce":3776085849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6823","nonce":3776085849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6821","nonce":3776085849}]},"public_addr":"172.21.15.40:6817/3776085849","cluster_addr":"172.21.15.40:6819/3776085849","heartbeat_back_addr":"172.21.15.40:6823/3776085849","heartbeat_front_addr":"172.21.15.40:6821/3776085849","state":["exists","up"]},{"osd":8,"uuid":"f60407df-91f5-477a-9702-ddaef24faf82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6817","nonce":2736546122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6819","nonce":2736546122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6823","nonce":2736546122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6821","nonce":2736546122}]},"public_addr":"172.21.15.169:6817/2736546122","cluster_addr":"172.21.15.169:6819/2736546122","heartbeat_back_addr":"172.21.15.169:6823/2736546122","heartbeat_front_addr":"172.21.15.169:6821/2736546122","state":["exists","up"]},{"osd":9,"uuid":"7506a407-1960-49bc-9e38-7f2a8bf996ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6827","nonce":1187207880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6829","nonce":1187207880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6833","nonce":1187207880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6831","nonce":1187207880}]},"public_addr":"172.21.15.31:6827/1187207880","cluster_addr":"172.21.15.31:6829/1187207880","heartbeat_back_addr":"172.21.15.31:6833/1187207880","heartbeat_front_addr":"172.21.15.31:6831/1187207880","state":["exists","up"]},{"osd":10,"uuid":"831c48e2-b0d0-451f-959b-228e927542cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6825","nonce":3050148001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6827","nonce":3050148001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6831","nonce":3050148001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6829","nonce":3050148001}]},"public_addr":"172.21.15.40:6825/3050148001","cluster_addr":"172.21.15.40:6827/3050148001","heartbeat_back_addr":"172.21.15.40:6831/3050148001","heartbeat_front_addr":"172.21.15.40:6829/3050148001","state":["exists","up"]},{"osd":11,"uuid":"144fe0aa-f757-4b3c-90ad-99775e17d57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6825","nonce":262401944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6827","nonce":262401944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6831","nonce":262401944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6829","nonce":262401944}]},"public_addr":"172.21.15.169:6825/262401944","cluster_addr":"172.21.15.169:6827/262401944","heartbeat_back_addr":"172.21.15.169:6831/262401944","heartbeat_front_addr":"172.21.15.169:6829/262401944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:25.640995+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:51.036909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.635633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:48.467912+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:14.454971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:37.609315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:12.441807+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:38.239215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:02.230758+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:35.762553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:02.051405+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:26.569593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6801/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4004014066":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/1930101646":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/2638309002":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2238638505":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2902841144":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4190696323":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/480558521":"2024-04-26T01:04:57.149272+0000","172.21.15.31:6801/2638309002":"2024-04-26T01:04:57.149272+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:15:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.358+0000 7f12ea000640 1 -- 172.21.15.31:0/272698156 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f12cc0a2c10 msgr2=0x7f12cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.358+0000 7f12ea000640 1 --2- 172.21.15.31:0/272698156 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f12cc0a2c10 0x7f12cc0a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f12d8009780 tx=0x7f12d8030df0 comp rx=0 tx=0).stop 2024-04-25T01:15:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.358+0000 7f12ea000640 1 -- 172.21.15.31:0/272698156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12e4104970 msgr2=0x7f12e40ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.358+0000 7f12ea000640 1 --2- 172.21.15.31:0/272698156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12e4104970 0x7f12e40ffff0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f12d000e9c0 tx=0x7f12d000ee80 comp rx=0 tx=0).stop 2024-04-25T01:15:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.359+0000 7f12ea000640 1 -- 172.21.15.31:0/272698156 shutdown_connections 2024-04-25T01:15:13.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.359+0000 7f12ea000640 1 --2- 172.21.15.31:0/272698156 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f12cc0a2c10 0x7f12cc0a50d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:13.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.359+0000 7f12ea000640 1 --2- 172.21.15.31:0/272698156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12e41072a0 0x7f12e4100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:13.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.359+0000 7f12ea000640 1 --2- 172.21.15.31:0/272698156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12e4104970 0x7f12e40ffff0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:13.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.359+0000 7f12ea000640 1 --2- 172.21.15.31:0/272698156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12e4102040 0x7f12e41019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:13.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.359+0000 7f12ea000640 1 -- 172.21.15.31:0/272698156 >> 172.21.15.31:0/272698156 conn(0x7f12e40fba10 msgr2=0x7f12e4102900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:13.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.359+0000 7f12ea000640 1 -- 172.21.15.31:0/272698156 shutdown_connections 2024-04-25T01:15:13.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.359+0000 7f12ea000640 1 -- 172.21.15.31:0/272698156 wait complete. 2024-04-25T01:15:13.364 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:18.662391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:13:29.258771+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:13:30.480996+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:15:13.365 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:15:13.443 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:13 smithi169 ceph-mon[32759]: pgmap v369: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T01:15:13.444 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:13 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2745127556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:15:13.444 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:13 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1700740485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:15:13.447 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:13 smithi040 ceph-mon[32518]: pgmap v369: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T01:15:13.447 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:13 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2745127556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:15:13.447 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:13 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1700740485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:15:13.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.581+0000 7f1ac1800640 1 -- 172.21.15.31:0/940453725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1abc104a70 msgr2=0x7f1abc104ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:13.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.581+0000 7f1ac1800640 1 --2- 172.21.15.31:0/940453725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1abc104a70 0x7f1abc104ef0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009900 tx=0x7f1aa402a3f0 comp rx=0 tx=0).stop 2024-04-25T01:15:13.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.581+0000 7f1ac1800640 1 -- 172.21.15.31:0/940453725 shutdown_connections 2024-04-25T01:15:13.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.581+0000 7f1ac1800640 1 --2- 172.21.15.31:0/940453725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1abc105430 0x7f1abc10bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:13.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.581+0000 7f1ac1800640 1 --2- 172.21.15.31:0/940453725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1abc104a70 0x7f1abc104ef0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:13.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.581+0000 7f1ac1800640 1 --2- 172.21.15.31:0/940453725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1abc103880 0x7f1abc103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:13.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.581+0000 7f1ac1800640 1 -- 172.21.15.31:0/940453725 >> 172.21.15.31:0/940453725 conn(0x7f1abc0fedf0 msgr2=0x7f1abc101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:13.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.581+0000 7f1ac1800640 1 -- 172.21.15.31:0/940453725 shutdown_connections 2024-04-25T01:15:13.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.581+0000 7f1ac1800640 1 -- 172.21.15.31:0/940453725 wait complete. 2024-04-25T01:15:13.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.582+0000 7f1ac1800640 1 Processor -- start 2024-04-25T01:15:13.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.582+0000 7f1ac1800640 1 -- start start 2024-04-25T01:15:13.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.583+0000 7f1ac1800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1abc103880 0x7f1abc199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:13.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.583+0000 7f1ac1800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1abc104a70 0x7f1abc19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:13.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.583+0000 7f1ac1800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1abc105430 0x7f1abc1a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:13.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.583+0000 7f1ac1800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc10f530 con 0x7f1abc105430 2024-04-25T01:15:13.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.583+0000 7f1ac1800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc10f3b0 con 0x7f1abc104a70 2024-04-25T01:15:13.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.583+0000 7f1ac1800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc10f6b0 con 0x7f1abc103880 2024-04-25T01:15:13.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.583+0000 7f1ac0e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1abc105430 0x7f1abc1a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:13.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.583+0000 7f1ac0e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1abc105430 0x7f1abc1a1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46086/0 (socket says 172.21.15.31:46086) 2024-04-25T01:15:13.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.583+0000 7f1ac0e00640 1 -- 172.21.15.31:0/2335809967 learned_addr learned my addr 172.21.15.31:0/2335809967 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:13.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.583+0000 7f1abbe00640 1 --2- 172.21.15.31:0/2335809967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1abc103880 0x7f1abc199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:13.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.583+0000 7f1ab3e00640 1 --2- 172.21.15.31:0/2335809967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1abc104a70 0x7f1abc19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:13.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.583+0000 7f1ac0e00640 1 -- 172.21.15.31:0/2335809967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1abc103880 msgr2=0x7f1abc199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:13.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.583+0000 7f1ac0e00640 1 --2- 172.21.15.31:0/2335809967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1abc103880 0x7f1abc199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:13.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.584+0000 7f1ac0e00640 1 -- 172.21.15.31:0/2335809967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1abc104a70 msgr2=0x7f1abc19a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:13.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.584+0000 7f1ac0e00640 1 --2- 172.21.15.31:0/2335809967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1abc104a70 0x7f1abc19a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:13.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.584+0000 7f1ac0e00640 1 -- 172.21.15.31:0/2335809967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1abc1a17a0 con 0x7f1abc105430 2024-04-25T01:15:13.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.584+0000 7f1abbe00640 1 --2- 172.21.15.31:0/2335809967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1abc103880 0x7f1abc199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:13.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.584+0000 7f1ac0e00640 1 --2- 172.21.15.31:0/2335809967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1abc105430 0x7f1abc1a1130 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400efd0 tx=0x7f1ab400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:13.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.584+0000 7f1ab9600640 1 -- 172.21.15.31:0/2335809967 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4003b70 con 0x7f1abc105430 2024-04-25T01:15:13.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.584+0000 7f1ac1800640 1 -- 172.21.15.31:0/2335809967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1abc1a1a90 con 0x7f1abc105430 2024-04-25T01:15:13.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.584+0000 7f1ab9600640 1 -- 172.21.15.31:0/2335809967 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ab4003d10 con 0x7f1abc105430 2024-04-25T01:15:13.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.584+0000 7f1ab9600640 1 -- 172.21.15.31:0/2335809967 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab40109f0 con 0x7f1abc105430 2024-04-25T01:15:13.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.584+0000 7f1ac1800640 1 -- 172.21.15.31:0/2335809967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1abc1a1f60 con 0x7f1abc105430 2024-04-25T01:15:13.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.586+0000 7f1ac1800640 1 -- 172.21.15.31:0/2335809967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1abc0688d0 con 0x7f1abc105430 2024-04-25T01:15:13.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.588+0000 7f1ab9600640 1 -- 172.21.15.31:0/2335809967 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1ab4004020 con 0x7f1abc105430 2024-04-25T01:15:13.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.589+0000 7f1ab9600640 1 --2- 172.21.15.31:0/2335809967 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1aa00a2b90 0x7f1aa00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:13.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.589+0000 7f1ab9600640 1 -- 172.21.15.31:0/2335809967 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f1ab40cf490 con 0x7f1abc105430 2024-04-25T01:15:13.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.590+0000 7f1abbe00640 1 --2- 172.21.15.31:0/2335809967 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1aa00a2b90 0x7f1aa00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:13.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.590+0000 7f1abbe00640 1 --2- 172.21.15.31:0/2335809967 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1aa00a2b90 0x7f1aa00a5050 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1abc1048d0 tx=0x7f1aac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:13.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.594+0000 7f1ab9600640 1 -- 172.21.15.31:0/2335809967 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1ab409c3e0 con 0x7f1abc105430 2024-04-25T01:15:13.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.800+0000 7f1ac1800640 1 -- 172.21.15.31:0/2335809967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1abc062340 con 0x7f1abc105430 2024-04-25T01:15:13.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.801+0000 7f1ab9600640 1 -- 172.21.15.31:0/2335809967 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f1ab409bce0 con 0x7f1abc105430 2024-04-25T01:15:13.787 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T01:15:13.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.804+0000 7f1ac1800640 1 -- 172.21.15.31:0/2335809967 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1aa00a2b90 msgr2=0x7f1aa00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:13.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.804+0000 7f1ac1800640 1 --2- 172.21.15.31:0/2335809967 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1aa00a2b90 0x7f1aa00a5050 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1abc1048d0 tx=0x7f1aac005f90 comp rx=0 tx=0).stop 2024-04-25T01:15:13.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.804+0000 7f1ac1800640 1 -- 172.21.15.31:0/2335809967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1abc105430 msgr2=0x7f1abc1a1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:13.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.804+0000 7f1ac1800640 1 --2- 172.21.15.31:0/2335809967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1abc105430 0x7f1abc1a1130 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400efd0 tx=0x7f1ab400c6a0 comp rx=0 tx=0).stop 2024-04-25T01:15:13.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.804+0000 7f1ac1800640 1 -- 172.21.15.31:0/2335809967 shutdown_connections 2024-04-25T01:15:13.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.804+0000 7f1ac1800640 1 --2- 172.21.15.31:0/2335809967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1abc105430 0x7f1abc1a1130 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:13.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.804+0000 7f1ac1800640 1 --2- 172.21.15.31:0/2335809967 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1aa00a2b90 0x7f1aa00a5050 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:13.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.804+0000 7f1ac1800640 1 --2- 172.21.15.31:0/2335809967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1abc104a70 0x7f1abc19a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:13.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.804+0000 7f1ac1800640 1 --2- 172.21.15.31:0/2335809967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1abc103880 0x7f1abc199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:13.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.804+0000 7f1ac1800640 1 -- 172.21.15.31:0/2335809967 >> 172.21.15.31:0/2335809967 conn(0x7f1abc0fedf0 msgr2=0x7f1abc107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:13.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.804+0000 7f1ac1800640 1 -- 172.21.15.31:0/2335809967 shutdown_connections 2024-04-25T01:15:13.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:13.804+0000 7f1ac1800640 1 -- 172.21.15.31:0/2335809967 wait complete. 2024-04-25T01:15:13.810 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:15:14.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.028+0000 7fa371600640 1 -- 172.21.15.31:0/2647965062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa36c1008a0 msgr2=0x7fa36c100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:14.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.028+0000 7fa371600640 1 --2- 172.21.15.31:0/2647965062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa36c1008a0 0x7fa36c100d00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa354009860 tx=0x7fa35402a3d0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.028+0000 7fa371600640 1 -- 172.21.15.31:0/2647965062 shutdown_connections 2024-04-25T01:15:14.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.028+0000 7fa371600640 1 --2- 172.21.15.31:0/2647965062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa36c101240 0x7fa36c1016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.028+0000 7fa371600640 1 --2- 172.21.15.31:0/2647965062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa36c1008a0 0x7fa36c100d00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.028+0000 7fa371600640 1 --2- 172.21.15.31:0/2647965062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa36c0fff40 0x7fa36c100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.028+0000 7fa371600640 1 -- 172.21.15.31:0/2647965062 >> 172.21.15.31:0/2647965062 conn(0x7fa36c0fbae0 msgr2=0x7fa36c0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:14.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.028+0000 7fa371600640 1 -- 172.21.15.31:0/2647965062 shutdown_connections 2024-04-25T01:15:14.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.028+0000 7fa371600640 1 -- 172.21.15.31:0/2647965062 wait complete. 2024-04-25T01:15:14.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.029+0000 7fa371600640 1 Processor -- start 2024-04-25T01:15:14.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.029+0000 7fa371600640 1 -- start start 2024-04-25T01:15:14.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.029+0000 7fa371600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa36c0fff40 0x7fa36c105ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:14.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.030+0000 7fa371600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa36c1008a0 0x7fa36c106000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:14.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.030+0000 7fa371600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa36c101240 0x7fa36c102630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:14.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.030+0000 7fa371600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c10f500 con 0x7fa36c0fff40 2024-04-25T01:15:14.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.030+0000 7fa371600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c10f380 con 0x7fa36c101240 2024-04-25T01:15:14.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.030+0000 7fa371600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c10f680 con 0x7fa36c1008a0 2024-04-25T01:15:14.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.030+0000 7fa36be00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa36c0fff40 0x7fa36c105ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:14.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.030+0000 7fa36be00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa36c0fff40 0x7fa36c105ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46096/0 (socket says 172.21.15.31:46096) 2024-04-25T01:15:14.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.030+0000 7fa36be00640 1 -- 172.21.15.31:0/3220412232 learned_addr learned my addr 172.21.15.31:0/3220412232 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:14.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.030+0000 7fa36b400640 1 --2- 172.21.15.31:0/3220412232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa36c1008a0 0x7fa36c106000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:14.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.030+0000 7fa370c00640 1 --2- 172.21.15.31:0/3220412232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa36c101240 0x7fa36c102630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:14.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.030+0000 7fa36be00640 1 -- 172.21.15.31:0/3220412232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa36c1008a0 msgr2=0x7fa36c106000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:14.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.030+0000 7fa36be00640 1 --2- 172.21.15.31:0/3220412232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa36c1008a0 0x7fa36c106000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.030+0000 7fa36be00640 1 -- 172.21.15.31:0/3220412232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa36c101240 msgr2=0x7fa36c102630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:14.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.030+0000 7fa36be00640 1 --2- 172.21.15.31:0/3220412232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa36c101240 0x7fa36c102630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.030+0000 7fa36be00640 1 -- 172.21.15.31:0/3220412232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36c102ef0 con 0x7fa36c0fff40 2024-04-25T01:15:14.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.031+0000 7fa36be00640 1 --2- 172.21.15.31:0/3220412232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa36c0fff40 0x7fa36c105ac0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00ebc0 tx=0x7fa35c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:14.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.031+0000 7fa368c00640 1 -- 172.21.15.31:0/3220412232 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c010630 con 0x7fa36c0fff40 2024-04-25T01:15:14.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.031+0000 7fa371600640 1 -- 172.21.15.31:0/3220412232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa36c1a5f90 con 0x7fa36c0fff40 2024-04-25T01:15:14.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.031+0000 7fa368c00640 1 -- 172.21.15.31:0/3220412232 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa35c010cb0 con 0x7fa36c0fff40 2024-04-25T01:15:14.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.031+0000 7fa368c00640 1 -- 172.21.15.31:0/3220412232 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c018bd0 con 0x7fa36c0fff40 2024-04-25T01:15:14.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.031+0000 7fa371600640 1 -- 172.21.15.31:0/3220412232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa36c1a6340 con 0x7fa36c0fff40 2024-04-25T01:15:14.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.032+0000 7fa371600640 1 -- 172.21.15.31:0/3220412232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa36c10ac90 con 0x7fa36c0fff40 2024-04-25T01:15:14.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.034+0000 7fa368c00640 1 -- 172.21.15.31:0/3220412232 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa35c0107d0 con 0x7fa36c0fff40 2024-04-25T01:15:14.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.036+0000 7fa368c00640 1 --2- 172.21.15.31:0/3220412232 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa3500a2b90 0x7fa3500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:14.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.036+0000 7fa368c00640 1 -- 172.21.15.31:0/3220412232 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fa35c0cf720 con 0x7fa36c0fff40 2024-04-25T01:15:14.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.036+0000 7fa36b400640 1 --2- 172.21.15.31:0/3220412232 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa3500a2b90 0x7fa3500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:14.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.037+0000 7fa36b400640 1 --2- 172.21.15.31:0/3220412232 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa3500a2b90 0x7fa3500a5050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa35402a9c0 tx=0x7fa354030ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:14.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.040+0000 7fa368c00640 1 -- 172.21.15.31:0/3220412232 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa35c09c670 con 0x7fa36c0fff40 2024-04-25T01:15:14.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.244+0000 7fa371600640 1 -- 172.21.15.31:0/3220412232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa36c02d420 con 0x7fa36c0fff40 2024-04-25T01:15:14.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.245+0000 7fa368c00640 1 -- 172.21.15.31:0/3220412232 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fa35c09bf70 con 0x7fa36c0fff40 2024-04-25T01:15:14.230 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-25T01:15:14.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.247+0000 7fa371600640 1 -- 172.21.15.31:0/3220412232 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa3500a2b90 msgr2=0x7fa3500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:14.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.247+0000 7fa371600640 1 --2- 172.21.15.31:0/3220412232 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa3500a2b90 0x7fa3500a5050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa35402a9c0 tx=0x7fa354030ab0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.247+0000 7fa371600640 1 -- 172.21.15.31:0/3220412232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa36c0fff40 msgr2=0x7fa36c105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:14.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.247+0000 7fa371600640 1 --2- 172.21.15.31:0/3220412232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa36c0fff40 0x7fa36c105ac0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00ebc0 tx=0x7fa35c00c6a0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.247+0000 7fa371600640 1 -- 172.21.15.31:0/3220412232 shutdown_connections 2024-04-25T01:15:14.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.247+0000 7fa371600640 1 --2- 172.21.15.31:0/3220412232 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa3500a2b90 0x7fa3500a5050 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.247+0000 7fa371600640 1 --2- 172.21.15.31:0/3220412232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa36c101240 0x7fa36c102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.247+0000 7fa371600640 1 --2- 172.21.15.31:0/3220412232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa36c1008a0 0x7fa36c106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.247+0000 7fa371600640 1 --2- 172.21.15.31:0/3220412232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa36c0fff40 0x7fa36c105ac0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.247+0000 7fa371600640 1 -- 172.21.15.31:0/3220412232 >> 172.21.15.31:0/3220412232 conn(0x7fa36c0fbae0 msgr2=0x7fa36c0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:14.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.248+0000 7fa371600640 1 -- 172.21.15.31:0/3220412232 shutdown_connections 2024-04-25T01:15:14.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.248+0000 7fa371600640 1 -- 172.21.15.31:0/3220412232 wait complete. 2024-04-25T01:15:14.249 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:14 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/272698156' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:15:14.250 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:14 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2335809967' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:15:14.252 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:15:14.443 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:14 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/272698156' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:15:14.444 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:14 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2335809967' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:15:14.447 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:14 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/272698156' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:15:14.447 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:14 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2335809967' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:15:14.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.475+0000 7f6a7b800640 1 -- 172.21.15.31:0/2519955688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a74104b30 msgr2=0x7f6a74106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:14.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.475+0000 7f6a7b800640 1 --2- 172.21.15.31:0/2519955688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a74104b30 0x7f6a74106f20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009900 tx=0x7f6a5c02a3f0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.475+0000 7f6a7b800640 1 -- 172.21.15.31:0/2519955688 shutdown_connections 2024-04-25T01:15:14.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.475+0000 7f6a7b800640 1 --2- 172.21.15.31:0/2519955688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a74107460 0x7f6a741098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.475+0000 7f6a7b800640 1 --2- 172.21.15.31:0/2519955688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a74104b30 0x7f6a74106f20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.475+0000 7f6a7b800640 1 --2- 172.21.15.31:0/2519955688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a74102200 0x7f6a741045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.475+0000 7f6a7b800640 1 -- 172.21.15.31:0/2519955688 >> 172.21.15.31:0/2519955688 conn(0x7f6a740fbbf0 msgr2=0x7f6a740fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:14.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.476+0000 7f6a7b800640 1 -- 172.21.15.31:0/2519955688 shutdown_connections 2024-04-25T01:15:14.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.476+0000 7f6a7b800640 1 -- 172.21.15.31:0/2519955688 wait complete. 2024-04-25T01:15:14.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.477+0000 7f6a7b800640 1 Processor -- start 2024-04-25T01:15:14.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.477+0000 7f6a7b800640 1 -- start start 2024-04-25T01:15:14.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.477+0000 7f6a7b800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a74102200 0x7f6a74199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:14.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.477+0000 7f6a7b800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a74104b30 0x7f6a7419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:14.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.477+0000 7f6a7b800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a74107460 0x7f6a741a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:14.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.477+0000 7f6a7b800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7410d450 con 0x7f6a74107460 2024-04-25T01:15:14.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.477+0000 7f6a7b800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7410d2d0 con 0x7f6a74104b30 2024-04-25T01:15:14.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.477+0000 7f6a7b800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7410d5d0 con 0x7f6a74102200 2024-04-25T01:15:14.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.478+0000 7f6a73e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a74104b30 0x7f6a7419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:14.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.478+0000 7f6a73e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a74104b30 0x7f6a7419a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:42756/0 (socket says 172.21.15.31:42756) 2024-04-25T01:15:14.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.478+0000 7f6a73e00640 1 -- 172.21.15.31:0/1500104341 learned_addr learned my addr 172.21.15.31:0/1500104341 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:14.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.478+0000 7f6a79800640 1 --2- 172.21.15.31:0/1500104341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a74107460 0x7f6a741a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:14.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.478+0000 7f6a78e00640 1 --2- 172.21.15.31:0/1500104341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a74102200 0x7f6a74199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:14.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.478+0000 7f6a73e00640 1 -- 172.21.15.31:0/1500104341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a74102200 msgr2=0x7f6a74199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:14.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.478+0000 7f6a73e00640 1 --2- 172.21.15.31:0/1500104341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a74102200 0x7f6a74199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.478+0000 7f6a73e00640 1 -- 172.21.15.31:0/1500104341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a74107460 msgr2=0x7f6a741a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:14.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.478+0000 7f6a73e00640 1 --2- 172.21.15.31:0/1500104341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a74107460 0x7f6a741a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.478+0000 7f6a73e00640 1 -- 172.21.15.31:0/1500104341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a741a1960 con 0x7f6a74104b30 2024-04-25T01:15:14.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.479+0000 7f6a73e00640 1 --2- 172.21.15.31:0/1500104341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a74104b30 0x7f6a7419a280 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c031440 tx=0x7f6a5c032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:14.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.479+0000 7f6a71600640 1 -- 172.21.15.31:0/1500104341 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c03f030 con 0x7f6a74104b30 2024-04-25T01:15:14.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.479+0000 7f6a71600640 1 -- 172.21.15.31:0/1500104341 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a5c003ce0 con 0x7f6a74104b30 2024-04-25T01:15:14.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.479+0000 7f6a71600640 1 -- 172.21.15.31:0/1500104341 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c035070 con 0x7f6a74104b30 2024-04-25T01:15:14.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.479+0000 7f6a7b800640 1 -- 172.21.15.31:0/1500104341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a741a1bf0 con 0x7f6a74104b30 2024-04-25T01:15:14.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.479+0000 7f6a7b800640 1 -- 172.21.15.31:0/1500104341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a741a2130 con 0x7f6a74104b30 2024-04-25T01:15:14.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.480+0000 7f6a7b800640 1 -- 172.21.15.31:0/1500104341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a74108fb0 con 0x7f6a74104b30 2024-04-25T01:15:14.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.482+0000 7f6a71600640 1 -- 172.21.15.31:0/1500104341 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6a5c042030 con 0x7f6a74104b30 2024-04-25T01:15:14.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.484+0000 7f6a71600640 1 --2- 172.21.15.31:0/1500104341 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6a580a2c10 0x7f6a580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:14.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.484+0000 7f6a71600640 1 -- 172.21.15.31:0/1500104341 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f6a5c0ef500 con 0x7f6a74104b30 2024-04-25T01:15:14.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.484+0000 7f6a78e00640 1 --2- 172.21.15.31:0/1500104341 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6a580a2c10 0x7f6a580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:14.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.485+0000 7f6a78e00640 1 --2- 172.21.15.31:0/1500104341 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6a580a2c10 0x7f6a580a50d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6a64009bd0 tx=0x7f6a64005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:14.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.486+0000 7f6a71600640 1 -- 172.21.15.31:0/1500104341 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6a5c0bbe00 con 0x7f6a74104b30 2024-04-25T01:15:14.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.692+0000 7f6a7b800640 1 -- 172.21.15.31:0/1500104341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6a7402d3c0 con 0x7f6a74104b30 2024-04-25T01:15:14.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.692+0000 7f6a71600640 1 -- 172.21.15.31:0/1500104341 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f6a5c0c0100 con 0x7f6a74104b30 2024-04-25T01:15:14.678 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-04-25T01:15:14.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.695+0000 7f6a7b800640 1 -- 172.21.15.31:0/1500104341 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6a580a2c10 msgr2=0x7f6a580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:14.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.695+0000 7f6a7b800640 1 --2- 172.21.15.31:0/1500104341 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6a580a2c10 0x7f6a580a50d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6a64009bd0 tx=0x7f6a64005f90 comp rx=0 tx=0).stop 2024-04-25T01:15:14.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.695+0000 7f6a7b800640 1 -- 172.21.15.31:0/1500104341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a74104b30 msgr2=0x7f6a7419a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:14.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.695+0000 7f6a7b800640 1 --2- 172.21.15.31:0/1500104341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a74104b30 0x7f6a7419a280 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c031440 tx=0x7f6a5c032ce0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.695+0000 7f6a7b800640 1 -- 172.21.15.31:0/1500104341 shutdown_connections 2024-04-25T01:15:14.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.695+0000 7f6a7b800640 1 --2- 172.21.15.31:0/1500104341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a74107460 0x7f6a741a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.695+0000 7f6a7b800640 1 --2- 172.21.15.31:0/1500104341 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6a580a2c10 0x7f6a580a50d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.695+0000 7f6a7b800640 1 --2- 172.21.15.31:0/1500104341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a74104b30 0x7f6a7419a280 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.695+0000 7f6a7b800640 1 --2- 172.21.15.31:0/1500104341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a74102200 0x7f6a74199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.695+0000 7f6a7b800640 1 -- 172.21.15.31:0/1500104341 >> 172.21.15.31:0/1500104341 conn(0x7f6a740fbbf0 msgr2=0x7f6a74101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:14.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.695+0000 7f6a7b800640 1 -- 172.21.15.31:0/1500104341 shutdown_connections 2024-04-25T01:15:14.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.696+0000 7f6a7b800640 1 -- 172.21.15.31:0/1500104341 wait complete. 2024-04-25T01:15:14.701 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:15:14.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.923+0000 7fc68be00640 1 -- 172.21.15.31:0/134407335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc68c1004c0 msgr2=0x7fc68c1008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:14.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.923+0000 7fc68be00640 1 --2- 172.21.15.31:0/134407335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc68c1004c0 0x7fc68c1008e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009900 tx=0x7fc67c02a3f0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.923+0000 7fc68be00640 1 -- 172.21.15.31:0/134407335 shutdown_connections 2024-04-25T01:15:14.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.923+0000 7fc68be00640 1 --2- 172.21.15.31:0/134407335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc68c102070 0x7fc68c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.923+0000 7fc68be00640 1 --2- 172.21.15.31:0/134407335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc68c1016b0 0x7fc68c101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.923+0000 7fc68be00640 1 --2- 172.21.15.31:0/134407335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc68c1004c0 0x7fc68c1008e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.923+0000 7fc68be00640 1 -- 172.21.15.31:0/134407335 >> 172.21.15.31:0/134407335 conn(0x7fc68c0fba10 msgr2=0x7fc68c0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:14.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.923+0000 7fc68be00640 1 -- 172.21.15.31:0/134407335 shutdown_connections 2024-04-25T01:15:14.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.923+0000 7fc68be00640 1 -- 172.21.15.31:0/134407335 wait complete. 2024-04-25T01:15:14.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.924+0000 7fc68be00640 1 Processor -- start 2024-04-25T01:15:14.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.924+0000 7fc68be00640 1 -- start start 2024-04-25T01:15:14.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.925+0000 7fc68be00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc68c1004c0 0x7fc68c199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:14.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.925+0000 7fc68be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc68c1016b0 0x7fc68c19a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:14.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.925+0000 7fc68be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc68c102070 0x7fc68c1a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:14.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.925+0000 7fc68be00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c10c1a0 con 0x7fc68c1004c0 2024-04-25T01:15:14.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.925+0000 7fc68be00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c10c020 con 0x7fc68c102070 2024-04-25T01:15:14.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.925+0000 7fc68be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c10c320 con 0x7fc68c1016b0 2024-04-25T01:15:14.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.925+0000 7fc68aa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc68c1004c0 0x7fc68c199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:14.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.925+0000 7fc68aa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc68c1004c0 0x7fc68c199c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46138/0 (socket says 172.21.15.31:46138) 2024-04-25T01:15:14.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.925+0000 7fc68aa00640 1 -- 172.21.15.31:0/4058795973 learned_addr learned my addr 172.21.15.31:0/4058795973 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:14.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.925+0000 7fc68b400640 1 --2- 172.21.15.31:0/4058795973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc68c102070 0x7fc68c1a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:14.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.925+0000 7fc68a000640 1 --2- 172.21.15.31:0/4058795973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc68c1016b0 0x7fc68c19a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:14.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.925+0000 7fc68aa00640 1 -- 172.21.15.31:0/4058795973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc68c1016b0 msgr2=0x7fc68c19a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:14.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.925+0000 7fc68aa00640 1 --2- 172.21.15.31:0/4058795973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc68c1016b0 0x7fc68c19a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.925+0000 7fc68aa00640 1 -- 172.21.15.31:0/4058795973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc68c102070 msgr2=0x7fc68c1a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:14.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.925+0000 7fc68aa00640 1 --2- 172.21.15.31:0/4058795973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc68c102070 0x7fc68c1a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:14.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.925+0000 7fc68aa00640 1 -- 172.21.15.31:0/4058795973 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68c1a1770 con 0x7fc68c1004c0 2024-04-25T01:15:14.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.925+0000 7fc68a000640 1 --2- 172.21.15.31:0/4058795973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc68c1016b0 0x7fc68c19a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:15:14.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.926+0000 7fc68aa00640 1 --2- 172.21.15.31:0/4058795973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc68c1004c0 0x7fc68c199c10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc67c031440 tx=0x7fc67c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:14.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.926+0000 7fc683400640 1 -- 172.21.15.31:0/4058795973 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c03f030 con 0x7fc68c1004c0 2024-04-25T01:15:14.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.926+0000 7fc68be00640 1 -- 172.21.15.31:0/4058795973 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc68c1a1a00 con 0x7fc68c1004c0 2024-04-25T01:15:14.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.926+0000 7fc683400640 1 -- 172.21.15.31:0/4058795973 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc67c003ce0 con 0x7fc68c1004c0 2024-04-25T01:15:14.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.926+0000 7fc683400640 1 -- 172.21.15.31:0/4058795973 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c035070 con 0x7fc68c1004c0 2024-04-25T01:15:14.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.926+0000 7fc68be00640 1 -- 172.21.15.31:0/4058795973 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc68c1a1f40 con 0x7fc68c1004c0 2024-04-25T01:15:14.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.927+0000 7fc68be00640 1 -- 172.21.15.31:0/4058795973 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc68c1a2110 con 0x7fc68c1004c0 2024-04-25T01:15:14.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.929+0000 7fc683400640 1 -- 172.21.15.31:0/4058795973 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc67c042030 con 0x7fc68c1004c0 2024-04-25T01:15:14.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.931+0000 7fc683400640 1 --2- 172.21.15.31:0/4058795973 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc6700a2b90 0x7fc6700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:14.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.931+0000 7fc683400640 1 -- 172.21.15.31:0/4058795973 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fc67c0ef560 con 0x7fc68c1004c0 2024-04-25T01:15:14.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.931+0000 7fc68a000640 1 --2- 172.21.15.31:0/4058795973 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc6700a2b90 0x7fc6700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:14.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.932+0000 7fc68a000640 1 --2- 172.21.15.31:0/4058795973 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc6700a2b90 0x7fc6700a5050 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc68c19b060 tx=0x7fc678005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:14.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:14.935+0000 7fc683400640 1 -- 172.21.15.31:0/4058795973 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc67c0bc4b0 con 0x7fc68c1004c0 2024-04-25T01:15:15.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.186+0000 7fc68be00640 1 -- 172.21.15.31:0/4058795973 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc68c1a22e0 con 0x7fc68c1004c0 2024-04-25T01:15:15.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.188+0000 7fc683400640 1 -- 172.21.15.31:0/4058795973 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11854 (secure 0 0 0) 0x7fc67c0bbdb0 con 0x7fc68c1004c0 2024-04-25T01:15:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:15:15.174 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/498218714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":498218714},{"type":"v1","addr":"172.21.15.31:6835","nonce":498218714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2647199475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2647199475},{"type":"v1","addr":"172.21.15.31:6839","nonce":2647199475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/30518311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":30518311},{"type":"v1","addr":"172.21.15.31:6841","nonce":30518311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/420362823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":420362823},{"type":"v1","addr":"172.21.15.169:6833","nonce":420362823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1943710438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1943710438},{"type":"v1","addr":"172.21.15.40:6833","nonce":1943710438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/4191344115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":4191344115},{"type":"v1","addr":"172.21.15.31:6837","nonce":4191344115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:13:30.975848+0000","modified":"2024-04-25T01:13:36.545846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24565,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6835/2916305801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2916305801},{"type":"v1","addr":"172.21.15.40:6835","nonce":2916305801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.169:6835/2253264784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2253264784},{"type":"v1","addr":"172.21.15.169:6835","nonce":2253264784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6837/2883308641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2883308641},{"type":"v1","addr":"172.21.15.169:6837","nonce":2883308641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/590109562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":590109562},{"type":"v1","addr":"172.21.15.40:6837","nonce":590109562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/4185710218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":4185710218},{"type":"v1","addr":"172.21.15.169:6839","nonce":4185710218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6839/927194004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":927194004},{"type":"v1","addr":"172.21.15.40:6839","nonce":927194004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24532,24565,24596]},"id":1}]} 2024-04-25T01:15:15.174 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 23 2024-04-25T01:15:15.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.190+0000 7fc68be00640 1 -- 172.21.15.31:0/4058795973 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc6700a2b90 msgr2=0x7fc6700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:15.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.190+0000 7fc68be00640 1 --2- 172.21.15.31:0/4058795973 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc6700a2b90 0x7fc6700a5050 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc68c19b060 tx=0x7fc678005f90 comp rx=0 tx=0).stop 2024-04-25T01:15:15.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.190+0000 7fc68be00640 1 -- 172.21.15.31:0/4058795973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc68c1004c0 msgr2=0x7fc68c199c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:15.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.190+0000 7fc68be00640 1 --2- 172.21.15.31:0/4058795973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc68c1004c0 0x7fc68c199c10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc67c031440 tx=0x7fc67c032ce0 comp rx=0 tx=0).stop 2024-04-25T01:15:15.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.190+0000 7fc68be00640 1 -- 172.21.15.31:0/4058795973 shutdown_connections 2024-04-25T01:15:15.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.191+0000 7fc68be00640 1 --2- 172.21.15.31:0/4058795973 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc6700a2b90 0x7fc6700a5050 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:15.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.191+0000 7fc68be00640 1 --2- 172.21.15.31:0/4058795973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc68c102070 0x7fc68c1a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:15.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.191+0000 7fc68be00640 1 --2- 172.21.15.31:0/4058795973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc68c1016b0 0x7fc68c19a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:15.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.191+0000 7fc68be00640 1 --2- 172.21.15.31:0/4058795973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc68c1004c0 0x7fc68c199c10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:15.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.191+0000 7fc68be00640 1 -- 172.21.15.31:0/4058795973 >> 172.21.15.31:0/4058795973 conn(0x7fc68c0fba10 msgr2=0x7fc68c0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:15.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.191+0000 7fc68be00640 1 -- 172.21.15.31:0/4058795973 shutdown_connections 2024-04-25T01:15:15.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.191+0000 7fc68be00640 1 -- 172.21.15.31:0/4058795973 wait complete. 2024-04-25T01:15:15.194 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:15:15.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.417+0000 7f8b5ea00640 1 -- 172.21.15.31:0/3245197128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b581049b0 msgr2=0x7f8b58104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:15.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.417+0000 7f8b5ea00640 1 --2- 172.21.15.31:0/3245197128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b581049b0 0x7f8b58104e30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009900 tx=0x7f8b4802a3f0 comp rx=0 tx=0).stop 2024-04-25T01:15:15.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.417+0000 7f8b5ea00640 1 -- 172.21.15.31:0/3245197128 shutdown_connections 2024-04-25T01:15:15.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.417+0000 7f8b5ea00640 1 --2- 172.21.15.31:0/3245197128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b58105370 0x7f8b5810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:15.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.417+0000 7f8b5ea00640 1 --2- 172.21.15.31:0/3245197128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b581049b0 0x7f8b58104e30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:15.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.417+0000 7f8b5ea00640 1 --2- 172.21.15.31:0/3245197128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b581037c0 0x7f8b58103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:15.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.417+0000 7f8b5ea00640 1 -- 172.21.15.31:0/3245197128 >> 172.21.15.31:0/3245197128 conn(0x7f8b580fed50 msgr2=0x7f8b58101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:15.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.417+0000 7f8b5ea00640 1 -- 172.21.15.31:0/3245197128 shutdown_connections 2024-04-25T01:15:15.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.417+0000 7f8b5ea00640 1 -- 172.21.15.31:0/3245197128 wait complete. 2024-04-25T01:15:15.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.419+0000 7f8b5ea00640 1 Processor -- start 2024-04-25T01:15:15.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.419+0000 7f8b5ea00640 1 -- start start 2024-04-25T01:15:15.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.419+0000 7f8b5ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b581037c0 0x7f8b58199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:15.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.419+0000 7f8b5ea00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b581049b0 0x7f8b58199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:15.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.419+0000 7f8b5ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b58105370 0x7f8b581a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:15.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.419+0000 7f8b5ea00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5810f420 con 0x7f8b581049b0 2024-04-25T01:15:15.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.419+0000 7f8b5ea00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5810f2a0 con 0x7f8b581037c0 2024-04-25T01:15:15.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.419+0000 7f8b5ea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5810f5a0 con 0x7f8b58105370 2024-04-25T01:15:15.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.420+0000 7f8b5cc00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b581049b0 0x7f8b58199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:15.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.420+0000 7f8b5d600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b581037c0 0x7f8b58199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:15.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.420+0000 7f8b5cc00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b581049b0 0x7f8b58199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46162/0 (socket says 172.21.15.31:46162) 2024-04-25T01:15:15.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.420+0000 7f8b5cc00640 1 -- 172.21.15.31:0/1612668196 learned_addr learned my addr 172.21.15.31:0/1612668196 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:15.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.420+0000 7f8b5e000640 1 --2- 172.21.15.31:0/1612668196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b58105370 0x7f8b581a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:15.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.420+0000 7f8b5cc00640 1 -- 172.21.15.31:0/1612668196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b58105370 msgr2=0x7f8b581a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:15.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.420+0000 7f8b5cc00640 1 --2- 172.21.15.31:0/1612668196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b58105370 0x7f8b581a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:15.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.420+0000 7f8b5cc00640 1 -- 172.21.15.31:0/1612668196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b581037c0 msgr2=0x7f8b58199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:15.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.420+0000 7f8b5cc00640 1 --2- 172.21.15.31:0/1612668196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b581037c0 0x7f8b58199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:15.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.420+0000 7f8b5cc00640 1 -- 172.21.15.31:0/1612668196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b581a1690 con 0x7f8b581049b0 2024-04-25T01:15:15.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.420+0000 7f8b5d600640 1 --2- 172.21.15.31:0/1612668196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b581037c0 0x7f8b58199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:15.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.420+0000 7f8b5cc00640 1 --2- 172.21.15.31:0/1612668196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b581049b0 0x7f8b58199fb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009230 tx=0x7f8b480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:15.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.421+0000 7f8b56000640 1 -- 172.21.15.31:0/1612668196 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4803f030 con 0x7f8b581049b0 2024-04-25T01:15:15.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.421+0000 7f8b5ea00640 1 -- 172.21.15.31:0/1612668196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b581a1920 con 0x7f8b581049b0 2024-04-25T01:15:15.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.421+0000 7f8b56000640 1 -- 172.21.15.31:0/1612668196 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b48003ce0 con 0x7f8b581049b0 2024-04-25T01:15:15.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.421+0000 7f8b56000640 1 -- 172.21.15.31:0/1612668196 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b48035070 con 0x7f8b581049b0 2024-04-25T01:15:15.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.421+0000 7f8b5ea00640 1 -- 172.21.15.31:0/1612668196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b581a1e60 con 0x7f8b581049b0 2024-04-25T01:15:15.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.422+0000 7f8b5ea00640 1 -- 172.21.15.31:0/1612668196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b5810aae0 con 0x7f8b581049b0 2024-04-25T01:15:15.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.425+0000 7f8b56000640 1 -- 172.21.15.31:0/1612668196 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8b48042030 con 0x7f8b581049b0 2024-04-25T01:15:15.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.426+0000 7f8b56000640 1 --2- 172.21.15.31:0/1612668196 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f8b400a2c10 0x7f8b400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:15.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.426+0000 7f8b56000640 1 -- 172.21.15.31:0/1612668196 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f8b480ef420 con 0x7f8b581049b0 2024-04-25T01:15:15.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.427+0000 7f8b5d600640 1 --2- 172.21.15.31:0/1612668196 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f8b400a2c10 0x7f8b400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:15.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.428+0000 7f8b5d600640 1 --2- 172.21.15.31:0/1612668196 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f8b400a2c10 0x7f8b400a50d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8b58104810 tx=0x7f8b4c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:15.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.432+0000 7f8b56000640 1 -- 172.21.15.31:0/1612668196 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8b480bbd20 con 0x7f8b581049b0 2024-04-25T01:15:15.444 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:15 smithi169 ceph-mon[32759]: pgmap v370: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T01:15:15.444 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:15 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3220412232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:15:15.444 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:15 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1500104341' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:15:15.447 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:15 smithi040 ceph-mon[32518]: pgmap v370: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T01:15:15.447 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:15 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3220412232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:15:15.447 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:15 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1500104341' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:15:15.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:15 smithi031 ceph-mon[28053]: pgmap v370: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T01:15:15.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:15 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3220412232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:15:15.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:15 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1500104341' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:15:15.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.641+0000 7f8b5ea00640 1 -- 172.21.15.31:0/1612668196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b58062340 con 0x7f8b581049b0 2024-04-25T01:15:15.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.644+0000 7f8b56000640 1 -- 172.21.15.31:0/1612668196 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22530 (secure 0 0 0) 0x7f8b480c0020 con 0x7f8b581049b0 2024-04-25T01:15:15.630 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:15:15.631 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":89,"fsid":"95849b94-029f-11ef-bc93-c7b262605968","created":"2024-04-25T01:04:23.310856+0000","modified":"2024-04-25T01:14:33.793638+0000","last_up_change":"2024-04-25T01:12:29.393907+0000","last_in_change":"2024-04-25T01:12:08.978148+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:18.662391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:13:29.258771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:13:30.480996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa08e4c-fe6b-4116-bdf5-1adf5b465d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6803","nonce":788174091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6805","nonce":788174091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6809","nonce":788174091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6807","nonce":788174091}]},"public_addr":"172.21.15.31:6803/788174091","cluster_addr":"172.21.15.31:6805/788174091","heartbeat_back_addr":"172.21.15.31:6809/788174091","heartbeat_front_addr":"172.21.15.31:6807/788174091","state":["exists","up"]},{"osd":1,"uuid":"85339fe7-0dbc-4485-9334-2b039993c1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6801","nonce":3196394509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6803","nonce":3196394509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6807","nonce":3196394509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6805","nonce":3196394509}]},"public_addr":"172.21.15.40:6801/3196394509","cluster_addr":"172.21.15.40:6803/3196394509","heartbeat_back_addr":"172.21.15.40:6807/3196394509","heartbeat_front_addr":"172.21.15.40:6805/3196394509","state":["exists","up"]},{"osd":2,"uuid":"ab62ecb2-8aae-4987-9e2e-589e0f2fee5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6801","nonce":1955211197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6803","nonce":1955211197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6807","nonce":1955211197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6805","nonce":1955211197}]},"public_addr":"172.21.15.169:6801/1955211197","cluster_addr":"172.21.15.169:6803/1955211197","heartbeat_back_addr":"172.21.15.169:6807/1955211197","heartbeat_front_addr":"172.21.15.169:6805/1955211197","state":["exists","up"]},{"osd":3,"uuid":"8b5a7881-3401-495c-8ca4-c90685919d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6811","nonce":1228688643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6813","nonce":1228688643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6817","nonce":1228688643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6815","nonce":1228688643}]},"public_addr":"172.21.15.31:6811/1228688643","cluster_addr":"172.21.15.31:6813/1228688643","heartbeat_back_addr":"172.21.15.31:6817/1228688643","heartbeat_front_addr":"172.21.15.31:6815/1228688643","state":["exists","up"]},{"osd":4,"uuid":"56deb61d-a334-4cf2-8ac7-fbb6e6947092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6809","nonce":1913779091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6811","nonce":1913779091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6815","nonce":1913779091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6813","nonce":1913779091}]},"public_addr":"172.21.15.40:6809/1913779091","cluster_addr":"172.21.15.40:6811/1913779091","heartbeat_back_addr":"172.21.15.40:6815/1913779091","heartbeat_front_addr":"172.21.15.40:6813/1913779091","state":["exists","up"]},{"osd":5,"uuid":"8da55693-f9c7-49a7-99e0-165078f8001b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6809","nonce":2549650781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6811","nonce":2549650781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6815","nonce":2549650781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6813","nonce":2549650781}]},"public_addr":"172.21.15.169:6809/2549650781","cluster_addr":"172.21.15.169:6811/2549650781","heartbeat_back_addr":"172.21.15.169:6815/2549650781","heartbeat_front_addr":"172.21.15.169:6813/2549650781","state":["exists","up"]},{"osd":6,"uuid":"fc8350d2-1ad6-43cb-89f4-d336afc99ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6819","nonce":611507624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6821","nonce":611507624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6825","nonce":611507624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6823","nonce":611507624}]},"public_addr":"172.21.15.31:6819/611507624","cluster_addr":"172.21.15.31:6821/611507624","heartbeat_back_addr":"172.21.15.31:6825/611507624","heartbeat_front_addr":"172.21.15.31:6823/611507624","state":["exists","up"]},{"osd":7,"uuid":"3121b58b-8c94-40b5-9d48-f56552d7c39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6817","nonce":3776085849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6819","nonce":3776085849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6823","nonce":3776085849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6821","nonce":3776085849}]},"public_addr":"172.21.15.40:6817/3776085849","cluster_addr":"172.21.15.40:6819/3776085849","heartbeat_back_addr":"172.21.15.40:6823/3776085849","heartbeat_front_addr":"172.21.15.40:6821/3776085849","state":["exists","up"]},{"osd":8,"uuid":"f60407df-91f5-477a-9702-ddaef24faf82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6817","nonce":2736546122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6819","nonce":2736546122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6823","nonce":2736546122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6821","nonce":2736546122}]},"public_addr":"172.21.15.169:6817/2736546122","cluster_addr":"172.21.15.169:6819/2736546122","heartbeat_back_addr":"172.21.15.169:6823/2736546122","heartbeat_front_addr":"172.21.15.169:6821/2736546122","state":["exists","up"]},{"osd":9,"uuid":"7506a407-1960-49bc-9e38-7f2a8bf996ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6827","nonce":1187207880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6829","nonce":1187207880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6833","nonce":1187207880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6831","nonce":1187207880}]},"public_addr":"172.21.15.31:6827/1187207880","cluster_addr":"172.21.15.31:6829/1187207880","heartbeat_back_addr":"172.21.15.31:6833/1187207880","heartbeat_front_addr":"172.21.15.31:6831/1187207880","state":["exists","up"]},{"osd":10,"uuid":"831c48e2-b0d0-451f-959b-228e927542cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6825","nonce":3050148001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6827","nonce":3050148001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6831","nonce":3050148001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6829","nonce":3050148001}]},"public_addr":"172.21.15.40:6825/3050148001","cluster_addr":"172.21.15.40:6827/3050148001","heartbeat_back_addr":"172.21.15.40:6831/3050148001","heartbeat_front_addr":"172.21.15.40:6829/3050148001","state":["exists","up"]},{"osd":11,"uuid":"144fe0aa-f757-4b3c-90ad-99775e17d57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6825","nonce":262401944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6827","nonce":262401944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6831","nonce":262401944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6829","nonce":262401944}]},"public_addr":"172.21.15.169:6825/262401944","cluster_addr":"172.21.15.169:6827/262401944","heartbeat_back_addr":"172.21.15.169:6831/262401944","heartbeat_front_addr":"172.21.15.169:6829/262401944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:25.640995+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:51.036909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.635633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:48.467912+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:14.454971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:37.609315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:12.441807+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:38.239215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:02.230758+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:35.762553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:02.051405+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:26.569593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6801/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4004014066":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/1930101646":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/2638309002":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2238638505":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2902841144":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4190696323":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/480558521":"2024-04-26T01:04:57.149272+0000","172.21.15.31:6801/2638309002":"2024-04-26T01:04:57.149272+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:15:15.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.647+0000 7f8b5ea00640 1 -- 172.21.15.31:0/1612668196 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f8b400a2c10 msgr2=0x7f8b400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:15.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.647+0000 7f8b5ea00640 1 --2- 172.21.15.31:0/1612668196 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f8b400a2c10 0x7f8b400a50d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8b58104810 tx=0x7f8b4c009500 comp rx=0 tx=0).stop 2024-04-25T01:15:15.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.647+0000 7f8b5ea00640 1 -- 172.21.15.31:0/1612668196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b581049b0 msgr2=0x7f8b58199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:15.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.647+0000 7f8b5ea00640 1 --2- 172.21.15.31:0/1612668196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b581049b0 0x7f8b58199fb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009230 tx=0x7f8b480042f0 comp rx=0 tx=0).stop 2024-04-25T01:15:15.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.647+0000 7f8b5ea00640 1 -- 172.21.15.31:0/1612668196 shutdown_connections 2024-04-25T01:15:15.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.647+0000 7f8b5ea00640 1 --2- 172.21.15.31:0/1612668196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b58105370 0x7f8b581a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:15.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.647+0000 7f8b5ea00640 1 --2- 172.21.15.31:0/1612668196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b581049b0 0x7f8b58199fb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:15.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.647+0000 7f8b5ea00640 1 --2- 172.21.15.31:0/1612668196 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f8b400a2c10 0x7f8b400a50d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:15.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.647+0000 7f8b5ea00640 1 --2- 172.21.15.31:0/1612668196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b581037c0 0x7f8b58199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:15.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.647+0000 7f8b5ea00640 1 -- 172.21.15.31:0/1612668196 >> 172.21.15.31:0/1612668196 conn(0x7f8b580fed50 msgr2=0x7f8b58107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:15.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.648+0000 7f8b5ea00640 1 -- 172.21.15.31:0/1612668196 shutdown_connections 2024-04-25T01:15:15.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.648+0000 7f8b5ea00640 1 -- 172.21.15.31:0/1612668196 wait complete. 2024-04-25T01:15:15.655 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:15:15.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.879+0000 7fbaf3e00640 1 -- 172.21.15.31:0/1962440962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaf41037c0 msgr2=0x7fbaf4103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:15.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.879+0000 7fbaf3e00640 1 --2- 172.21.15.31:0/1962440962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaf41037c0 0x7fbaf4103be0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbae4009900 tx=0x7fbae402a410 comp rx=0 tx=0).stop 2024-04-25T01:15:15.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.879+0000 7fbaf3e00640 1 -- 172.21.15.31:0/1962440962 shutdown_connections 2024-04-25T01:15:15.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.879+0000 7fbaf3e00640 1 --2- 172.21.15.31:0/1962440962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf4105370 0x7fbaf410bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:15.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.879+0000 7fbaf3e00640 1 --2- 172.21.15.31:0/1962440962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbaf41049b0 0x7fbaf4104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:15.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.879+0000 7fbaf3e00640 1 --2- 172.21.15.31:0/1962440962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaf41037c0 0x7fbaf4103be0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:15.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.879+0000 7fbaf3e00640 1 -- 172.21.15.31:0/1962440962 >> 172.21.15.31:0/1962440962 conn(0x7fbaf40fed50 msgr2=0x7fbaf4101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:15.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.879+0000 7fbaf3e00640 1 -- 172.21.15.31:0/1962440962 shutdown_connections 2024-04-25T01:15:15.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.879+0000 7fbaf3e00640 1 -- 172.21.15.31:0/1962440962 wait complete. 2024-04-25T01:15:15.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.880+0000 7fbaf3e00640 1 Processor -- start 2024-04-25T01:15:15.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.880+0000 7fbaf3e00640 1 -- start start 2024-04-25T01:15:15.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.881+0000 7fbaf3e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaf41037c0 0x7fbaf4199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:15.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.881+0000 7fbaf3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbaf41049b0 0x7fbaf419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:15.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.881+0000 7fbaf3e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf4105370 0x7fbaf41a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:15.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.881+0000 7fbaf3e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf410f460 con 0x7fbaf41037c0 2024-04-25T01:15:15.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.881+0000 7fbaf3e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf410f2e0 con 0x7fbaf4105370 2024-04-25T01:15:15.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.881+0000 7fbaf3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf410f5e0 con 0x7fbaf41049b0 2024-04-25T01:15:15.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.881+0000 7fbaf2a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaf41037c0 0x7fbaf4199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:15.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.881+0000 7fbaf2a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaf41037c0 0x7fbaf4199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46184/0 (socket says 172.21.15.31:46184) 2024-04-25T01:15:15.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.881+0000 7fbaf2a00640 1 -- 172.21.15.31:0/3966636429 learned_addr learned my addr 172.21.15.31:0/3966636429 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:15.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.881+0000 7fbaea000640 1 --2- 172.21.15.31:0/3966636429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbaf41049b0 0x7fbaf419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:15.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.881+0000 7fbaf3400640 1 --2- 172.21.15.31:0/3966636429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf4105370 0x7fbaf41a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:15.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.881+0000 7fbaf2a00640 1 -- 172.21.15.31:0/3966636429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbaf41049b0 msgr2=0x7fbaf419a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:15.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.881+0000 7fbaf2a00640 1 --2- 172.21.15.31:0/3966636429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbaf41049b0 0x7fbaf419a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:15.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.881+0000 7fbaf2a00640 1 -- 172.21.15.31:0/3966636429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf4105370 msgr2=0x7fbaf41a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:15.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.881+0000 7fbaf2a00640 1 --2- 172.21.15.31:0/3966636429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf4105370 0x7fbaf41a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:15.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.881+0000 7fbaf2a00640 1 -- 172.21.15.31:0/3966636429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf41a16f0 con 0x7fbaf41037c0 2024-04-25T01:15:15.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.881+0000 7fbaf3400640 1 --2- 172.21.15.31:0/3966636429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf4105370 0x7fbaf41a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:15:15.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.882+0000 7fbaf2a00640 1 --2- 172.21.15.31:0/3966636429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaf41037c0 0x7fbaf4199ad0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbae4031440 tx=0x7fbae4032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:15.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.882+0000 7fbaebe00640 1 -- 172.21.15.31:0/3966636429 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae403f030 con 0x7fbaf41037c0 2024-04-25T01:15:15.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.882+0000 7fbaebe00640 1 -- 172.21.15.31:0/3966636429 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae4003ce0 con 0x7fbaf41037c0 2024-04-25T01:15:15.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.882+0000 7fbaebe00640 1 -- 172.21.15.31:0/3966636429 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae4035070 con 0x7fbaf41037c0 2024-04-25T01:15:15.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.882+0000 7fbaf3e00640 1 -- 172.21.15.31:0/3966636429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf41a1980 con 0x7fbaf41037c0 2024-04-25T01:15:15.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.882+0000 7fbaf3e00640 1 -- 172.21.15.31:0/3966636429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf41a1e70 con 0x7fbaf41037c0 2024-04-25T01:15:15.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.883+0000 7fbaf3e00640 1 -- 172.21.15.31:0/3966636429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaf410aae0 con 0x7fbaf41037c0 2024-04-25T01:15:15.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.885+0000 7fbaebe00640 1 -- 172.21.15.31:0/3966636429 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbae4042030 con 0x7fbaf41037c0 2024-04-25T01:15:15.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.887+0000 7fbaebe00640 1 --2- 172.21.15.31:0/3966636429 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbad80a2b90 0x7fbad80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:15.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.887+0000 7fbaebe00640 1 -- 172.21.15.31:0/3966636429 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fbae40ef610 con 0x7fbaf41037c0 2024-04-25T01:15:15.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.887+0000 7fbaea000640 1 --2- 172.21.15.31:0/3966636429 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbad80a2b90 0x7fbad80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:15.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.888+0000 7fbaea000640 1 --2- 172.21.15.31:0/3966636429 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbad80a2b90 0x7fbad80a5050 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbaf419afe0 tx=0x7fbadc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:15.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:15.891+0000 7fbaebe00640 1 -- 172.21.15.31:0/3966636429 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbae40bc560 con 0x7fbaf41037c0 2024-04-25T01:15:16.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.140+0000 7fbaf3e00640 1 -- 172.21.15.31:0/3966636429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbaf4062370 con 0x7fbaf41037c0 2024-04-25T01:15:16.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.142+0000 7fbaebe00640 1 -- 172.21.15.31:0/3966636429 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11854 (secure 0 0 0) 0x7fbae40bbe60 con 0x7fbaf41037c0 2024-04-25T01:15:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:15:16.128 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/498218714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":498218714},{"type":"v1","addr":"172.21.15.31:6835","nonce":498218714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2647199475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2647199475},{"type":"v1","addr":"172.21.15.31:6839","nonce":2647199475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/30518311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":30518311},{"type":"v1","addr":"172.21.15.31:6841","nonce":30518311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/420362823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":420362823},{"type":"v1","addr":"172.21.15.169:6833","nonce":420362823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1943710438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1943710438},{"type":"v1","addr":"172.21.15.40:6833","nonce":1943710438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/4191344115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":4191344115},{"type":"v1","addr":"172.21.15.31:6837","nonce":4191344115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:13:30.975848+0000","modified":"2024-04-25T01:13:36.545846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24565,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6835/2916305801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2916305801},{"type":"v1","addr":"172.21.15.40:6835","nonce":2916305801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.169:6835/2253264784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2253264784},{"type":"v1","addr":"172.21.15.169:6835","nonce":2253264784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6837/2883308641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2883308641},{"type":"v1","addr":"172.21.15.169:6837","nonce":2883308641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/590109562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":590109562},{"type":"v1","addr":"172.21.15.40:6837","nonce":590109562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/4185710218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":4185710218},{"type":"v1","addr":"172.21.15.169:6839","nonce":4185710218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6839/927194004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":927194004},{"type":"v1","addr":"172.21.15.40:6839","nonce":927194004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24532,24565,24596]},"id":1}]} 2024-04-25T01:15:16.128 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 23 2024-04-25T01:15:16.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.144+0000 7fbaf3e00640 1 -- 172.21.15.31:0/3966636429 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbad80a2b90 msgr2=0x7fbad80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.144+0000 7fbaf3e00640 1 --2- 172.21.15.31:0/3966636429 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbad80a2b90 0x7fbad80a5050 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbaf419afe0 tx=0x7fbadc005f90 comp rx=0 tx=0).stop 2024-04-25T01:15:16.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.145+0000 7fbaf3e00640 1 -- 172.21.15.31:0/3966636429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaf41037c0 msgr2=0x7fbaf4199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.145+0000 7fbaf3e00640 1 --2- 172.21.15.31:0/3966636429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaf41037c0 0x7fbaf4199ad0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbae4031440 tx=0x7fbae4032ce0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.145+0000 7fbaf3e00640 1 -- 172.21.15.31:0/3966636429 shutdown_connections 2024-04-25T01:15:16.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.145+0000 7fbaf3e00640 1 --2- 172.21.15.31:0/3966636429 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbad80a2b90 0x7fbad80a5050 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.145+0000 7fbaf3e00640 1 --2- 172.21.15.31:0/3966636429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf4105370 0x7fbaf41a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.145+0000 7fbaf3e00640 1 --2- 172.21.15.31:0/3966636429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbaf41049b0 0x7fbaf419a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.145+0000 7fbaf3e00640 1 --2- 172.21.15.31:0/3966636429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaf41037c0 0x7fbaf4199ad0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.145+0000 7fbaf3e00640 1 -- 172.21.15.31:0/3966636429 >> 172.21.15.31:0/3966636429 conn(0x7fbaf40fed50 msgr2=0x7fbaf4107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:16.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.145+0000 7fbaf3e00640 1 -- 172.21.15.31:0/3966636429 shutdown_connections 2024-04-25T01:15:16.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.145+0000 7fbaf3e00640 1 -- 172.21.15.31:0/3966636429 wait complete. 2024-04-25T01:15:16.151 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T01:13:30.975848+0000', 'modified': '2024-04-25T01:13:36.545846+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24596, 'mds_1': 24565, 'mds_2': 24532}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24518': {'gid': 24518, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.40:6835/2916305801', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6834', 'nonce': 2916305801}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 2916305801}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24532': {'gid': 24532, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.169:6835/2253264784', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 2253264784}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 2253264784}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24544': {'gid': 24544, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.169:6837/2883308641', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6836', 'nonce': 2883308641}, {'type': 'v1', 'addr': '172.21.15.169:6837', 'nonce': 2883308641}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24548': {'gid': 24548, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.40:6837/590109562', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6836', 'nonce': 590109562}, {'type': 'v1', 'addr': '172.21.15.40:6837', 'nonce': 590109562}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24565': {'gid': 24565, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.169:6839/4185710218', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6838', 'nonce': 4185710218}, {'type': 'v1', 'addr': '172.21.15.169:6839', 'nonce': 4185710218}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24596': {'gid': 24596, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6839/927194004', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6838', 'nonce': 927194004}, {'type': 'v1', 'addr': '172.21.15.40:6839', 'nonce': 927194004}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24596, 'qdb_cluster': [24532, 24565, 24596]} 2024-04-25T01:15:16.151 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T01:15:16.151 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-25T01:15:16.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.378+0000 7f9080e00640 1 -- 172.21.15.31:0/2191724736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f907c108e70 msgr2=0x7f907c1092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.378+0000 7f9080e00640 1 --2- 172.21.15.31:0/2191724736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f907c108e70 0x7f907c1092f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9064009900 tx=0x7f906402a3f0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.378+0000 7f9080e00640 1 -- 172.21.15.31:0/2191724736 shutdown_connections 2024-04-25T01:15:16.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.378+0000 7f9080e00640 1 --2- 172.21.15.31:0/2191724736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907c109830 0x7f907c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.378+0000 7f9080e00640 1 --2- 172.21.15.31:0/2191724736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f907c108e70 0x7f907c1092f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.378+0000 7f9080e00640 1 --2- 172.21.15.31:0/2191724736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f907c107c80 0x7f907c1080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.378+0000 7f9080e00640 1 -- 172.21.15.31:0/2191724736 >> 172.21.15.31:0/2191724736 conn(0x7f907c076c10 msgr2=0x7f907c079050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:16.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.379+0000 7f9080e00640 1 -- 172.21.15.31:0/2191724736 shutdown_connections 2024-04-25T01:15:16.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.379+0000 7f9080e00640 1 -- 172.21.15.31:0/2191724736 wait complete. 2024-04-25T01:15:16.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.380+0000 7f9080e00640 1 Processor -- start 2024-04-25T01:15:16.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.380+0000 7f9080e00640 1 -- start start 2024-04-25T01:15:16.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.380+0000 7f9080e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f907c107c80 0x7f907c19df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.380+0000 7f9080e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907c108e70 0x7f907c19e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.380+0000 7f9080e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f907c109830 0x7f907c1a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.380+0000 7f9080e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f907c113900 con 0x7f907c109830 2024-04-25T01:15:16.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.380+0000 7f9080e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f907c113780 con 0x7f907c108e70 2024-04-25T01:15:16.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.380+0000 7f9080e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f907c113a80 con 0x7f907c107c80 2024-04-25T01:15:16.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.381+0000 7f907be00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f907c109830 0x7f907c1a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.381+0000 7f907be00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f907c109830 0x7f907c1a54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46196/0 (socket says 172.21.15.31:46196) 2024-04-25T01:15:16.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.381+0000 7f907be00640 1 -- 172.21.15.31:0/2091701283 learned_addr learned my addr 172.21.15.31:0/2091701283 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:16.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.381+0000 7f9073e00640 1 --2- 172.21.15.31:0/2091701283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907c108e70 0x7f907c19e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.381+0000 7f907b400640 1 --2- 172.21.15.31:0/2091701283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f907c107c80 0x7f907c19df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.381+0000 7f907be00640 1 -- 172.21.15.31:0/2091701283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f907c107c80 msgr2=0x7f907c19df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.381+0000 7f907be00640 1 --2- 172.21.15.31:0/2091701283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f907c107c80 0x7f907c19df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.381+0000 7f907be00640 1 -- 172.21.15.31:0/2091701283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907c108e70 msgr2=0x7f907c19e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.381+0000 7f907be00640 1 --2- 172.21.15.31:0/2091701283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907c108e70 0x7f907c19e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.381+0000 7f907be00640 1 -- 172.21.15.31:0/2091701283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907c1a5b40 con 0x7f907c109830 2024-04-25T01:15:16.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.381+0000 7f907be00640 1 --2- 172.21.15.31:0/2091701283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f907c109830 0x7f907c1a54d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f907400eb10 tx=0x7f907400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:16.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.382+0000 7f9078c00640 1 -- 172.21.15.31:0/2091701283 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9074004b20 con 0x7f907c109830 2024-04-25T01:15:16.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.382+0000 7f9078c00640 1 -- 172.21.15.31:0/2091701283 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9074004cc0 con 0x7f907c109830 2024-04-25T01:15:16.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.382+0000 7f9080e00640 1 -- 172.21.15.31:0/2091701283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907c1a5e30 con 0x7f907c109830 2024-04-25T01:15:16.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.382+0000 7f9078c00640 1 -- 172.21.15.31:0/2091701283 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9074010760 con 0x7f907c109830 2024-04-25T01:15:16.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.382+0000 7f9080e00640 1 -- 172.21.15.31:0/2091701283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f907c1a62f0 con 0x7f907c109830 2024-04-25T01:15:16.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.385+0000 7f9078c00640 1 -- 172.21.15.31:0/2091701283 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9074005020 con 0x7f907c109830 2024-04-25T01:15:16.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.385+0000 7f90851e8740 1 Processor -- start 2024-04-25T01:15:16.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.386+0000 7f90851e8740 1 -- start start 2024-04-25T01:15:16.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.386+0000 7f90851e8740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff52c0c970 0x55ff52c0cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.386+0000 7f90851e8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0d360 0x55ff52c11e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.386+0000 7f90851e8740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff52c124d0 0x55ff52c14960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.386+0000 7f90851e8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ff52b0ced0 con 0x55ff52c0d360 2024-04-25T01:15:16.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.386+0000 7f90851e8740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ff52b6fdf0 con 0x55ff52c0c970 2024-04-25T01:15:16.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.386+0000 7f90851e8740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ff52b188d0 con 0x55ff52c124d0 2024-04-25T01:15:16.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.386+0000 7f9073e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff52c0c970 0x55ff52c0cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.386+0000 7f907b400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0d360 0x55ff52c11e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.386+0000 7f9073e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff52c0c970 0x55ff52c0cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:42836/0 (socket says 172.21.15.31:42836) 2024-04-25T01:15:16.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.386+0000 7f9073e00640 1 -- 172.21.15.31:0/2242682689 learned_addr learned my addr 172.21.15.31:0/2242682689 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:16.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.386+0000 7f9073e00640 1 --2- 172.21.15.31:0/2242682689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff52c124d0 0x55ff52c14960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.387+0000 7f907b400640 1 -- 172.21.15.31:0/2242682689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff52c124d0 msgr2=0x55ff52c14960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:16.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.387+0000 7f907b400640 1 --2- 172.21.15.31:0/2242682689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff52c124d0 0x55ff52c14960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.387+0000 7f907b400640 1 -- 172.21.15.31:0/2242682689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff52c0c970 msgr2=0x55ff52c0cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:16.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.387+0000 7f907b400640 1 --2- 172.21.15.31:0/2242682689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff52c0c970 0x55ff52c0cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.387+0000 7f907b400640 1 -- 172.21.15.31:0/2242682689 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff528842d0 con 0x55ff52c0d360 2024-04-25T01:15:16.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.387+0000 7f9078c00640 1 --2- 172.21.15.31:0/2091701283 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f90600a2b90 0x7f90600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.387+0000 7f907b400640 1 --2- 172.21.15.31:0/2242682689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0d360 0x55ff52c11e60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f906c009bf0 tx=0x7f906c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6288c8a27a6ebb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:16.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.387+0000 7f9078c00640 1 -- 172.21.15.31:0/2091701283 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f90740cf6f0 con 0x7f907c109830 2024-04-25T01:15:16.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.387+0000 7f9073e00640 1 --2- 172.21.15.31:0/2091701283 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f90600a2b90 0x7f90600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.388+0000 7f9070c00640 1 -- 172.21.15.31:0/2242682689 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c004050 con 0x55ff52c0d360 2024-04-25T01:15:16.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.388+0000 7f9070c00640 1 -- 172.21.15.31:0/2242682689 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f906c005500 con 0x55ff52c0d360 2024-04-25T01:15:16.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.388+0000 7f9070c00640 1 -- 172.21.15.31:0/2242682689 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c004d40 con 0x55ff52c0d360 2024-04-25T01:15:16.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.388+0000 7f90851e8740 1 -- 172.21.15.31:0/2242682689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0d360 msgr2=0x55ff52c11e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.388+0000 7f90851e8740 1 --2- 172.21.15.31:0/2242682689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0d360 0x55ff52c11e60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f906c009bf0 tx=0x7f906c011810 comp rx=0 tx=0).stop 2024-04-25T01:15:16.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.388+0000 7f9073e00640 1 --2- 172.21.15.31:0/2091701283 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f90600a2b90 0x7f90600a5050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9064005ec0 tx=0x7f9064005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:16.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.389+0000 7f90851e8740 1 -- 172.21.15.31:0/2242682689 shutdown_connections 2024-04-25T01:15:16.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.389+0000 7f90851e8740 1 --2- 172.21.15.31:0/2242682689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff52c124d0 0x55ff52c14960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.389+0000 7f90851e8740 1 --2- 172.21.15.31:0/2242682689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0d360 0x55ff52c11e60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.389+0000 7f90851e8740 1 --2- 172.21.15.31:0/2242682689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff52c0c970 0x55ff52c0cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.389+0000 7f90851e8740 1 -- 172.21.15.31:0/2242682689 >> 172.21.15.31:0/2242682689 conn(0x55ff52c07fe0 msgr2=0x55ff52c083e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:16.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.389+0000 7f90851e8740 1 -- 172.21.15.31:0/2242682689 shutdown_connections 2024-04-25T01:15:16.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.389+0000 7f90851e8740 1 -- 172.21.15.31:0/2242682689 wait complete. 2024-04-25T01:15:16.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.389+0000 7f90851e8740 1 -- start start 2024-04-25T01:15:16.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.389+0000 7f90851e8740 1 Processor -- start 2024-04-25T01:15:16.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.390+0000 7f90851e8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0c970 0x55ff52c16c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.390+0000 7f90851e8740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff52c0d360 0x55ff52c17150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.390+0000 7f90851e8740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff52c124d0 0x55ff52c1e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.390+0000 7f90851e8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ff52b2bd20 con 0x55ff52c0c970 2024-04-25T01:15:16.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.390+0000 7f90851e8740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ff52b6bdb0 con 0x55ff52c0d360 2024-04-25T01:15:16.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.390+0000 7f90851e8740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ff52c15480 con 0x55ff52c124d0 2024-04-25T01:15:16.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.390+0000 7f90851e8740 10 client.0 _ll_register_callbacks cb 0x55ff52b66f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:15:16.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.390+0000 7f907b400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0c970 0x55ff52c16c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.390+0000 7f907b400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0c970 0x55ff52c16c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46206/0 (socket says 172.21.15.31:46206) 2024-04-25T01:15:16.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.390+0000 7f907b400640 1 -- 172.21.15.31:0/626759987 learned_addr learned my addr 172.21.15.31:0/626759987 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:16.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.390+0000 7f907be00640 1 --2- 172.21.15.31:0/626759987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff52c124d0 0x55ff52c1e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.390+0000 7f9073e00640 1 --2- 172.21.15.31:0/626759987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff52c0d360 0x55ff52c17150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.390+0000 7f907b400640 1 -- 172.21.15.31:0/626759987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff52c124d0 msgr2=0x55ff52c1e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:16.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.391+0000 7f907b400640 1 --2- 172.21.15.31:0/626759987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff52c124d0 0x55ff52c1e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.391+0000 7f907b400640 1 -- 172.21.15.31:0/626759987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff52c0d360 msgr2=0x55ff52c17150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:16.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.391+0000 7f907b400640 1 --2- 172.21.15.31:0/626759987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff52c0d360 0x55ff52c17150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.391+0000 7f907b400640 1 -- 172.21.15.31:0/626759987 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff5294e6b0 con 0x55ff52c0c970 2024-04-25T01:15:16.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.391+0000 7f907b400640 1 --2- 172.21.15.31:0/626759987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0c970 0x55ff52c16c10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f906c0097b0 tx=0x7f906c009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bec7592b285aacb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:16.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.391+0000 7f9069600640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T01:15:16.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.391+0000 7f9069600640 1 -- 172.21.15.31:0/626759987 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c029070 con 0x55ff52c0c970 2024-04-25T01:15:16.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.391+0000 7f9069600640 1 -- 172.21.15.31:0/626759987 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f906c003a00 con 0x55ff52c0c970 2024-04-25T01:15:16.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.391+0000 7f90851e8740 1 -- 172.21.15.31:0/626759987 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f906c005500 con 0x55ff52c0c970 2024-04-25T01:15:16.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.392+0000 7f9069600640 1 -- 172.21.15.31:0/626759987 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f906c035050 con 0x55ff52c0c970 2024-04-25T01:15:16.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.392+0000 7f90851e8740 10 client.14886 fetch_fsmap learned FSMap version 23 2024-04-25T01:15:16.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.392+0000 7f90851e8740 1 -- 172.21.15.31:0/626759987 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ff52c21500 con 0x55ff52c0c970 2024-04-25T01:15:16.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.392+0000 7f9069600640 1 -- 172.21.15.31:0/626759987 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f906c005500 con 0x55ff52c0c970 2024-04-25T01:15:16.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.392+0000 7f9069600640 1 -- 172.21.15.31:0/626759987 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f906c0051a0 con 0x55ff52c0c970 2024-04-25T01:15:16.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.392+0000 7f90851e8740 10 client.14886 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T01:15:16.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.392+0000 7f90851e8740 10 client.14886 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T01:15:16.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.392+0000 7f90851e8740 20 client.14886 populate_metadata read hostname 'smithi031' 2024-04-25T01:15:16.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.392+0000 7f90851e8740 1 --2- 172.21.15.31:0/626759987 >> [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] conn(0x55ff52c23930 0x55ff52c24da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.392+0000 7f90851e8740 4 client.14886 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:15:16.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.392+0000 7f90851e8740 1 -- 172.21.15.31:0/626759987 --> [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff52b7d330 con 0x55ff52c23930 2024-04-25T01:15:16.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.393+0000 7f9073e00640 1 --2- 172.21.15.31:0/626759987 >> [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] conn(0x55ff52c23930 0x55ff52c24da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.393+0000 7f9073e00640 1 --2- 172.21.15.31:0/626759987 >> [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] conn(0x55ff52c23930 0x55ff52c24da0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ad8aae9e217412df server_cookie=35f348699661a5a2 in_seq=0 out_seq=0 2024-04-25T01:15:16.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.393+0000 7f9069600640 10 client.14886 ms_handle_connect on v2:172.21.15.169:6834/2253264784 2024-04-25T01:15:16.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.394+0000 7f9069600640 1 -- 172.21.15.31:0/626759987 <== mds.2 v2:172.21.15.169:6834/2253264784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ff52b7d330 con 0x55ff52c23930 2024-04-25T01:15:16.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.394+0000 7f9069600640 10 client.14886 handle_command_reply: tid=0 2024-04-25T01:15:16.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.394+0000 7f90851e8740 1 client.14886 shutdown 2024-04-25T01:15:16.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.395+0000 7f90851e8740 1 -- 172.21.15.31:0/626759987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0c970 msgr2=0x55ff52c16c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.395+0000 7f90851e8740 1 --2- 172.21.15.31:0/626759987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0c970 0x55ff52c16c10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f906c0097b0 tx=0x7f906c009fd0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.395+0000 7f90851e8740 1 -- 172.21.15.31:0/626759987 shutdown_connections 2024-04-25T01:15:16.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.395+0000 7f90851e8740 1 --2- 172.21.15.31:0/626759987 >> [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] conn(0x55ff52c23930 0x55ff52c24da0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.395+0000 7f90851e8740 1 --2- 172.21.15.31:0/626759987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff52c124d0 0x55ff52c1e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.395+0000 7f90851e8740 1 --2- 172.21.15.31:0/626759987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff52c0d360 0x55ff52c17150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.395+0000 7f90851e8740 1 --2- 172.21.15.31:0/626759987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0c970 0x55ff52c16c10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.395+0000 7f9069600640 0 client.14886 ms_handle_reset on v2:172.21.15.169:6834/2253264784 2024-04-25T01:15:16.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.395+0000 7f90851e8740 1 -- 172.21.15.31:0/626759987 >> 172.21.15.31:0/626759987 conn(0x55ff52c07fe0 msgr2=0x55ff52c07120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:16.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.395+0000 7f90851e8740 1 -- 172.21.15.31:0/626759987 shutdown_connections 2024-04-25T01:15:16.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.395+0000 7f90851e8740 1 -- 172.21.15.31:0/626759987 wait complete. 2024-04-25T01:15:16.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.395+0000 7f90851e8740 20 client.14886 trim_cache size 0 max 16384 2024-04-25T01:15:16.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.415+0000 7f90851e8740 1 Processor -- start 2024-04-25T01:15:16.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.415+0000 7f90851e8740 1 -- start start 2024-04-25T01:15:16.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.415+0000 7f90851e8740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff52c0c970 0x55ff52c02d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.415+0000 7f90851e8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0d360 0x55ff52c1e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.415+0000 7f90851e8740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff52c124d0 0x55ff52c083c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.415+0000 7f90851e8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ff52c230e0 con 0x55ff52c0d360 2024-04-25T01:15:16.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.415+0000 7f90851e8740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ff52c237b0 con 0x55ff52c0c970 2024-04-25T01:15:16.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.415+0000 7f90851e8740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ff52c23e80 con 0x55ff52c124d0 2024-04-25T01:15:16.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.416+0000 7f9073e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0d360 0x55ff52c1e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.416+0000 7f907b400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff52c0c970 0x55ff52c02d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.416+0000 7f907be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff52c124d0 0x55ff52c083c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.416+0000 7f9073e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0d360 0x55ff52c1e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46218/0 (socket says 172.21.15.31:46218) 2024-04-25T01:15:16.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.416+0000 7f9073e00640 1 -- 172.21.15.31:0/2868470739 learned_addr learned my addr 172.21.15.31:0/2868470739 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:16.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.416+0000 7f907b400640 1 -- 172.21.15.31:0/2868470739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff52c124d0 msgr2=0x55ff52c083c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:16.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.416+0000 7f907b400640 1 --2- 172.21.15.31:0/2868470739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff52c124d0 0x55ff52c083c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.416+0000 7f907b400640 1 -- 172.21.15.31:0/2868470739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0d360 msgr2=0x55ff52c1e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:16.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.416+0000 7f907b400640 1 --2- 172.21.15.31:0/2868470739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0d360 0x55ff52c1e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.416+0000 7f907b400640 1 -- 172.21.15.31:0/2868470739 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff52c08d50 con 0x55ff52c0c970 2024-04-25T01:15:16.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.416+0000 7f907be00640 1 --2- 172.21.15.31:0/2868470739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff52c124d0 0x55ff52c083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:16.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.416+0000 7f9073e00640 1 --2- 172.21.15.31:0/2868470739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0d360 0x55ff52c1e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:16.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.416+0000 7f907b400640 1 --2- 172.21.15.31:0/2868470739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff52c0c970 0x55ff52c02d40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f906c0099b0 tx=0x7f906c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91fc3fa8c7804324 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:16.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.417+0000 7f906be00640 1 -- 172.21.15.31:0/2868470739 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c029030 con 0x55ff52c0c970 2024-04-25T01:15:16.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.417+0000 7f906be00640 1 -- 172.21.15.31:0/2868470739 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f906c005520 con 0x55ff52c0c970 2024-04-25T01:15:16.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.417+0000 7f906be00640 1 -- 172.21.15.31:0/2868470739 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c0044f0 con 0x55ff52c0c970 2024-04-25T01:15:16.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.417+0000 7f90851e8740 1 -- 172.21.15.31:0/2868470739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff52c0c970 msgr2=0x55ff52c02d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.417+0000 7f90851e8740 1 --2- 172.21.15.31:0/2868470739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff52c0c970 0x55ff52c02d40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f906c0099b0 tx=0x7f906c005f90 comp rx=0 tx=0).stop 2024-04-25T01:15:16.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.417+0000 7f90851e8740 1 -- 172.21.15.31:0/2868470739 shutdown_connections 2024-04-25T01:15:16.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.417+0000 7f90851e8740 1 --2- 172.21.15.31:0/2868470739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff52c124d0 0x55ff52c083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.417+0000 7f90851e8740 1 --2- 172.21.15.31:0/2868470739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0d360 0x55ff52c1e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.417+0000 7f90851e8740 1 --2- 172.21.15.31:0/2868470739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff52c0c970 0x55ff52c02d40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.417+0000 7f90851e8740 1 -- 172.21.15.31:0/2868470739 >> 172.21.15.31:0/2868470739 conn(0x55ff52c07fe0 msgr2=0x55ff52c07120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:16.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.418+0000 7f90851e8740 1 -- 172.21.15.31:0/2868470739 shutdown_connections 2024-04-25T01:15:16.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.418+0000 7f90851e8740 1 -- 172.21.15.31:0/2868470739 wait complete. 2024-04-25T01:15:16.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.418+0000 7f90851e8740 1 -- start start 2024-04-25T01:15:16.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.418+0000 7f90851e8740 1 Processor -- start 2024-04-25T01:15:16.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.419+0000 7f90851e8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0c970 0x55ff52c1a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.419+0000 7f90851e8740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff52c0d360 0x55ff52c1a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.419+0000 7f90851e8740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff52c124d0 0x55ff52c1d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.419+0000 7f90851e8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ff52c24c20 con 0x55ff52c0c970 2024-04-25T01:15:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.419+0000 7f90851e8740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ff52c24550 con 0x55ff52c0d360 2024-04-25T01:15:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.419+0000 7f90851e8740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ff52c252f0 con 0x55ff52c124d0 2024-04-25T01:15:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.419+0000 7f90851e8740 10 client.0 _ll_register_callbacks cb 0x7f90740093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:15:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.419+0000 7f907b400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0c970 0x55ff52c1a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.419+0000 7f9073e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff52c0d360 0x55ff52c1a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.419+0000 7f907b400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0c970 0x55ff52c1a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46222/0 (socket says 172.21.15.31:46222) 2024-04-25T01:15:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.419+0000 7f907b400640 1 -- 172.21.15.31:0/3468272495 learned_addr learned my addr 172.21.15.31:0/3468272495 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.419+0000 7f907be00640 1 --2- 172.21.15.31:0/3468272495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff52c124d0 0x55ff52c1d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.419+0000 7f9073e00640 1 -- 172.21.15.31:0/3468272495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff52c124d0 msgr2=0x55ff52c1d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.419+0000 7f9073e00640 1 --2- 172.21.15.31:0/3468272495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff52c124d0 0x55ff52c1d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.419+0000 7f9073e00640 1 -- 172.21.15.31:0/3468272495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0c970 msgr2=0x55ff52c1a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.419+0000 7f9073e00640 1 --2- 172.21.15.31:0/3468272495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0c970 0x55ff52c1a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.419+0000 7f9073e00640 1 -- 172.21.15.31:0/3468272495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff52c121b0 con 0x55ff52c0d360 2024-04-25T01:15:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.419+0000 7f907be00640 1 --2- 172.21.15.31:0/3468272495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff52c124d0 0x55ff52c1d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:15:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.419+0000 7f907b400640 1 --2- 172.21.15.31:0/3468272495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0c970 0x55ff52c1a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.420+0000 7f9073e00640 1 --2- 172.21.15.31:0/3468272495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff52c0d360 0x55ff52c1a890 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9064005ab0 tx=0x7f9064000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d2ded1baf9da6be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.420+0000 7f9069600640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T01:15:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.420+0000 7f9069600640 1 -- 172.21.15.31:0/3468272495 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9064035040 con 0x55ff52c0d360 2024-04-25T01:15:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.420+0000 7f90851e8740 1 -- 172.21.15.31:0/3468272495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f906c005520 con 0x55ff52c0d360 2024-04-25T01:15:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.420+0000 7f9069600640 1 -- 172.21.15.31:0/3468272495 <== mon.2 v2:172.21.15.40:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f9064004980 con 0x55ff52c0d360 2024-04-25T01:15:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.420+0000 7f9069600640 1 -- 172.21.15.31:0/3468272495 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9064063050 con 0x55ff52c0d360 2024-04-25T01:15:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.420+0000 7f90851e8740 10 client.25013 fetch_fsmap learned FSMap version 23 2024-04-25T01:15:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.420+0000 7f90851e8740 1 -- 172.21.15.31:0/3468272495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ff52c89d60 con 0x55ff52c0d360 2024-04-25T01:15:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.421+0000 7f9069600640 1 -- 172.21.15.31:0/3468272495 <== mon.2 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f906c005520 con 0x55ff52c0d360 2024-04-25T01:15:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.421+0000 7f9069600640 1 -- 172.21.15.31:0/3468272495 <== mon.2 v2:172.21.15.40:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f906403d450 con 0x55ff52c0d360 2024-04-25T01:15:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.421+0000 7f90851e8740 10 client.25013 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T01:15:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.421+0000 7f90851e8740 10 client.25013 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T01:15:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.421+0000 7f90851e8740 20 client.25013 populate_metadata read hostname 'smithi031' 2024-04-25T01:15:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.421+0000 7f90851e8740 1 --2- 172.21.15.31:0/3468272495 >> [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] conn(0x55ff52c8e950 0x55ff52c8ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.421+0000 7f90851e8740 4 client.25013 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T01:15:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.421+0000 7f90851e8740 1 -- 172.21.15.31:0/3468272495 --> [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] -- command(tid 0: {"prefix": "status"}) -- 0x55ff52b77680 con 0x55ff52c8e950 2024-04-25T01:15:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.421+0000 7f907b400640 1 --2- 172.21.15.31:0/3468272495 >> [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] conn(0x55ff52c8e950 0x55ff52c8ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.422+0000 7f907b400640 1 --2- 172.21.15.31:0/3468272495 >> [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] conn(0x55ff52c8e950 0x55ff52c8ad90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=174ca309f3959871 server_cookie=8cc5513e299e91dc in_seq=0 out_seq=0 2024-04-25T01:15:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.422+0000 7f9069600640 10 client.25013 ms_handle_connect on v2:172.21.15.169:6834/2253264784 2024-04-25T01:15:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.422+0000 7f9069600640 1 -- 172.21.15.31:0/3468272495 <== mds.2 v2:172.21.15.169:6834/2253264784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ff52b77680 con 0x55ff52c8e950 2024-04-25T01:15:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.422+0000 7f9069600640 10 client.25013 handle_command_reply: tid=0 2024-04-25T01:15:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.422+0000 7f90851e8740 1 client.25013 shutdown 2024-04-25T01:15:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.422+0000 7f90851e8740 1 -- 172.21.15.31:0/3468272495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff52c0d360 msgr2=0x55ff52c1a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.422+0000 7f90851e8740 1 --2- 172.21.15.31:0/3468272495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff52c0d360 0x55ff52c1a890 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9064005ab0 tx=0x7f9064000bc0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.422+0000 7f90851e8740 1 -- 172.21.15.31:0/3468272495 shutdown_connections 2024-04-25T01:15:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.422+0000 7f90851e8740 1 --2- 172.21.15.31:0/3468272495 >> [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] conn(0x55ff52c8e950 0x55ff52c8ad90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.422+0000 7f90851e8740 1 --2- 172.21.15.31:0/3468272495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff52c124d0 0x55ff52c1d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.422+0000 7f90851e8740 1 --2- 172.21.15.31:0/3468272495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ff52c0d360 0x55ff52c1a890 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.422+0000 7f90851e8740 1 --2- 172.21.15.31:0/3468272495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff52c0c970 0x55ff52c1a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.422+0000 7f9069600640 0 client.25013 ms_handle_reset on v2:172.21.15.169:6834/2253264784 2024-04-25T01:15:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.422+0000 7f90851e8740 1 -- 172.21.15.31:0/3468272495 >> 172.21.15.31:0/3468272495 conn(0x55ff52c07fe0 msgr2=0x55ff52c07120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.422+0000 7f90851e8740 1 -- 172.21.15.31:0/3468272495 shutdown_connections 2024-04-25T01:15:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.422+0000 7f90851e8740 1 -- 172.21.15.31:0/3468272495 wait complete. 2024-04-25T01:15:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.423+0000 7f90851e8740 20 client.25013 trim_cache size 0 max 16384 2024-04-25T01:15:16.413 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T01:15:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "95849b94-029f-11ef-bc93-c7b262605968", 2024-04-25T01:15:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 2, 2024-04-25T01:15:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24532, 2024-04-25T01:15:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-04-25T01:15:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-04-25T01:15:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-04-25T01:15:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 100.885242835, 2024-04-25T01:15:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 23, 2024-04-25T01:15:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 89, 2024-04-25T01:15:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 87, 2024-04-25T01:15:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 131.74406385699999 2024-04-25T01:15:16.413 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T01:15:16.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.423+0000 7f9080e00640 1 -- 172.21.15.31:0/2091701283 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f90600a2b90 msgr2=0x7f90600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.423+0000 7f9080e00640 1 --2- 172.21.15.31:0/2091701283 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f90600a2b90 0x7f90600a5050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9064005ec0 tx=0x7f9064005fd0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.423+0000 7f9080e00640 1 -- 172.21.15.31:0/2091701283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f907c109830 msgr2=0x7f907c1a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.423+0000 7f9080e00640 1 --2- 172.21.15.31:0/2091701283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f907c109830 0x7f907c1a54d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f907400eb10 tx=0x7f907400c2d0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.423+0000 7f9080e00640 1 -- 172.21.15.31:0/2091701283 shutdown_connections 2024-04-25T01:15:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.423+0000 7f9080e00640 1 --2- 172.21.15.31:0/2091701283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f907c109830 0x7f907c1a54d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.423+0000 7f9080e00640 1 --2- 172.21.15.31:0/2091701283 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f90600a2b90 0x7f90600a5050 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.423+0000 7f9080e00640 1 --2- 172.21.15.31:0/2091701283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907c108e70 0x7f907c19e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.423+0000 7f9080e00640 1 --2- 172.21.15.31:0/2091701283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f907c107c80 0x7f907c19df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.423+0000 7f9080e00640 1 -- 172.21.15.31:0/2091701283 >> 172.21.15.31:0/2091701283 conn(0x7f907c076c10 msgr2=0x7f907c10c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.424+0000 7f9080e00640 1 -- 172.21.15.31:0/2091701283 shutdown_connections 2024-04-25T01:15:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.424+0000 7f9080e00640 1 -- 172.21.15.31:0/2091701283 wait complete. 2024-04-25T01:15:16.425 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:16 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/4058795973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:15:16.425 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:16 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1612668196' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:15:16.429 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T01:15:16.443 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:16 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/4058795973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:15:16.444 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:16 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1612668196' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:15:16.446 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:16 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/4058795973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:15:16.446 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:16 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1612668196' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:15:16.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.657+0000 7fa3f0000640 1 -- 172.21.15.31:0/3117030228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3e8104b90 msgr2=0x7fa3e8105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.657+0000 7fa3f0000640 1 --2- 172.21.15.31:0/3117030228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3e8104b90 0x7fa3e8105010 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4009860 tx=0x7fa3d402a340 comp rx=0 tx=0).stop 2024-04-25T01:15:16.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.657+0000 7fa3f0000640 1 -- 172.21.15.31:0/3117030228 shutdown_connections 2024-04-25T01:15:16.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.657+0000 7fa3f0000640 1 --2- 172.21.15.31:0/3117030228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3e8105550 0x7fa3e810bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.657+0000 7fa3f0000640 1 --2- 172.21.15.31:0/3117030228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3e8104b90 0x7fa3e8105010 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.657+0000 7fa3f0000640 1 --2- 172.21.15.31:0/3117030228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3e81039a0 0x7fa3e8103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.657+0000 7fa3f0000640 1 -- 172.21.15.31:0/3117030228 >> 172.21.15.31:0/3117030228 conn(0x7fa3e80fef30 msgr2=0x7fa3e8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:16.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.657+0000 7fa3f0000640 1 -- 172.21.15.31:0/3117030228 shutdown_connections 2024-04-25T01:15:16.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.657+0000 7fa3f0000640 1 -- 172.21.15.31:0/3117030228 wait complete. 2024-04-25T01:15:16.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.659+0000 7fa3f0000640 1 Processor -- start 2024-04-25T01:15:16.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.659+0000 7fa3f0000640 1 -- start start 2024-04-25T01:15:16.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.659+0000 7fa3f0000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3e81039a0 0x7fa3e8199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.659+0000 7fa3f0000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3e8104b90 0x7fa3e819a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.659+0000 7fa3f0000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3e8105550 0x7fa3e81a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.659+0000 7fa3f0000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e810f600 con 0x7fa3e81039a0 2024-04-25T01:15:16.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.659+0000 7fa3f0000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e810f480 con 0x7fa3e8104b90 2024-04-25T01:15:16.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.659+0000 7fa3f0000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e810f780 con 0x7fa3e8105550 2024-04-25T01:15:16.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.660+0000 7fa3ed600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3e81039a0 0x7fa3e8199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.660+0000 7fa3ed600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3e81039a0 0x7fa3e8199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46248/0 (socket says 172.21.15.31:46248) 2024-04-25T01:15:16.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.660+0000 7fa3ed600640 1 -- 172.21.15.31:0/3119825418 learned_addr learned my addr 172.21.15.31:0/3119825418 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:16.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.660+0000 7fa3ecc00640 1 --2- 172.21.15.31:0/3119825418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3e8104b90 0x7fa3e819a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.660+0000 7fa3ee000640 1 --2- 172.21.15.31:0/3119825418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3e8105550 0x7fa3e81a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.660+0000 7fa3ecc00640 1 -- 172.21.15.31:0/3119825418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3e8105550 msgr2=0x7fa3e81a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.660+0000 7fa3ecc00640 1 --2- 172.21.15.31:0/3119825418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3e8105550 0x7fa3e81a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.660+0000 7fa3ecc00640 1 -- 172.21.15.31:0/3119825418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3e81039a0 msgr2=0x7fa3e8199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.660+0000 7fa3ecc00640 1 --2- 172.21.15.31:0/3119825418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3e81039a0 0x7fa3e8199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.660+0000 7fa3ecc00640 1 -- 172.21.15.31:0/3119825418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e81a18d0 con 0x7fa3e8104b90 2024-04-25T01:15:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.660+0000 7fa3ed600640 1 --2- 172.21.15.31:0/3119825418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3e81039a0 0x7fa3e8199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.661+0000 7fa3ecc00640 1 --2- 172.21.15.31:0/3119825418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3e8104b90 0x7fa3e819a1d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4035040 tx=0x7fa3d4033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.661+0000 7fa3e6000640 1 -- 172.21.15.31:0/3119825418 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4037040 con 0x7fa3e8104b90 2024-04-25T01:15:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.661+0000 7fa3e6000640 1 -- 172.21.15.31:0/3119825418 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d4030850 con 0x7fa3e8104b90 2024-04-25T01:15:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.661+0000 7fa3e6000640 1 -- 172.21.15.31:0/3119825418 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4005650 con 0x7fa3e8104b90 2024-04-25T01:15:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.661+0000 7fa3f0000640 1 -- 172.21.15.31:0/3119825418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e81a1b60 con 0x7fa3e8104b90 2024-04-25T01:15:16.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.661+0000 7fa3f0000640 1 -- 172.21.15.31:0/3119825418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e81a2020 con 0x7fa3e8104b90 2024-04-25T01:15:16.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.664+0000 7fa3f2e00740 1 Processor -- start 2024-04-25T01:15:16.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.664+0000 7fa3f2e00740 1 -- start start 2024-04-25T01:15:16.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.665+0000 7fa3e6000640 1 -- 172.21.15.31:0/3119825418 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa3d4004050 con 0x7fa3e8104b90 2024-04-25T01:15:16.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.665+0000 7fa3f2e00740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f00760 0x564966f00b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.665+0000 7fa3f2e00740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564966f01150 0x564966f05c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.665+0000 7fa3f2e00740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564966f062c0 0x564966f08750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.665+0000 7fa3f2e00740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564966e00ed0 con 0x564966f00760 2024-04-25T01:15:16.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.665+0000 7fa3f2e00740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564966e63df0 con 0x564966f062c0 2024-04-25T01:15:16.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.665+0000 7fa3f2e00740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564966e0c8d0 con 0x564966f01150 2024-04-25T01:15:16.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.665+0000 7fa3ee000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f00760 0x564966f00b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.665+0000 7fa3ee000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f00760 0x564966f00b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46256/0 (socket says 172.21.15.31:46256) 2024-04-25T01:15:16.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.665+0000 7fa3ee000640 1 -- 172.21.15.31:0/2296218350 learned_addr learned my addr 172.21.15.31:0/2296218350 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:16.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.665+0000 7fa3ecc00640 1 --2- 172.21.15.31:0/2296218350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564966f062c0 0x564966f08750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.665+0000 7fa3ed600640 1 --2- 172.21.15.31:0/2296218350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564966f01150 0x564966f05c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.665+0000 7fa3ee000640 1 -- 172.21.15.31:0/2296218350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564966f01150 msgr2=0x564966f05c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:16.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.665+0000 7fa3ee000640 1 --2- 172.21.15.31:0/2296218350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564966f01150 0x564966f05c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.665+0000 7fa3ee000640 1 -- 172.21.15.31:0/2296218350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564966f062c0 msgr2=0x564966f08750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:16.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.665+0000 7fa3ee000640 1 --2- 172.21.15.31:0/2296218350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564966f062c0 0x564966f08750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.666+0000 7fa3ee000640 1 -- 172.21.15.31:0/2296218350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564966b782d0 con 0x564966f00760 2024-04-25T01:15:16.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.666+0000 7fa3ecc00640 1 --2- 172.21.15.31:0/2296218350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564966f062c0 0x564966f08750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:15:16.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.666+0000 7fa3ee000640 1 --2- 172.21.15.31:0/2296218350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f00760 0x564966f00b80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00039f0 tx=0x7fa3e0003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfbd9bedb39d52ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:16.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.666+0000 7fa3da600640 1 -- 172.21.15.31:0/2296218350 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e001a470 con 0x564966f00760 2024-04-25T01:15:16.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.666+0000 7fa3da600640 1 -- 172.21.15.31:0/2296218350 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3e0015070 con 0x564966f00760 2024-04-25T01:15:16.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.666+0000 7fa3da600640 1 -- 172.21.15.31:0/2296218350 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0010900 con 0x564966f00760 2024-04-25T01:15:16.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.666+0000 7fa3e6000640 1 --2- 172.21.15.31:0/3119825418 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa3d00a2b90 0x7fa3d00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.667+0000 7fa3e6000640 1 -- 172.21.15.31:0/3119825418 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fa3d40eeeb0 con 0x7fa3e8104b90 2024-04-25T01:15:16.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.667+0000 7fa3ed600640 1 --2- 172.21.15.31:0/3119825418 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa3d00a2b90 0x7fa3d00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.667+0000 7fa3f2e00740 1 -- 172.21.15.31:0/2296218350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f00760 msgr2=0x564966f00b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.667+0000 7fa3f2e00740 1 --2- 172.21.15.31:0/2296218350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f00760 0x564966f00b80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00039f0 tx=0x7fa3e0003eb0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.667+0000 7fa3f2e00740 1 -- 172.21.15.31:0/2296218350 shutdown_connections 2024-04-25T01:15:16.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.667+0000 7fa3f2e00740 1 --2- 172.21.15.31:0/2296218350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564966f062c0 0x564966f08750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.667+0000 7fa3f2e00740 1 --2- 172.21.15.31:0/2296218350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564966f01150 0x564966f05c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.667+0000 7fa3f2e00740 1 --2- 172.21.15.31:0/2296218350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f00760 0x564966f00b80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.667+0000 7fa3f2e00740 1 -- 172.21.15.31:0/2296218350 >> 172.21.15.31:0/2296218350 conn(0x564966efbdd0 msgr2=0x564966efc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:16.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.667+0000 7fa3f2e00740 1 -- 172.21.15.31:0/2296218350 shutdown_connections 2024-04-25T01:15:16.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.667+0000 7fa3f2e00740 1 -- 172.21.15.31:0/2296218350 wait complete. 2024-04-25T01:15:16.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.667+0000 7fa3ed600640 1 --2- 172.21.15.31:0/3119825418 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa3d00a2b90 0x7fa3d00a5050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x564966f01550 tx=0x7fa3dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:16.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.668+0000 7fa3f2e00740 1 -- start start 2024-04-25T01:15:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.668+0000 7fa3f2e00740 1 Processor -- start 2024-04-25T01:15:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.668+0000 7fa3f2e00740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f00760 0x564966f0aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.669+0000 7fa3f2e00740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564966f01150 0x564966f0b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.669+0000 7fa3f2e00740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564966f062c0 0x564966f11fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.669+0000 7fa3f2e00740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564966e1fd20 con 0x564966f00760 2024-04-25T01:15:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.669+0000 7fa3f2e00740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564966e5fdb0 con 0x564966f062c0 2024-04-25T01:15:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.669+0000 7fa3f2e00740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564966f09270 con 0x564966f01150 2024-04-25T01:15:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.669+0000 7fa3f2e00740 10 client.0 _ll_register_callbacks cb 0x564966e5af10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:15:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.669+0000 7fa3ed600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f00760 0x564966f0aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.669+0000 7fa3ed600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f00760 0x564966f0aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46264/0 (socket says 172.21.15.31:46264) 2024-04-25T01:15:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.669+0000 7fa3ed600640 1 -- 172.21.15.31:0/1343990527 learned_addr learned my addr 172.21.15.31:0/1343990527 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.669+0000 7fa3ecc00640 1 --2- 172.21.15.31:0/1343990527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564966f01150 0x564966f0b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.669+0000 7fa3ee000640 1 --2- 172.21.15.31:0/1343990527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564966f062c0 0x564966f11fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.669+0000 7fa3ed600640 1 -- 172.21.15.31:0/1343990527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564966f01150 msgr2=0x564966f0b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.669+0000 7fa3ed600640 1 --2- 172.21.15.31:0/1343990527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564966f01150 0x564966f0b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.669+0000 7fa3ed600640 1 -- 172.21.15.31:0/1343990527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564966f062c0 msgr2=0x564966f11fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.669+0000 7fa3ed600640 1 --2- 172.21.15.31:0/1343990527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564966f062c0 0x564966f11fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.669+0000 7fa3ed600640 1 -- 172.21.15.31:0/1343990527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564966c426b0 con 0x564966f00760 2024-04-25T01:15:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.669+0000 7fa3ecc00640 1 --2- 172.21.15.31:0/1343990527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564966f01150 0x564966f0b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.670+0000 7fa3ed600640 1 --2- 172.21.15.31:0/1343990527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f00760 0x564966f0aac0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc003670 tx=0x7fa3dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a301b5d0fead92c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.670+0000 7fa3cea00640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T01:15:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.670+0000 7fa3cea00640 1 -- 172.21.15.31:0/1343990527 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc02c060 con 0x564966f00760 2024-04-25T01:15:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.670+0000 7fa3cea00640 1 -- 172.21.15.31:0/1343990527 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fa3dc00fa50 con 0x564966f00760 2024-04-25T01:15:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.670+0000 7fa3f2e00740 1 -- 172.21.15.31:0/1343990527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3e0015070 con 0x564966f00760 2024-04-25T01:15:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.670+0000 7fa3cea00640 1 -- 172.21.15.31:0/1343990527 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa3dc00fc50 con 0x564966f00760 2024-04-25T01:15:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.670+0000 7fa3f2e00740 10 client.14904 fetch_fsmap learned FSMap version 23 2024-04-25T01:15:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.670+0000 7fa3f2e00740 1 -- 172.21.15.31:0/1343990527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x564966f15460 con 0x564966f00760 2024-04-25T01:15:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.671+0000 7fa3cea00640 1 -- 172.21.15.31:0/1343990527 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3e0015070 con 0x564966f00760 2024-04-25T01:15:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.671+0000 7fa3cea00640 1 -- 172.21.15.31:0/1343990527 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fa3dc026030 con 0x564966f00760 2024-04-25T01:15:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.671+0000 7fa3f2e00740 10 client.14904 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T01:15:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.671+0000 7fa3f2e00740 10 client.14904 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T01:15:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.671+0000 7fa3f2e00740 20 client.14904 populate_metadata read hostname 'smithi031' 2024-04-25T01:15:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.671+0000 7fa3f2e00740 1 --2- 172.21.15.31:0/1343990527 >> [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] conn(0x564966f17b80 0x564966f18ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.671+0000 7fa3f2e00740 4 client.14904 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:15:16.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.671+0000 7fa3f2e00740 1 -- 172.21.15.31:0/1343990527 --> [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564966e71330 con 0x564966f17b80 2024-04-25T01:15:16.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.672+0000 7fa3ecc00640 1 --2- 172.21.15.31:0/1343990527 >> [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] conn(0x564966f17b80 0x564966f18ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.672+0000 7fa3ecc00640 1 --2- 172.21.15.31:0/1343990527 >> [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] conn(0x564966f17b80 0x564966f18ff0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dda1566f9fe1c034 server_cookie=832685b51f909764 in_seq=0 out_seq=0 2024-04-25T01:15:16.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.672+0000 7fa3cea00640 10 client.14904 ms_handle_connect on v2:172.21.15.169:6838/4185710218 2024-04-25T01:15:16.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.673+0000 7fa3cea00640 1 -- 172.21.15.31:0/1343990527 <== mds.1 v2:172.21.15.169:6838/4185710218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564966e71330 con 0x564966f17b80 2024-04-25T01:15:16.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.673+0000 7fa3cea00640 10 client.14904 handle_command_reply: tid=0 2024-04-25T01:15:16.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.673+0000 7fa3f2e00740 1 client.14904 shutdown 2024-04-25T01:15:16.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.673+0000 7fa3f2e00740 1 -- 172.21.15.31:0/1343990527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f00760 msgr2=0x564966f0aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.673+0000 7fa3f2e00740 1 --2- 172.21.15.31:0/1343990527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f00760 0x564966f0aac0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc003670 tx=0x7fa3dc004730 comp rx=0 tx=0).stop 2024-04-25T01:15:16.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.673+0000 7fa3f2e00740 1 -- 172.21.15.31:0/1343990527 shutdown_connections 2024-04-25T01:15:16.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.673+0000 7fa3f2e00740 1 --2- 172.21.15.31:0/1343990527 >> [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] conn(0x564966f17b80 0x564966f18ff0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.673+0000 7fa3f2e00740 1 --2- 172.21.15.31:0/1343990527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564966f062c0 0x564966f11fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.673+0000 7fa3f2e00740 1 --2- 172.21.15.31:0/1343990527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564966f01150 0x564966f0b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.673+0000 7fa3f2e00740 1 --2- 172.21.15.31:0/1343990527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f00760 0x564966f0aac0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.673+0000 7fa3cea00640 0 client.14904 ms_handle_reset on v2:172.21.15.169:6838/4185710218 2024-04-25T01:15:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.673+0000 7fa3f2e00740 1 -- 172.21.15.31:0/1343990527 >> 172.21.15.31:0/1343990527 conn(0x564966efbdd0 msgr2=0x564966efaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.673+0000 7fa3f2e00740 1 -- 172.21.15.31:0/1343990527 shutdown_connections 2024-04-25T01:15:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.674+0000 7fa3f2e00740 1 -- 172.21.15.31:0/1343990527 wait complete. 2024-04-25T01:15:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.674+0000 7fa3f2e00740 20 client.14904 trim_cache size 0 max 16384 2024-04-25T01:15:16.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.694+0000 7fa3f2e00740 1 Processor -- start 2024-04-25T01:15:16.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.694+0000 7fa3f2e00740 1 -- start start 2024-04-25T01:15:16.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.694+0000 7fa3f2e00740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564966f00760 0x564966f086b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.694+0000 7fa3f2e00740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f01150 0x564966f11f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.694+0000 7fa3f2e00740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564966f062c0 0x564966efc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.694+0000 7fa3f2e00740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564966f17330 con 0x564966f01150 2024-04-25T01:15:16.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.694+0000 7fa3f2e00740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564966f17a00 con 0x564966f062c0 2024-04-25T01:15:16.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.694+0000 7fa3f2e00740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564966f180d0 con 0x564966f00760 2024-04-25T01:15:16.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.694+0000 7fa3ecc00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f01150 0x564966f11f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.694+0000 7fa3ecc00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f01150 0x564966f11f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46272/0 (socket says 172.21.15.31:46272) 2024-04-25T01:15:16.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.694+0000 7fa3ecc00640 1 -- 172.21.15.31:0/768881434 learned_addr learned my addr 172.21.15.31:0/768881434 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:16.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.694+0000 7fa3ed600640 1 --2- 172.21.15.31:0/768881434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564966f00760 0x564966f086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.695+0000 7fa3ee000640 1 --2- 172.21.15.31:0/768881434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564966f062c0 0x564966efc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.695+0000 7fa3ecc00640 1 -- 172.21.15.31:0/768881434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564966f00760 msgr2=0x564966f086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:16.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.695+0000 7fa3ecc00640 1 --2- 172.21.15.31:0/768881434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564966f00760 0x564966f086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.695+0000 7fa3ecc00640 1 -- 172.21.15.31:0/768881434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564966f062c0 msgr2=0x564966efc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:16.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.695+0000 7fa3ecc00640 1 --2- 172.21.15.31:0/768881434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564966f062c0 0x564966efc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.695+0000 7fa3ecc00640 1 -- 172.21.15.31:0/768881434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564966efcb40 con 0x564966f01150 2024-04-25T01:15:16.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.695+0000 7fa3ecc00640 1 --2- 172.21.15.31:0/768881434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f01150 0x564966f11f60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa3d402a930 tx=0x7fa3d40a6040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43fcd1ba06681a1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:16.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.695+0000 7fa3d9c00640 1 -- 172.21.15.31:0/768881434 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d40d4070 con 0x564966f01150 2024-04-25T01:15:16.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.695+0000 7fa3d9c00640 1 -- 172.21.15.31:0/768881434 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d40bbe00 con 0x564966f01150 2024-04-25T01:15:16.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.695+0000 7fa3d9c00640 1 -- 172.21.15.31:0/768881434 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d40ecab0 con 0x564966f01150 2024-04-25T01:15:16.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.696+0000 7fa3f2e00740 1 -- 172.21.15.31:0/768881434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f01150 msgr2=0x564966f11f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.696+0000 7fa3f2e00740 1 --2- 172.21.15.31:0/768881434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f01150 0x564966f11f60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa3d402a930 tx=0x7fa3d40a6040 comp rx=0 tx=0).stop 2024-04-25T01:15:16.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.696+0000 7fa3f2e00740 1 -- 172.21.15.31:0/768881434 shutdown_connections 2024-04-25T01:15:16.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.696+0000 7fa3f2e00740 1 --2- 172.21.15.31:0/768881434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564966f062c0 0x564966efc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.696+0000 7fa3f2e00740 1 --2- 172.21.15.31:0/768881434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f01150 0x564966f11f60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.696+0000 7fa3f2e00740 1 --2- 172.21.15.31:0/768881434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564966f00760 0x564966f086b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.696+0000 7fa3f2e00740 1 -- 172.21.15.31:0/768881434 >> 172.21.15.31:0/768881434 conn(0x564966efbdd0 msgr2=0x564966ef6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:16.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.696+0000 7fa3f2e00740 1 -- 172.21.15.31:0/768881434 shutdown_connections 2024-04-25T01:15:16.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.696+0000 7fa3f2e00740 1 -- 172.21.15.31:0/768881434 wait complete. 2024-04-25T01:15:16.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.696+0000 7fa3f2e00740 1 -- start start 2024-04-25T01:15:16.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.697+0000 7fa3f2e00740 1 Processor -- start 2024-04-25T01:15:16.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.697+0000 7fa3f2e00740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f00760 0x564966f0d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.697+0000 7fa3f2e00740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564966f01150 0x564966f0dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.697+0000 7fa3f2e00740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564966f062c0 0x564966f10af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.697+0000 7fa3f2e00740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564966f18e70 con 0x564966f00760 2024-04-25T01:15:16.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.697+0000 7fa3f2e00740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564966f187a0 con 0x564966f062c0 2024-04-25T01:15:16.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.697+0000 7fa3f2e00740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564966f19540 con 0x564966f01150 2024-04-25T01:15:16.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.697+0000 7fa3f2e00740 10 client.0 _ll_register_callbacks cb 0x7fa3dc004550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:15:16.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.697+0000 7fa3ed600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f00760 0x564966f0d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.697+0000 7fa3ed600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f00760 0x564966f0d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46286/0 (socket says 172.21.15.31:46286) 2024-04-25T01:15:16.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.697+0000 7fa3ed600640 1 -- 172.21.15.31:0/3780250814 learned_addr learned my addr 172.21.15.31:0/3780250814 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:16.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.698+0000 7fa3ee000640 1 --2- 172.21.15.31:0/3780250814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564966f062c0 0x564966f10af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.698+0000 7fa3ecc00640 1 --2- 172.21.15.31:0/3780250814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564966f01150 0x564966f0dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.698+0000 7fa3ed600640 1 -- 172.21.15.31:0/3780250814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564966f01150 msgr2=0x564966f0dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:16.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.698+0000 7fa3ed600640 1 --2- 172.21.15.31:0/3780250814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564966f01150 0x564966f0dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.698+0000 7fa3ed600640 1 -- 172.21.15.31:0/3780250814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564966f062c0 msgr2=0x564966f10af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:16.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.698+0000 7fa3ed600640 1 --2- 172.21.15.31:0/3780250814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564966f062c0 0x564966f10af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.698+0000 7fa3ed600640 1 -- 172.21.15.31:0/3780250814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564966f110e0 con 0x564966f00760 2024-04-25T01:15:16.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.698+0000 7fa3ed600640 1 --2- 172.21.15.31:0/3780250814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f00760 0x564966f0d590 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc003640 tx=0x7fa3dc009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1e936a3a47315e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:16.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.698+0000 7fa3cea00640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T01:15:16.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.698+0000 7fa3cea00640 1 -- 172.21.15.31:0/3780250814 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc01ed80 con 0x564966f00760 2024-04-25T01:15:16.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.698+0000 7fa3cea00640 1 -- 172.21.15.31:0/3780250814 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fa3dc03b480 con 0x564966f00760 2024-04-25T01:15:16.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.698+0000 7fa3f2e00740 1 -- 172.21.15.31:0/3780250814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3d40bbe00 con 0x564966f00760 2024-04-25T01:15:16.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.699+0000 7fa3cea00640 1 -- 172.21.15.31:0/3780250814 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa3dc005020 con 0x564966f00760 2024-04-25T01:15:16.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.699+0000 7fa3f2e00740 10 client.14910 fetch_fsmap learned FSMap version 23 2024-04-25T01:15:16.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.699+0000 7fa3f2e00740 1 -- 172.21.15.31:0/3780250814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x564966f86340 con 0x564966f00760 2024-04-25T01:15:16.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.699+0000 7fa3cea00640 1 -- 172.21.15.31:0/3780250814 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d40bbe00 con 0x564966f00760 2024-04-25T01:15:16.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.699+0000 7fa3cea00640 1 -- 172.21.15.31:0/3780250814 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fa3dc026040 con 0x564966f00760 2024-04-25T01:15:16.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.699+0000 7fa3f2e00740 10 client.14910 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T01:15:16.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.699+0000 7fa3f2e00740 10 client.14910 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T01:15:16.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.699+0000 7fa3f2e00740 20 client.14910 populate_metadata read hostname 'smithi031' 2024-04-25T01:15:16.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.699+0000 7fa3f2e00740 1 --2- 172.21.15.31:0/3780250814 >> [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] conn(0x564966f8af90 0x564966f89ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.699+0000 7fa3f2e00740 4 client.14910 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T01:15:16.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.699+0000 7fa3f2e00740 1 -- 172.21.15.31:0/3780250814 --> [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] -- command(tid 0: {"prefix": "status"}) -- 0x564966e6b680 con 0x564966f8af90 2024-04-25T01:15:16.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.700+0000 7fa3ecc00640 1 --2- 172.21.15.31:0/3780250814 >> [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] conn(0x564966f8af90 0x564966f89ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.700+0000 7fa3ecc00640 1 --2- 172.21.15.31:0/3780250814 >> [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] conn(0x564966f8af90 0x564966f89ca0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8b8b390c1471fc26 server_cookie=777be5d97070802d in_seq=0 out_seq=0 2024-04-25T01:15:16.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.700+0000 7fa3cea00640 10 client.14910 ms_handle_connect on v2:172.21.15.169:6838/4185710218 2024-04-25T01:15:16.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.700+0000 7fa3cea00640 1 -- 172.21.15.31:0/3780250814 <== mds.1 v2:172.21.15.169:6838/4185710218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564966e6b680 con 0x564966f8af90 2024-04-25T01:15:16.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.700+0000 7fa3cea00640 10 client.14910 handle_command_reply: tid=0 2024-04-25T01:15:16.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.700+0000 7fa3f2e00740 1 client.14910 shutdown 2024-04-25T01:15:16.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.701+0000 7fa3f2e00740 1 -- 172.21.15.31:0/3780250814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f00760 msgr2=0x564966f0d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.701+0000 7fa3f2e00740 1 --2- 172.21.15.31:0/3780250814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f00760 0x564966f0d590 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc003640 tx=0x7fa3dc009fd0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.701+0000 7fa3f2e00740 1 -- 172.21.15.31:0/3780250814 shutdown_connections 2024-04-25T01:15:16.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.701+0000 7fa3f2e00740 1 --2- 172.21.15.31:0/3780250814 >> [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] conn(0x564966f8af90 0x564966f89ca0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.701+0000 7fa3f2e00740 1 --2- 172.21.15.31:0/3780250814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564966f062c0 0x564966f10af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.701+0000 7fa3f2e00740 1 --2- 172.21.15.31:0/3780250814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564966f01150 0x564966f0dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.701+0000 7fa3f2e00740 1 --2- 172.21.15.31:0/3780250814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564966f00760 0x564966f0d590 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.701+0000 7fa3cea00640 0 client.14910 ms_handle_reset on v2:172.21.15.169:6838/4185710218 2024-04-25T01:15:16.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.701+0000 7fa3f2e00740 1 -- 172.21.15.31:0/3780250814 >> 172.21.15.31:0/3780250814 conn(0x564966efbdd0 msgr2=0x564966ef6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:16.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.701+0000 7fa3f2e00740 1 -- 172.21.15.31:0/3780250814 shutdown_connections 2024-04-25T01:15:16.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.701+0000 7fa3f2e00740 1 -- 172.21.15.31:0/3780250814 wait complete. 2024-04-25T01:15:16.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.701+0000 7fa3f2e00740 20 client.14910 trim_cache size 0 max 16384 2024-04-25T01:15:16.691 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T01:15:16.691 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "95849b94-029f-11ef-bc93-c7b262605968", 2024-04-25T01:15:16.691 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 1, 2024-04-25T01:15:16.691 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24565, 2024-04-25T01:15:16.691 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-04-25T01:15:16.691 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-04-25T01:15:16.691 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-04-25T01:15:16.692 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 102.11868651100001, 2024-04-25T01:15:16.692 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 23, 2024-04-25T01:15:16.692 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 89, 2024-04-25T01:15:16.692 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 87, 2024-04-25T01:15:16.692 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 124.016624517 2024-04-25T01:15:16.692 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T01:15:16.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.702+0000 7fa3f0000640 1 -- 172.21.15.31:0/3119825418 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa3d00a2b90 msgr2=0x7fa3d00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.702+0000 7fa3f0000640 1 --2- 172.21.15.31:0/3119825418 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa3d00a2b90 0x7fa3d00a5050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x564966f01550 tx=0x7fa3dc009260 comp rx=0 tx=0).stop 2024-04-25T01:15:16.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.702+0000 7fa3f0000640 1 -- 172.21.15.31:0/3119825418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3e8104b90 msgr2=0x7fa3e819a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.702+0000 7fa3f0000640 1 --2- 172.21.15.31:0/3119825418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3e8104b90 0x7fa3e819a1d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4035040 tx=0x7fa3d4033d90 comp rx=0 tx=0).stop 2024-04-25T01:15:16.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.702+0000 7fa3f0000640 1 -- 172.21.15.31:0/3119825418 shutdown_connections 2024-04-25T01:15:16.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.702+0000 7fa3f0000640 1 --2- 172.21.15.31:0/3119825418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3e8105550 0x7fa3e81a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.702+0000 7fa3f0000640 1 --2- 172.21.15.31:0/3119825418 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa3d00a2b90 0x7fa3d00a5050 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.702+0000 7fa3f0000640 1 --2- 172.21.15.31:0/3119825418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3e8104b90 0x7fa3e819a1d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.702+0000 7fa3f0000640 1 --2- 172.21.15.31:0/3119825418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3e81039a0 0x7fa3e8199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.702+0000 7fa3f0000640 1 -- 172.21.15.31:0/3119825418 >> 172.21.15.31:0/3119825418 conn(0x7fa3e80fef30 msgr2=0x7fa3e8107f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:16.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.702+0000 7fa3f0000640 1 -- 172.21.15.31:0/3119825418 shutdown_connections 2024-04-25T01:15:16.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.702+0000 7fa3f0000640 1 -- 172.21.15.31:0/3119825418 wait complete. 2024-04-25T01:15:16.706 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T01:15:16.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.932+0000 7f776ac00640 1 -- 172.21.15.31:0/4288544792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7764105430 msgr2=0x7f776410bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.932+0000 7f776ac00640 1 --2- 172.21.15.31:0/4288544792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7764105430 0x7f776410bcc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f775c009ca0 tx=0x7f775c02a8c0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.932+0000 7f776ac00640 1 -- 172.21.15.31:0/4288544792 shutdown_connections 2024-04-25T01:15:16.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.932+0000 7f776ac00640 1 --2- 172.21.15.31:0/4288544792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7764105430 0x7f776410bcc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.932+0000 7f776ac00640 1 --2- 172.21.15.31:0/4288544792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7764104a70 0x7f7764104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.932+0000 7f776ac00640 1 --2- 172.21.15.31:0/4288544792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7764103880 0x7f7764103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.932+0000 7f776ac00640 1 -- 172.21.15.31:0/4288544792 >> 172.21.15.31:0/4288544792 conn(0x7f77640fedf0 msgr2=0x7f7764101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:16.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.932+0000 7f776ac00640 1 -- 172.21.15.31:0/4288544792 shutdown_connections 2024-04-25T01:15:16.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.932+0000 7f776ac00640 1 -- 172.21.15.31:0/4288544792 wait complete. 2024-04-25T01:15:16.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.933+0000 7f776ac00640 1 Processor -- start 2024-04-25T01:15:16.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.933+0000 7f776ac00640 1 -- start start 2024-04-25T01:15:16.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.934+0000 7f776ac00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7764103880 0x7f7764199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.934+0000 7f776ac00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7764104a70 0x7f776419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.934+0000 7f776ac00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7764105430 0x7f77641a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.934+0000 7f776ac00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f776410f500 con 0x7f7764103880 2024-04-25T01:15:16.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.934+0000 7f776ac00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f776410f380 con 0x7f7764105430 2024-04-25T01:15:16.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.934+0000 7f776ac00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f776410f680 con 0x7f7764104a70 2024-04-25T01:15:16.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.934+0000 7f7769800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7764103880 0x7f7764199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.934+0000 7f7769800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7764103880 0x7f7764199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36986/0 (socket says 172.21.15.31:36986) 2024-04-25T01:15:16.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.934+0000 7f7769800640 1 -- 172.21.15.31:0/2089312452 learned_addr learned my addr 172.21.15.31:0/2089312452 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:16.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.934+0000 7f7768e00640 1 --2- 172.21.15.31:0/2089312452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7764104a70 0x7f776419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.934+0000 7f776a200640 1 --2- 172.21.15.31:0/2089312452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7764105430 0x7f77641a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.935+0000 7f7768e00640 1 -- 172.21.15.31:0/2089312452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7764105430 msgr2=0x7f77641a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.935+0000 7f7768e00640 1 --2- 172.21.15.31:0/2089312452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7764105430 0x7f77641a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.935+0000 7f7768e00640 1 -- 172.21.15.31:0/2089312452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7764103880 msgr2=0x7f7764199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.935+0000 7f7768e00640 1 --2- 172.21.15.31:0/2089312452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7764103880 0x7f7764199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.935+0000 7f7768e00640 1 -- 172.21.15.31:0/2089312452 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77641a1760 con 0x7f7764104a70 2024-04-25T01:15:16.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.935+0000 7f7769800640 1 --2- 172.21.15.31:0/2089312452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7764103880 0x7f7764199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:16.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.935+0000 7f7768e00640 1 --2- 172.21.15.31:0/2089312452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7764104a70 0x7f776419a080 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f775000e9f0 tx=0x7f775000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:16.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.935+0000 7f7762000640 1 -- 172.21.15.31:0/2089312452 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775000cd80 con 0x7f7764104a70 2024-04-25T01:15:16.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.935+0000 7f776ac00640 1 -- 172.21.15.31:0/2089312452 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77641a1a50 con 0x7f7764104a70 2024-04-25T01:15:16.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.935+0000 7f7762000640 1 -- 172.21.15.31:0/2089312452 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77500035f0 con 0x7f7764104a70 2024-04-25T01:15:16.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.935+0000 7f7762000640 1 -- 172.21.15.31:0/2089312452 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7750010670 con 0x7f7764104a70 2024-04-25T01:15:16.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.936+0000 7f776ac00640 1 -- 172.21.15.31:0/2089312452 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77641a1ff0 con 0x7f7764104a70 2024-04-25T01:15:16.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.939+0000 7f776f005740 1 Processor -- start 2024-04-25T01:15:16.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.939+0000 7f776f005740 1 -- start start 2024-04-25T01:15:16.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.939+0000 7f776f005740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559f0119e970 0x559f0119ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.939+0000 7f776f005740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f0119f360 0x559f011a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.939+0000 7f776f005740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559f011a44d0 0x559f011a6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.939+0000 7f776f005740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559f0109eed0 con 0x559f0119e970 2024-04-25T01:15:16.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.939+0000 7f776f005740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x559f01101df0 con 0x559f0119f360 2024-04-25T01:15:16.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.939+0000 7f776f005740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559f010aa8d0 con 0x559f011a44d0 2024-04-25T01:15:16.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.940+0000 7f776a200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559f0119e970 0x559f0119ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.940+0000 7f7769800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f0119f360 0x559f011a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.940+0000 7f776a200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559f0119e970 0x559f0119ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36990/0 (socket says 172.21.15.31:36990) 2024-04-25T01:15:16.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.940+0000 7f7769800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f0119f360 0x559f011a3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:52808/0 (socket says 172.21.15.31:52808) 2024-04-25T01:15:16.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.940+0000 7f776a200640 1 -- 172.21.15.31:0/55268534 learned_addr learned my addr 172.21.15.31:0/55268534 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:16.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.940+0000 7f7762000640 1 -- 172.21.15.31:0/2089312452 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f77500037a0 con 0x7f7764104a70 2024-04-25T01:15:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.940+0000 7f776a200640 1 -- 172.21.15.31:0/55268534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559f011a44d0 msgr2=0x559f011a6960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T01:15:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.940+0000 7f776a200640 1 --2- 172.21.15.31:0/55268534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559f011a44d0 0x559f011a6960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.940+0000 7f776a200640 1 -- 172.21.15.31:0/55268534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f0119f360 msgr2=0x559f011a3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.940+0000 7f776a200640 1 --2- 172.21.15.31:0/55268534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f0119f360 0x559f011a3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.940+0000 7f776a200640 1 -- 172.21.15.31:0/55268534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f00e162d0 con 0x559f0119e970 2024-04-25T01:15:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.940+0000 7f7769800640 1 --2- 172.21.15.31:0/55268534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f0119f360 0x559f011a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.940+0000 7f776a200640 1 --2- 172.21.15.31:0/55268534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559f0119e970 0x559f0119ed90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f775c009ca0 tx=0x7f775c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea590a294e2acfd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.941+0000 7f7756a00640 1 -- 172.21.15.31:0/55268534 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c0098c0 con 0x559f0119e970 2024-04-25T01:15:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.941+0000 7f7756a00640 1 -- 172.21.15.31:0/55268534 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f775c036040 con 0x559f0119e970 2024-04-25T01:15:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.941+0000 7f7756a00640 1 -- 172.21.15.31:0/55268534 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c004290 con 0x559f0119e970 2024-04-25T01:15:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.941+0000 7f7762000640 1 --2- 172.21.15.31:0/2089312452 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f774c0a2b90 0x7f774c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.941+0000 7f776f005740 1 -- 172.21.15.31:0/55268534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559f0119e970 msgr2=0x559f0119ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.941+0000 7f776f005740 1 --2- 172.21.15.31:0/55268534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559f0119e970 0x559f0119ed90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f775c009ca0 tx=0x7f775c004a40 comp rx=0 tx=0).stop 2024-04-25T01:15:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.941+0000 7f7762000640 1 -- 172.21.15.31:0/2089312452 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f77500cf330 con 0x7f7764104a70 2024-04-25T01:15:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.942+0000 7f7769800640 1 --2- 172.21.15.31:0/2089312452 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f774c0a2b90 0x7f774c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.942+0000 7f776f005740 1 -- 172.21.15.31:0/55268534 shutdown_connections 2024-04-25T01:15:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.942+0000 7f776f005740 1 --2- 172.21.15.31:0/55268534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559f011a44d0 0x559f011a6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.942+0000 7f776f005740 1 --2- 172.21.15.31:0/55268534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f0119f360 0x559f011a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.942+0000 7f776f005740 1 --2- 172.21.15.31:0/55268534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559f0119e970 0x559f0119ed90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.942+0000 7f776f005740 1 -- 172.21.15.31:0/55268534 >> 172.21.15.31:0/55268534 conn(0x559f01199fe0 msgr2=0x559f0119a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.942+0000 7f776f005740 1 -- 172.21.15.31:0/55268534 shutdown_connections 2024-04-25T01:15:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.942+0000 7f776f005740 1 -- 172.21.15.31:0/55268534 wait complete. 2024-04-25T01:15:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.942+0000 7f7769800640 1 --2- 172.21.15.31:0/2089312452 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f774c0a2b90 0x7f774c0a5050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7758009fd0 tx=0x7f7758005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.942+0000 7f776f005740 1 -- start start 2024-04-25T01:15:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.943+0000 7f776f005740 1 Processor -- start 2024-04-25T01:15:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.943+0000 7f776f005740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f0119e970 0x559f011a8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.943+0000 7f776f005740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559f0119f360 0x559f011a8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.943+0000 7f776f005740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559f011a44d0 0x559f011affd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.943+0000 7f776f005740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559f010bdd20 con 0x559f011a44d0 2024-04-25T01:15:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.943+0000 7f776f005740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x559f010fddb0 con 0x559f0119e970 2024-04-25T01:15:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.943+0000 7f776f005740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559f011a7240 con 0x559f0119f360 2024-04-25T01:15:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.943+0000 7f776f005740 10 client.0 _ll_register_callbacks cb 0x559f010f8f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:15:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.943+0000 7f776a200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559f011a44d0 0x559f011affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.943+0000 7f7768e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559f0119f360 0x559f011a8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.943+0000 7f7769800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f0119e970 0x559f011a8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.943+0000 7f7768e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559f0119f360 0x559f011a8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:58222/0 (socket says 172.21.15.31:58222) 2024-04-25T01:15:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.943+0000 7f7768e00640 1 -- 172.21.15.31:0/1470407463 learned_addr learned my addr 172.21.15.31:0/1470407463 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.944+0000 7f7768e00640 1 -- 172.21.15.31:0/1470407463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f0119e970 msgr2=0x559f011a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.944+0000 7f7768e00640 1 --2- 172.21.15.31:0/1470407463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f0119e970 0x559f011a8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.944+0000 7f7768e00640 1 -- 172.21.15.31:0/1470407463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559f011a44d0 msgr2=0x559f011affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.944+0000 7f7768e00640 1 --2- 172.21.15.31:0/1470407463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559f011a44d0 0x559f011affd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.944+0000 7f7768e00640 1 -- 172.21.15.31:0/1470407463 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f00ee06b0 con 0x559f0119f360 2024-04-25T01:15:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.944+0000 7f7769800640 1 --2- 172.21.15.31:0/1470407463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f0119e970 0x559f011a8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.944+0000 7f776a200640 1 --2- 172.21.15.31:0/1470407463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559f011a44d0 0x559f011affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.944+0000 7f7768e00640 1 --2- 172.21.15.31:0/1470407463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559f0119f360 0x559f011a8f90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7750098550 tx=0x7f775009c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a21e7756f3e275fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.944+0000 7f774b400640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:15:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.944+0000 7f774b400640 1 -- 172.21.15.31:0/1470407463 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77500a47f0 con 0x559f0119f360 2024-04-25T01:15:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.944+0000 7f774b400640 1 -- 172.21.15.31:0/1470407463 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f7750003b30 con 0x559f0119f360 2024-04-25T01:15:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.944+0000 7f776f005740 1 -- 172.21.15.31:0/1470407463 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f775c036040 con 0x559f0119f360 2024-04-25T01:15:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.945+0000 7f774b400640 1 -- 172.21.15.31:0/1470407463 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f775007f050 con 0x559f0119f360 2024-04-25T01:15:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.945+0000 7f776f005740 10 client.25051 fetch_fsmap learned FSMap version 23 2024-04-25T01:15:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.945+0000 7f776f005740 1 -- 172.21.15.31:0/1470407463 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x559f011b33b0 con 0x559f0119f360 2024-04-25T01:15:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.945+0000 7f774b400640 1 -- 172.21.15.31:0/1470407463 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f775c036040 con 0x559f0119f360 2024-04-25T01:15:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.945+0000 7f774b400640 1 -- 172.21.15.31:0/1470407463 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f77500978d0 con 0x559f0119f360 2024-04-25T01:15:16.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.945+0000 7f776f005740 10 client.25051 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T01:15:16.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.945+0000 7f776f005740 10 client.25051 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T01:15:16.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.945+0000 7f776f005740 20 client.25051 populate_metadata read hostname 'smithi031' 2024-04-25T01:15:16.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.945+0000 7f776f005740 1 --2- 172.21.15.31:0/1470407463 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x559f011b59b0 0x559f011b6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.945+0000 7f776f005740 4 client.25051 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:15:16.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.945+0000 7f776f005740 1 -- 172.21.15.31:0/1470407463 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f0110f330 con 0x559f011b59b0 2024-04-25T01:15:16.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.946+0000 7f7769800640 1 --2- 172.21.15.31:0/1470407463 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x559f011b59b0 0x559f011b6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.946+0000 7f7769800640 1 --2- 172.21.15.31:0/1470407463 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x559f011b59b0 0x559f011b6e20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=164a81c711f2b10a server_cookie=280ba16c06f79998 in_seq=0 out_seq=0 2024-04-25T01:15:16.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.946+0000 7f774b400640 10 client.25051 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:15:16.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.947+0000 7f774b400640 1 -- 172.21.15.31:0/1470407463 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559f0110f330 con 0x559f011b59b0 2024-04-25T01:15:16.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.947+0000 7f774b400640 10 client.25051 handle_command_reply: tid=0 2024-04-25T01:15:16.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.947+0000 7f776f005740 1 client.25051 shutdown 2024-04-25T01:15:16.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.947+0000 7f776f005740 1 -- 172.21.15.31:0/1470407463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559f0119f360 msgr2=0x559f011a8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.947+0000 7f776f005740 1 --2- 172.21.15.31:0/1470407463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559f0119f360 0x559f011a8f90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7750098550 tx=0x7f775009c3c0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.948+0000 7f776f005740 1 -- 172.21.15.31:0/1470407463 shutdown_connections 2024-04-25T01:15:16.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.948+0000 7f776f005740 1 --2- 172.21.15.31:0/1470407463 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x559f011b59b0 0x559f011b6e20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.948+0000 7f776f005740 1 --2- 172.21.15.31:0/1470407463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559f011a44d0 0x559f011affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.948+0000 7f776f005740 1 --2- 172.21.15.31:0/1470407463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559f0119f360 0x559f011a8f90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.948+0000 7f776f005740 1 --2- 172.21.15.31:0/1470407463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f0119e970 0x559f011a8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.948+0000 7f774b400640 0 client.25051 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:15:16.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.948+0000 7f776f005740 1 -- 172.21.15.31:0/1470407463 >> 172.21.15.31:0/1470407463 conn(0x559f01199fe0 msgr2=0x559f01199120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:16.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.948+0000 7f776f005740 1 -- 172.21.15.31:0/1470407463 shutdown_connections 2024-04-25T01:15:16.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.948+0000 7f776f005740 1 -- 172.21.15.31:0/1470407463 wait complete. 2024-04-25T01:15:16.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.948+0000 7f776f005740 20 client.25051 trim_cache size 0 max 16384 2024-04-25T01:15:16.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.968+0000 7f776f005740 1 Processor -- start 2024-04-25T01:15:16.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.968+0000 7f776f005740 1 -- start start 2024-04-25T01:15:16.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.969+0000 7f776f005740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f0119e970 0x559f011a7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.969+0000 7f776f005740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559f0119f360 0x559f011aff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.969+0000 7f776f005740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559f011a44d0 0x559f011a0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.969+0000 7f776f005740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559f011b5160 con 0x559f0119f360 2024-04-25T01:15:16.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.969+0000 7f776f005740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x559f011b5830 con 0x559f0119e970 2024-04-25T01:15:16.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.969+0000 7f776f005740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559f011b5f00 con 0x559f011a44d0 2024-04-25T01:15:16.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.969+0000 7f7768e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559f0119f360 0x559f011aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.969+0000 7f7769800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f0119e970 0x559f011a7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.969+0000 7f7769800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f0119e970 0x559f011a7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:52818/0 (socket says 172.21.15.31:52818) 2024-04-25T01:15:16.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.969+0000 7f7769800640 1 -- 172.21.15.31:0/32700195 learned_addr learned my addr 172.21.15.31:0/32700195 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:16.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.969+0000 7f7768e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559f0119f360 0x559f011aff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37016/0 (socket says 172.21.15.31:37016) 2024-04-25T01:15:16.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.969+0000 7f776a200640 1 --2- 172.21.15.31:0/32700195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559f011a44d0 0x559f011a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.970+0000 7f7769800640 1 -- 172.21.15.31:0/32700195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559f011a44d0 msgr2=0x559f011a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:16.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.970+0000 7f7769800640 1 --2- 172.21.15.31:0/32700195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559f011a44d0 0x559f011a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.970+0000 7f7769800640 1 -- 172.21.15.31:0/32700195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559f0119f360 msgr2=0x559f011aff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:16.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.970+0000 7f7769800640 1 --2- 172.21.15.31:0/32700195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559f0119f360 0x559f011aff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.970+0000 7f7769800640 1 -- 172.21.15.31:0/32700195 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f0119fbf0 con 0x559f0119e970 2024-04-25T01:15:16.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.970+0000 7f7769800640 1 --2- 172.21.15.31:0/32700195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f0119e970 0x559f011a7610 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7758009600 tx=0x7f7758022040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a81439a76f5c1a84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:16.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.970+0000 7f7756000640 1 -- 172.21.15.31:0/32700195 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775802b470 con 0x559f0119e970 2024-04-25T01:15:16.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.970+0000 7f7756000640 1 -- 172.21.15.31:0/32700195 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7758011d70 con 0x559f0119e970 2024-04-25T01:15:16.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.970+0000 7f7756000640 1 -- 172.21.15.31:0/32700195 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77580053c0 con 0x559f0119e970 2024-04-25T01:15:16.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.971+0000 7f776f005740 1 -- 172.21.15.31:0/32700195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f0119e970 msgr2=0x559f011a7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.971+0000 7f776f005740 1 --2- 172.21.15.31:0/32700195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f0119e970 0x559f011a7610 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7758009600 tx=0x7f7758022040 comp rx=0 tx=0).stop 2024-04-25T01:15:16.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.971+0000 7f776f005740 1 -- 172.21.15.31:0/32700195 shutdown_connections 2024-04-25T01:15:16.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.971+0000 7f776f005740 1 --2- 172.21.15.31:0/32700195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559f011a44d0 0x559f011a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.971+0000 7f776f005740 1 --2- 172.21.15.31:0/32700195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559f0119f360 0x559f011aff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.971+0000 7f776f005740 1 --2- 172.21.15.31:0/32700195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f0119e970 0x559f011a7610 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.971+0000 7f776f005740 1 -- 172.21.15.31:0/32700195 >> 172.21.15.31:0/32700195 conn(0x559f01199fe0 msgr2=0x559f0119a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:16.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.971+0000 7f776f005740 1 -- 172.21.15.31:0/32700195 shutdown_connections 2024-04-25T01:15:16.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.971+0000 7f776f005740 1 -- 172.21.15.31:0/32700195 wait complete. 2024-04-25T01:15:16.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.971+0000 7f776f005740 1 -- start start 2024-04-25T01:15:16.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.972+0000 7f776f005740 1 Processor -- start 2024-04-25T01:15:16.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.972+0000 7f776f005740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559f0119e970 0x559f011adf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.972+0000 7f776f005740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559f0119f360 0x559f01217940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.972+0000 7f776f005740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f011a44d0 0x559f01219e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.972+0000 7f776f005740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559f011b6ca0 con 0x559f0119e970 2024-04-25T01:15:16.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.972+0000 7f776f005740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x559f011b65d0 con 0x559f011a44d0 2024-04-25T01:15:16.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.972+0000 7f776f005740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559f011b7370 con 0x559f0119f360 2024-04-25T01:15:16.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.972+0000 7f776f005740 10 client.0 _ll_register_callbacks cb 0x7f7750097700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:15:16.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.972+0000 7f7768e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559f0119f360 0x559f01217940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.972+0000 7f7769800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559f0119e970 0x559f011adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.972+0000 7f7768e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559f0119f360 0x559f01217940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:58242/0 (socket says 172.21.15.31:58242) 2024-04-25T01:15:16.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.972+0000 7f7768e00640 1 -- 172.21.15.31:0/3914352004 learned_addr learned my addr 172.21.15.31:0/3914352004 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:16.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.972+0000 7f776a200640 1 --2- 172.21.15.31:0/3914352004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f011a44d0 0x559f01219e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.973+0000 7f7768e00640 1 -- 172.21.15.31:0/3914352004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f011a44d0 msgr2=0x559f01219e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:16.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.973+0000 7f7768e00640 1 --2- 172.21.15.31:0/3914352004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f011a44d0 0x559f01219e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.973+0000 7f7768e00640 1 -- 172.21.15.31:0/3914352004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559f0119e970 msgr2=0x559f011adf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:16.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.973+0000 7f7768e00640 1 --2- 172.21.15.31:0/3914352004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559f0119e970 0x559f011adf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.973+0000 7f7768e00640 1 -- 172.21.15.31:0/3914352004 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f0121a420 con 0x559f0119f360 2024-04-25T01:15:16.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.973+0000 7f776a200640 1 --2- 172.21.15.31:0/3914352004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f011a44d0 0x559f01219e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:15:16.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.973+0000 7f7768e00640 1 --2- 172.21.15.31:0/3914352004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559f0119f360 0x559f01217940 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7750097c90 tx=0x7f7750097e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25ead412a0676f0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:16.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.973+0000 7f774b400640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:15:16.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.973+0000 7f774b400640 1 -- 172.21.15.31:0/3914352004 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77500a47f0 con 0x559f0119f360 2024-04-25T01:15:16.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.973+0000 7f774b400640 1 -- 172.21.15.31:0/3914352004 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f77500981f0 con 0x559f0119f360 2024-04-25T01:15:16.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.973+0000 7f776f005740 1 -- 172.21.15.31:0/3914352004 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7758011d70 con 0x559f0119f360 2024-04-25T01:15:16.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.973+0000 7f774b400640 1 -- 172.21.15.31:0/3914352004 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f775009aae0 con 0x559f0119f360 2024-04-25T01:15:16.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.973+0000 7f776f005740 10 client.25057 fetch_fsmap learned FSMap version 23 2024-04-25T01:15:16.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.973+0000 7f776f005740 1 -- 172.21.15.31:0/3914352004 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x559f0121b7c0 con 0x559f0119f360 2024-04-25T01:15:16.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.974+0000 7f774b400640 1 -- 172.21.15.31:0/3914352004 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7758011d70 con 0x559f0119f360 2024-04-25T01:15:16.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.974+0000 7f774b400640 1 -- 172.21.15.31:0/3914352004 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f77500a10c0 con 0x559f0119f360 2024-04-25T01:15:16.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.974+0000 7f776f005740 10 client.25057 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T01:15:16.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.974+0000 7f776f005740 10 client.25057 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T01:15:16.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.974+0000 7f776f005740 20 client.25057 populate_metadata read hostname 'smithi031' 2024-04-25T01:15:16.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.974+0000 7f776f005740 1 --2- 172.21.15.31:0/3914352004 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x559f012201d0 0x559f0121eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:16.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.974+0000 7f776f005740 4 client.25057 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T01:15:16.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.974+0000 7f776f005740 1 -- 172.21.15.31:0/3914352004 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "status"}) -- 0x559f01109680 con 0x559f012201d0 2024-04-25T01:15:16.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.974+0000 7f7769800640 1 --2- 172.21.15.31:0/3914352004 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x559f012201d0 0x559f0121eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:16.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.975+0000 7f7769800640 1 --2- 172.21.15.31:0/3914352004 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x559f012201d0 0x559f0121eee0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bb85d545b0fa34 server_cookie=a57e4125f1d7fdee in_seq=0 out_seq=0 2024-04-25T01:15:16.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.975+0000 7f774b400640 10 client.25057 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:15:16.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.975+0000 7f774b400640 1 -- 172.21.15.31:0/3914352004 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x559f01109680 con 0x559f012201d0 2024-04-25T01:15:16.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.975+0000 7f774b400640 10 client.25057 handle_command_reply: tid=0 2024-04-25T01:15:16.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.975+0000 7f776f005740 1 client.25057 shutdown 2024-04-25T01:15:16.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.975+0000 7f776f005740 1 -- 172.21.15.31:0/3914352004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559f0119f360 msgr2=0x559f01217940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.975+0000 7f776f005740 1 --2- 172.21.15.31:0/3914352004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559f0119f360 0x559f01217940 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7750097c90 tx=0x7f7750097e90 comp rx=0 tx=0).stop 2024-04-25T01:15:16.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.975+0000 7f776f005740 1 -- 172.21.15.31:0/3914352004 shutdown_connections 2024-04-25T01:15:16.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.975+0000 7f776f005740 1 --2- 172.21.15.31:0/3914352004 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x559f012201d0 0x559f0121eee0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.975+0000 7f776f005740 1 --2- 172.21.15.31:0/3914352004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559f011a44d0 0x559f01219e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.975+0000 7f776f005740 1 --2- 172.21.15.31:0/3914352004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559f0119f360 0x559f01217940 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.975+0000 7f776f005740 1 --2- 172.21.15.31:0/3914352004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559f0119e970 0x559f011adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.975+0000 7f774b400640 0 client.25057 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:15:16.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.975+0000 7f776f005740 1 -- 172.21.15.31:0/3914352004 >> 172.21.15.31:0/3914352004 conn(0x559f01199fe0 msgr2=0x559f011a8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:16.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.976+0000 7f776f005740 1 -- 172.21.15.31:0/3914352004 shutdown_connections 2024-04-25T01:15:16.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.976+0000 7f776f005740 1 -- 172.21.15.31:0/3914352004 wait complete. 2024-04-25T01:15:16.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.976+0000 7f776f005740 20 client.25057 trim_cache size 0 max 16384 2024-04-25T01:15:16.966 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T01:15:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "95849b94-029f-11ef-bc93-c7b262605968", 2024-04-25T01:15:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2024-04-25T01:15:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24596, 2024-04-25T01:15:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-04-25T01:15:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-04-25T01:15:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-04-25T01:15:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 105.425220322, 2024-04-25T01:15:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 23, 2024-04-25T01:15:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 89, 2024-04-25T01:15:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 87, 2024-04-25T01:15:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 112.799462738 2024-04-25T01:15:16.967 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T01:15:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.976+0000 7f776ac00640 1 -- 172.21.15.31:0/2089312452 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f774c0a2b90 msgr2=0x7f774c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.976+0000 7f776ac00640 1 --2- 172.21.15.31:0/2089312452 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f774c0a2b90 0x7f774c0a5050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7758009fd0 tx=0x7f7758005f90 comp rx=0 tx=0).stop 2024-04-25T01:15:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.976+0000 7f776ac00640 1 -- 172.21.15.31:0/2089312452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7764104a70 msgr2=0x7f776419a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.976+0000 7f776ac00640 1 --2- 172.21.15.31:0/2089312452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7764104a70 0x7f776419a080 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f775000e9f0 tx=0x7f775000eeb0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.976+0000 7f776ac00640 1 -- 172.21.15.31:0/2089312452 shutdown_connections 2024-04-25T01:15:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.976+0000 7f776ac00640 1 --2- 172.21.15.31:0/2089312452 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f774c0a2b90 0x7f774c0a5050 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.976+0000 7f776ac00640 1 --2- 172.21.15.31:0/2089312452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7764105430 0x7f77641a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.976+0000 7f776ac00640 1 --2- 172.21.15.31:0/2089312452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7764104a70 0x7f776419a080 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.976+0000 7f776ac00640 1 --2- 172.21.15.31:0/2089312452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7764103880 0x7f7764199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.976+0000 7f776ac00640 1 -- 172.21.15.31:0/2089312452 >> 172.21.15.31:0/2089312452 conn(0x7f77640fedf0 msgr2=0x7f7764107d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:16.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.977+0000 7f776ac00640 1 -- 172.21.15.31:0/2089312452 shutdown_connections 2024-04-25T01:15:16.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:16.977+0000 7f776ac00640 1 -- 172.21.15.31:0/2089312452 wait complete. 2024-04-25T01:15:16.982 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T01:15:16.982 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T01:15:16.992 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T01:15:16.993 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T01:15:17.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:17 smithi031 ceph-mon[28053]: pgmap v371: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T01:15:17.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:17 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3966636429' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:15:17.444 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:17 smithi169 ceph-mon[32759]: pgmap v371: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T01:15:17.444 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:17 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3966636429' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:15:17.446 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:17 smithi040 ceph-mon[32518]: pgmap v371: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T01:15:17.446 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:17 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3966636429' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:15:17.461 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:15:19.247 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:19 smithi031 ceph-mon[28053]: pgmap v372: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T01:15:19.443 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:19 smithi169 ceph-mon[32759]: pgmap v372: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T01:15:19.446 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:19 smithi040 ceph-mon[32518]: pgmap v372: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T01:15:19.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.855+0000 7ff0e1000700 1 -- 172.21.15.31:0/2714492567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0dc105650 msgr2=0x7ff0dc107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:19.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.855+0000 7ff0e1000700 1 --2- 172.21.15.31:0/2714492567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0dc105650 0x7ff0dc107a40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009990 tx=0x7ff0c8009ca0 comp rx=0 tx=0).stop 2024-04-25T01:15:19.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.855+0000 7ff0e1000700 1 -- 172.21.15.31:0/2714492567 shutdown_connections 2024-04-25T01:15:19.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.855+0000 7ff0e1000700 1 --2- 172.21.15.31:0/2714492567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0dc107f80 0x7ff0dc10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:19.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.855+0000 7ff0e1000700 1 --2- 172.21.15.31:0/2714492567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0dc102d20 0x7ff0dc105110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:19.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.855+0000 7ff0e1000700 1 --2- 172.21.15.31:0/2714492567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0dc105650 0x7ff0dc107a40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:19.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.855+0000 7ff0e1000700 1 -- 172.21.15.31:0/2714492567 >> 172.21.15.31:0/2714492567 conn(0x7ff0dc0fc730 msgr2=0x7ff0dc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:19.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.855+0000 7ff0e1000700 1 -- 172.21.15.31:0/2714492567 shutdown_connections 2024-04-25T01:15:19.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.855+0000 7ff0e1000700 1 -- 172.21.15.31:0/2714492567 wait complete. 2024-04-25T01:15:19.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.856+0000 7ff0e1000700 1 Processor -- start 2024-04-25T01:15:19.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.857+0000 7ff0e1000700 1 -- start start 2024-04-25T01:15:19.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.857+0000 7ff0e1000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0dc102d20 0x7ff0dc19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:19.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.857+0000 7ff0e1000700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0dc105650 0x7ff0dc19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:19.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.857+0000 7ff0e1000700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0dc107f80 0x7ff0dc1a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:19.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.857+0000 7ff0e1000700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc10de70 con 0x7ff0dc107f80 2024-04-25T01:15:19.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.857+0000 7ff0e1000700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc10dcf0 con 0x7ff0dc105650 2024-04-25T01:15:19.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.857+0000 7ff0e1000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc10dff0 con 0x7ff0dc102d20 2024-04-25T01:15:19.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.857+0000 7ff0d9e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0dc102d20 0x7ff0dc19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:19.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.857+0000 7ff0d9400700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0dc105650 0x7ff0dc19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:19.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.857+0000 7ff0d9e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0dc102d20 0x7ff0dc19a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:58266/0 (socket says 172.21.15.31:58266) 2024-04-25T01:15:19.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.857+0000 7ff0d9e00700 1 -- 172.21.15.31:0/640285047 learned_addr learned my addr 172.21.15.31:0/640285047 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:19.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.858+0000 7ff0da800700 1 --2- 172.21.15.31:0/640285047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0dc107f80 0x7ff0dc1a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:19.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.858+0000 7ff0d9e00700 1 -- 172.21.15.31:0/640285047 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0dc105650 msgr2=0x7ff0dc19ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:19.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.858+0000 7ff0d9e00700 1 --2- 172.21.15.31:0/640285047 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0dc105650 0x7ff0dc19ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:19.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.858+0000 7ff0d9e00700 1 -- 172.21.15.31:0/640285047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0dc107f80 msgr2=0x7ff0dc1a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:19.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.858+0000 7ff0d9e00700 1 --2- 172.21.15.31:0/640285047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0dc107f80 0x7ff0dc1a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:19.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.858+0000 7ff0d9e00700 1 -- 172.21.15.31:0/640285047 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0dc1a2370 con 0x7ff0dc102d20 2024-04-25T01:15:19.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.858+0000 7ff0d9e00700 1 --2- 172.21.15.31:0/640285047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0dc102d20 0x7ff0dc19a710 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00e9d0 tx=0x7ff0cc00ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:19.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.858+0000 7ff0d2a00700 1 -- 172.21.15.31:0/640285047 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc00cbd0 con 0x7ff0dc102d20 2024-04-25T01:15:19.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.859+0000 7ff0d2a00700 1 -- 172.21.15.31:0/640285047 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0cc00ee70 con 0x7ff0dc102d20 2024-04-25T01:15:19.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.859+0000 7ff0e1000700 1 -- 172.21.15.31:0/640285047 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0dc1a2660 con 0x7ff0dc102d20 2024-04-25T01:15:19.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.859+0000 7ff0d2a00700 1 -- 172.21.15.31:0/640285047 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc018660 con 0x7ff0dc102d20 2024-04-25T01:15:19.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.859+0000 7ff0e1000700 1 -- 172.21.15.31:0/640285047 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0dc1a2bf0 con 0x7ff0dc102d20 2024-04-25T01:15:19.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.860+0000 7ff0e1000700 1 -- 172.21.15.31:0/640285047 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0dc109b00 con 0x7ff0dc102d20 2024-04-25T01:15:19.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.862+0000 7ff0d2a00700 1 -- 172.21.15.31:0/640285047 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff0cc010430 con 0x7ff0dc102d20 2024-04-25T01:15:19.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.863+0000 7ff0d2a00700 1 --2- 172.21.15.31:0/640285047 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff0c00a2ca0 0x7ff0c00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:19.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.864+0000 7ff0d2a00700 1 -- 172.21.15.31:0/640285047 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7ff0cc0cf3f0 con 0x7ff0dc102d20 2024-04-25T01:15:19.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.864+0000 7ff0d9400700 1 --2- 172.21.15.31:0/640285047 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff0c00a2ca0 0x7ff0c00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:19.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.864+0000 7ff0d9400700 1 --2- 172.21.15.31:0/640285047 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff0c00a2ca0 0x7ff0c00a5160 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800b7b0 tx=0x7ff0c800b880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:19.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:19.865+0000 7ff0d2a00700 1 -- 172.21.15.31:0/640285047 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff0cc09c340 con 0x7ff0dc102d20 2024-04-25T01:15:20.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:20.112+0000 7ff0e1000700 1 -- 172.21.15.31:0/640285047 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7ff0dc062670 con 0x7ff0c00a2ca0 2024-04-25T01:15:20.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:20.113+0000 7ff0d2a00700 1 -- 172.21.15.31:0/640285047 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7ff0dc062670 con 0x7ff0c00a2ca0 2024-04-25T01:15:20.100 INFO:teuthology.orchestra.run.smithi031.stdout:Backend: cephadm 2024-04-25T01:15:20.100 INFO:teuthology.orchestra.run.smithi031.stdout:Available: Yes 2024-04-25T01:15:20.100 INFO:teuthology.orchestra.run.smithi031.stdout:Paused: No 2024-04-25T01:15:20.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:20.116+0000 7ff0e1000700 1 -- 172.21.15.31:0/640285047 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff0c00a2ca0 msgr2=0x7ff0c00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:20.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:20.116+0000 7ff0e1000700 1 --2- 172.21.15.31:0/640285047 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff0c00a2ca0 0x7ff0c00a5160 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800b7b0 tx=0x7ff0c800b880 comp rx=0 tx=0).stop 2024-04-25T01:15:20.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:20.116+0000 7ff0e1000700 1 -- 172.21.15.31:0/640285047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0dc102d20 msgr2=0x7ff0dc19a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:20.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:20.116+0000 7ff0e1000700 1 --2- 172.21.15.31:0/640285047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0dc102d20 0x7ff0dc19a710 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00e9d0 tx=0x7ff0cc00ece0 comp rx=0 tx=0).stop 2024-04-25T01:15:20.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:20.117+0000 7ff0e1000700 1 -- 172.21.15.31:0/640285047 shutdown_connections 2024-04-25T01:15:20.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:20.117+0000 7ff0e1000700 1 --2- 172.21.15.31:0/640285047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0dc107f80 0x7ff0dc1a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:20.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:20.117+0000 7ff0e1000700 1 --2- 172.21.15.31:0/640285047 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff0c00a2ca0 0x7ff0c00a5160 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:20.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:20.117+0000 7ff0e1000700 1 --2- 172.21.15.31:0/640285047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0dc102d20 0x7ff0dc19a710 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:20.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:20.117+0000 7ff0e1000700 1 --2- 172.21.15.31:0/640285047 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0dc105650 0x7ff0dc19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:20.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:20.117+0000 7ff0e1000700 1 -- 172.21.15.31:0/640285047 >> 172.21.15.31:0/640285047 conn(0x7ff0dc0fc730 msgr2=0x7ff0dc102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:20.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:20.117+0000 7ff0e1000700 1 -- 172.21.15.31:0/640285047 shutdown_connections 2024-04-25T01:15:20.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:20.117+0000 7ff0e1000700 1 -- 172.21.15.31:0/640285047 wait complete. 2024-04-25T01:15:20.999 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T01:15:21.453 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:21 smithi169 ceph-mon[32759]: from='client.25063 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:15:21.453 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:21 smithi169 ceph-mon[32759]: pgmap v373: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:21.453 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:21 smithi040 ceph-mon[32518]: from='client.25063 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:15:21.454 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:21 smithi040 ceph-mon[32518]: pgmap v373: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:21.457 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:15:21.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:21 smithi031 ceph-mon[28053]: from='client.25063 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:15:21.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:21 smithi031 ceph-mon[28053]: pgmap v373: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:23.378 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:23 smithi031 ceph-mon[28053]: pgmap v374: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:23.443 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:23 smithi169 ceph-mon[32759]: pgmap v374: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:23.446 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:23 smithi040 ceph-mon[32518]: pgmap v374: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:23.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.687+0000 7f6a51e00700 1 -- 172.21.15.31:0/2742404081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a4c101e20 msgr2=0x7f6a4c1022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:23.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.687+0000 7f6a51e00700 1 --2- 172.21.15.31:0/2742404081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a4c101e20 0x7f6a4c1022a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009f90 tx=0x7f6a4400b330 comp rx=0 tx=0).stop 2024-04-25T01:15:23.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.687+0000 7f6a51e00700 1 -- 172.21.15.31:0/2742404081 shutdown_connections 2024-04-25T01:15:23.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.687+0000 7f6a51e00700 1 --2- 172.21.15.31:0/2742404081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a4c101e20 0x7f6a4c1022a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:23.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.687+0000 7f6a51e00700 1 --2- 172.21.15.31:0/2742404081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a4c100b00 0x7f6a4c100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:23.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.687+0000 7f6a51e00700 1 --2- 172.21.15.31:0/2742404081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a4c101460 0x7f6a4c1018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:23.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.687+0000 7f6a51e00700 1 -- 172.21.15.31:0/2742404081 >> 172.21.15.31:0/2742404081 conn(0x7f6a4c0fc720 msgr2=0x7f6a4c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:23.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.687+0000 7f6a51e00700 1 -- 172.21.15.31:0/2742404081 shutdown_connections 2024-04-25T01:15:23.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.687+0000 7f6a51e00700 1 -- 172.21.15.31:0/2742404081 wait complete. 2024-04-25T01:15:23.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.689+0000 7f6a51e00700 1 Processor -- start 2024-04-25T01:15:23.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.689+0000 7f6a51e00700 1 -- start start 2024-04-25T01:15:23.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.689+0000 7f6a51e00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a4c100b00 0x7f6a4c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:23.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.689+0000 7f6a51e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a4c101460 0x7f6a4c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:23.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.690+0000 7f6a51e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a4c101e20 0x7f6a4c1a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:23.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.690+0000 7f6a51e00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4c110050 con 0x7f6a4c101e20 2024-04-25T01:15:23.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.690+0000 7f6a51e00700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4c10fed0 con 0x7f6a4c100b00 2024-04-25T01:15:23.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.690+0000 7f6a51e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4c1101d0 con 0x7f6a4c101460 2024-04-25T01:15:23.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.690+0000 7f6a4aa00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a4c100b00 0x7f6a4c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:23.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.690+0000 7f6a4aa00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a4c100b00 0x7f6a4c19a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:52866/0 (socket says 172.21.15.31:52866) 2024-04-25T01:15:23.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.690+0000 7f6a4a000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a4c101460 0x7f6a4c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:23.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.690+0000 7f6a4aa00700 1 -- 172.21.15.31:0/1458886731 learned_addr learned my addr 172.21.15.31:0/1458886731 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:23.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.690+0000 7f6a4b400700 1 --2- 172.21.15.31:0/1458886731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a4c101e20 0x7f6a4c1a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:23.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.690+0000 7f6a4a000700 1 -- 172.21.15.31:0/1458886731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a4c100b00 msgr2=0x7f6a4c19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:23.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.690+0000 7f6a4a000700 1 --2- 172.21.15.31:0/1458886731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a4c100b00 0x7f6a4c19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:23.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.690+0000 7f6a4a000700 1 -- 172.21.15.31:0/1458886731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a4c101e20 msgr2=0x7f6a4c1a1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:23.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.690+0000 7f6a4a000700 1 --2- 172.21.15.31:0/1458886731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a4c101e20 0x7f6a4c1a1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:23.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.690+0000 7f6a4a000700 1 -- 172.21.15.31:0/1458886731 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4c1a22c0 con 0x7f6a4c101460 2024-04-25T01:15:23.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.691+0000 7f6a4a000700 1 --2- 172.21.15.31:0/1458886731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a4c101460 0x7f6a4c19aba0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800e9f0 tx=0x7f6a3800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:23.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.691+0000 7f6a50a00700 1 -- 172.21.15.31:0/1458886731 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3800c9c0 con 0x7f6a4c101460 2024-04-25T01:15:23.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.691+0000 7f6a50a00700 1 -- 172.21.15.31:0/1458886731 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a38010430 con 0x7f6a4c101460 2024-04-25T01:15:23.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.691+0000 7f6a51e00700 1 -- 172.21.15.31:0/1458886731 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a4c1a25b0 con 0x7f6a4c101460 2024-04-25T01:15:23.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.691+0000 7f6a50a00700 1 -- 172.21.15.31:0/1458886731 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38018680 con 0x7f6a4c101460 2024-04-25T01:15:23.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.691+0000 7f6a51e00700 1 -- 172.21.15.31:0/1458886731 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a4c1a2b40 con 0x7f6a4c101460 2024-04-25T01:15:23.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.692+0000 7f6a51e00700 1 -- 172.21.15.31:0/1458886731 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a4c068f20 con 0x7f6a4c101460 2024-04-25T01:15:23.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.694+0000 7f6a50a00700 1 -- 172.21.15.31:0/1458886731 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6a380105e0 con 0x7f6a4c101460 2024-04-25T01:15:23.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.696+0000 7f6a50a00700 1 --2- 172.21.15.31:0/1458886731 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6a300a2ca0 0x7f6a300a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:23.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.696+0000 7f6a50a00700 1 -- 172.21.15.31:0/1458886731 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f6a380cf480 con 0x7f6a4c101460 2024-04-25T01:15:23.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.696+0000 7f6a4aa00700 1 --2- 172.21.15.31:0/1458886731 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6a300a2ca0 0x7f6a300a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:23.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.697+0000 7f6a4aa00700 1 --2- 172.21.15.31:0/1458886731 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6a300a2ca0 0x7f6a300a5160 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c009d60 tx=0x7f6a3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:23.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.698+0000 7f6a50a00700 1 -- 172.21.15.31:0/1458886731 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6a3809c3d0 con 0x7f6a4c101460 2024-04-25T01:15:23.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.938+0000 7f6a51e00700 1 -- 172.21.15.31:0/1458886731 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f6a4c062730 con 0x7f6a300a2ca0 2024-04-25T01:15:23.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.965+0000 7f6a50a00700 1 -- 172.21.15.31:0/1458886731 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7f6a4c062730 con 0x7f6a300a2ca0 2024-04-25T01:15:23.952 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:15:23.952 INFO:teuthology.orchestra.run.smithi031.stdout:mds.a smithi031 running (2m) 24s ago 2m 12.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef 296222b64e93 2024-04-25T01:15:23.952 INFO:teuthology.orchestra.run.smithi031.stdout:mds.b smithi040 running (2m) 51s ago 119s 25.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef 2d7279fa8895 2024-04-25T01:15:23.952 INFO:teuthology.orchestra.run.smithi031.stdout:mds.c smithi169 running (2m) 51s ago 2m 16.2M - 19.0.0-2445-gd2c45d97 f68e37b807ef a80419ae6cb9 2024-04-25T01:15:23.952 INFO:teuthology.orchestra.run.smithi031.stdout:mds.d smithi031 running (2m) 24s ago 2m 12.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 6d1e33ce69ac 2024-04-25T01:15:23.953 INFO:teuthology.orchestra.run.smithi031.stdout:mds.e smithi040 running (2m) 51s ago 2m 17.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 69a334d0808f 2024-04-25T01:15:23.953 INFO:teuthology.orchestra.run.smithi031.stdout:mds.f smithi169 running (2m) 51s ago 2m 19.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef b6396cd0d501 2024-04-25T01:15:23.953 INFO:teuthology.orchestra.run.smithi031.stdout:mds.g smithi031 running (2m) 24s ago 2m 12.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef 1ac2c36f4ed6 2024-04-25T01:15:23.953 INFO:teuthology.orchestra.run.smithi031.stdout:mds.h smithi040 running (2m) 51s ago 2m 17.1M - 19.0.0-2445-gd2c45d97 f68e37b807ef 99bde325bbc0 2024-04-25T01:15:23.953 INFO:teuthology.orchestra.run.smithi031.stdout:mds.i smithi169 running (2m) 51s ago 2m 17.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d04b8c945324 2024-04-25T01:15:23.953 INFO:teuthology.orchestra.run.smithi031.stdout:mds.j smithi031 running (2m) 24s ago 2m 12.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 8ed92e04f583 2024-04-25T01:15:23.953 INFO:teuthology.orchestra.run.smithi031.stdout:mds.k smithi040 running (2m) 51s ago 2m 13.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef b6248292272e 2024-04-25T01:15:23.953 INFO:teuthology.orchestra.run.smithi031.stdout:mds.l smithi169 running (2m) 51s ago 2m 16.2M - 19.0.0-2445-gd2c45d97 f68e37b807ef 5ac256c12852 2024-04-25T01:15:23.953 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.x smithi031 *:9283,8765 running (10m) 24s ago 10m 515M - 19.0.0-2445-gd2c45d97 f68e37b807ef f54adae2952f 2024-04-25T01:15:23.953 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.y smithi040 *:8443,8765 running (8m) 51s ago 8m 454M - 19.0.0-2445-gd2c45d97 f68e37b807ef e8346f709fab 2024-04-25T01:15:23.953 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.z smithi169 *:8443,8765 running (8m) 51s ago 8m 453M - 19.0.0-2445-gd2c45d97 f68e37b807ef 4a052505ea91 2024-04-25T01:15:23.953 INFO:teuthology.orchestra.run.smithi031.stdout:mon.a smithi031 running (10m) 24s ago 11m 49.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef b4a98098cf56 2024-04-25T01:15:23.953 INFO:teuthology.orchestra.run.smithi031.stdout:mon.b smithi040 running (8m) 51s ago 8m 40.8M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef d9a4e55c713c 2024-04-25T01:15:23.953 INFO:teuthology.orchestra.run.smithi031.stdout:mon.c smithi169 running (8m) 51s ago 8m 40.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 1dac5730d761 2024-04-25T01:15:23.954 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (8m) 24s ago 8m 46.7M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef 09f1b5c3759a 2024-04-25T01:15:23.954 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi040 running (7m) 51s ago 7m 58.0M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef e071053113f3 2024-04-25T01:15:23.954 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi169 running (7m) 51s ago 7m 52.0M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 0627eecea4e5 2024-04-25T01:15:23.954 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi031 running (6m) 24s ago 6m 53.0M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef 6e15ece744dd 2024-04-25T01:15:23.954 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (6m) 51s ago 6m 45.2M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef a7ab7152ffad 2024-04-25T01:15:23.954 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi169 running (5m) 51s ago 5m 46.0M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 5e1d83861159 2024-04-25T01:15:23.954 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6 smithi031 running (5m) 24s ago 5m 50.5M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef 96a85b35987c 2024-04-25T01:15:23.954 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7 smithi040 running (4m) 51s ago 4m 54.6M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef 6554eb45317f 2024-04-25T01:15:23.954 INFO:teuthology.orchestra.run.smithi031.stdout:osd.8 smithi169 running (4m) 51s ago 4m 46.0M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 423c4ed22be8 2024-04-25T01:15:23.954 INFO:teuthology.orchestra.run.smithi031.stdout:osd.9 smithi031 running (3m) 24s ago 3m 43.6M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef ec9931861e58 2024-04-25T01:15:23.954 INFO:teuthology.orchestra.run.smithi031.stdout:osd.10 smithi040 running (3m) 51s ago 3m 56.9M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef 6aa70633137a 2024-04-25T01:15:23.954 INFO:teuthology.orchestra.run.smithi031.stdout:osd.11 smithi169 running (3m) 51s ago 3m 52.7M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 1dbc2f0f360c 2024-04-25T01:15:23.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.968+0000 7f6a51e00700 1 -- 172.21.15.31:0/1458886731 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6a300a2ca0 msgr2=0x7f6a300a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:23.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.968+0000 7f6a51e00700 1 --2- 172.21.15.31:0/1458886731 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6a300a2ca0 0x7f6a300a5160 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c009d60 tx=0x7f6a3c009260 comp rx=0 tx=0).stop 2024-04-25T01:15:23.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.968+0000 7f6a51e00700 1 -- 172.21.15.31:0/1458886731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a4c101460 msgr2=0x7f6a4c19aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:23.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.968+0000 7f6a51e00700 1 --2- 172.21.15.31:0/1458886731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a4c101460 0x7f6a4c19aba0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800e9f0 tx=0x7f6a3800ed00 comp rx=0 tx=0).stop 2024-04-25T01:15:23.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.969+0000 7f6a51e00700 1 -- 172.21.15.31:0/1458886731 shutdown_connections 2024-04-25T01:15:23.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.969+0000 7f6a51e00700 1 --2- 172.21.15.31:0/1458886731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a4c101e20 0x7f6a4c1a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:23.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.969+0000 7f6a51e00700 1 --2- 172.21.15.31:0/1458886731 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6a300a2ca0 0x7f6a300a5160 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:23.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.969+0000 7f6a51e00700 1 --2- 172.21.15.31:0/1458886731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a4c100b00 0x7f6a4c19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:23.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.969+0000 7f6a51e00700 1 --2- 172.21.15.31:0/1458886731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a4c101460 0x7f6a4c19aba0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:23.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.969+0000 7f6a51e00700 1 -- 172.21.15.31:0/1458886731 >> 172.21.15.31:0/1458886731 conn(0x7f6a4c0fc720 msgr2=0x7f6a4c0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:23.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.969+0000 7f6a51e00700 1 -- 172.21.15.31:0/1458886731 shutdown_connections 2024-04-25T01:15:23.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:23.969+0000 7f6a51e00700 1 -- 172.21.15.31:0/1458886731 wait complete. 2024-04-25T01:15:24.443 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:24 smithi169 ceph-mon[32759]: from='client.25069 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:15:24.446 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:24 smithi040 ceph-mon[32518]: from='client.25069 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:15:24.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:24 smithi031 ceph-mon[28053]: from='client.25069 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:15:24.759 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T01:15:25.232 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:15:25.443 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:25 smithi169 ceph-mon[32759]: pgmap v375: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:25.446 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:25 smithi040 ceph-mon[32518]: pgmap v375: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:25.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:25 smithi031 ceph-mon[28053]: pgmap v375: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:27.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:27 smithi031 ceph-mon[28053]: pgmap v376: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T01:15:27.444 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:27 smithi169 ceph-mon[32759]: pgmap v376: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T01:15:27.446 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:27 smithi040 ceph-mon[32518]: pgmap v376: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T01:15:27.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.519+0000 7f1f7ac00700 1 -- 172.21.15.31:0/2095378752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f74106070 msgr2=0x7f1f7410c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:27.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.519+0000 7f1f7ac00700 1 --2- 172.21.15.31:0/2095378752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f74106070 0x7f1f7410c920 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c00b0d0 tx=0x7f1f6c00b3e0 comp rx=0 tx=0).stop 2024-04-25T01:15:27.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.520+0000 7f1f7ac00700 1 -- 172.21.15.31:0/2095378752 shutdown_connections 2024-04-25T01:15:27.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.520+0000 7f1f7ac00700 1 --2- 172.21.15.31:0/2095378752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f74106070 0x7f1f7410c920 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:27.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.520+0000 7f1f7ac00700 1 --2- 172.21.15.31:0/2095378752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f741044d0 0x7f1f741048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:27.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.520+0000 7f1f7ac00700 1 --2- 172.21.15.31:0/2095378752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f741056d0 0x7f1f74105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:27.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.520+0000 7f1f7ac00700 1 -- 172.21.15.31:0/2095378752 >> 172.21.15.31:0/2095378752 conn(0x7f1f740ffa90 msgr2=0x7f1f74101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:27.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.520+0000 7f1f7ac00700 1 -- 172.21.15.31:0/2095378752 shutdown_connections 2024-04-25T01:15:27.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.520+0000 7f1f7ac00700 1 -- 172.21.15.31:0/2095378752 wait complete. 2024-04-25T01:15:27.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.521+0000 7f1f7ac00700 1 Processor -- start 2024-04-25T01:15:27.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.521+0000 7f1f7ac00700 1 -- start start 2024-04-25T01:15:27.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.521+0000 7f1f7ac00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f741044d0 0x7f1f7419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:27.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.521+0000 7f1f7ac00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f741056d0 0x7f1f7419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:27.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.521+0000 7f1f7ac00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f74106070 0x7f1f741a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:27.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.521+0000 7f1f7ac00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1f741100f0 con 0x7f1f741044d0 2024-04-25T01:15:27.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.521+0000 7f1f7ac00700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7410ff70 con 0x7f1f741056d0 2024-04-25T01:15:27.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.521+0000 7f1f7ac00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1f74110270 con 0x7f1f74106070 2024-04-25T01:15:27.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.522+0000 7f1f73e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f741044d0 0x7f1f7419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:27.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.522+0000 7f1f73e00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f741044d0 0x7f1f7419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:32938/0 (socket says 172.21.15.31:32938) 2024-04-25T01:15:27.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.522+0000 7f1f73400700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f741056d0 0x7f1f7419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:27.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.522+0000 7f1f73e00700 1 -- 172.21.15.31:0/3680632536 learned_addr learned my addr 172.21.15.31:0/3680632536 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:27.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.522+0000 7f1f78c00700 1 --2- 172.21.15.31:0/3680632536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f74106070 0x7f1f741a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:27.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.522+0000 7f1f73e00700 1 -- 172.21.15.31:0/3680632536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f74106070 msgr2=0x7f1f741a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:27.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.522+0000 7f1f73e00700 1 --2- 172.21.15.31:0/3680632536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f74106070 0x7f1f741a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:27.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.522+0000 7f1f73e00700 1 -- 172.21.15.31:0/3680632536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f741056d0 msgr2=0x7f1f7419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:27.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.522+0000 7f1f73e00700 1 --2- 172.21.15.31:0/3680632536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f741056d0 0x7f1f7419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:27.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.522+0000 7f1f73e00700 1 -- 172.21.15.31:0/3680632536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f741a22a0 con 0x7f1f741044d0 2024-04-25T01:15:27.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.522+0000 7f1f73400700 1 --2- 172.21.15.31:0/3680632536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f741056d0 0x7f1f7419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:27.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.523+0000 7f1f73e00700 1 --2- 172.21.15.31:0/3680632536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f741044d0 0x7f1f7419a640 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400e9a0 tx=0x7f1f6400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:27.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.523+0000 7f1f70c00700 1 -- 172.21.15.31:0/3680632536 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6400cbd0 con 0x7f1f741044d0 2024-04-25T01:15:27.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.523+0000 7f1f7ac00700 1 -- 172.21.15.31:0/3680632536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f741a2590 con 0x7f1f741044d0 2024-04-25T01:15:27.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.523+0000 7f1f7ac00700 1 -- 172.21.15.31:0/3680632536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f741a2b20 con 0x7f1f741044d0 2024-04-25T01:15:27.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.523+0000 7f1f70c00700 1 -- 172.21.15.31:0/3680632536 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f64004500 con 0x7f1f741044d0 2024-04-25T01:15:27.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.523+0000 7f1f70c00700 1 -- 172.21.15.31:0/3680632536 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f64003e00 con 0x7f1f741044d0 2024-04-25T01:15:27.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.524+0000 7f1f7ac00700 1 -- 172.21.15.31:0/3680632536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f74068f20 con 0x7f1f741044d0 2024-04-25T01:15:27.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.526+0000 7f1f70c00700 1 -- 172.21.15.31:0/3680632536 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1f64003fa0 con 0x7f1f741044d0 2024-04-25T01:15:27.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.527+0000 7f1f70c00700 1 --2- 172.21.15.31:0/3680632536 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1f580a2ca0 0x7f1f580a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:27.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.528+0000 7f1f70c00700 1 -- 172.21.15.31:0/3680632536 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f1f640cf2d0 con 0x7f1f741044d0 2024-04-25T01:15:27.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.528+0000 7f1f73400700 1 --2- 172.21.15.31:0/3680632536 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1f580a2ca0 0x7f1f580a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:27.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.528+0000 7f1f73400700 1 --2- 172.21.15.31:0/3680632536 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1f580a2ca0 0x7f1f580a5160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1f60011490 tx=0x7f1f60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:27.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.532+0000 7f1f70c00700 1 -- 172.21.15.31:0/3680632536 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1f6409c220 con 0x7f1f741044d0 2024-04-25T01:15:27.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.769+0000 7f1f7ac00700 1 -- 172.21.15.31:0/3680632536 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f1f74062670 con 0x7f1f580a2ca0 2024-04-25T01:15:27.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.777+0000 7f1f70c00700 1 -- 172.21.15.31:0/3680632536 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f1f74062670 con 0x7f1f580a2ca0 2024-04-25T01:15:27.769 INFO:teuthology.orchestra.run.smithi031.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T01:15:27.769 INFO:teuthology.orchestra.run.smithi031.stdout:mds.all 12/12 55s ago 2m smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi040=b;smithi040=e;smithi040=h;smithi040=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l;count:12 2024-04-25T01:15:27.769 INFO:teuthology.orchestra.run.smithi031.stdout:mgr 3/3 55s ago 8m smithi031=x;smithi040=y;smithi169=z;count:3 2024-04-25T01:15:27.769 INFO:teuthology.orchestra.run.smithi031.stdout:mon 3/3 55s ago 9m smithi031:172.21.15.31=a;smithi040:172.21.15.40=b;smithi169:172.21.15.169=c;count:3 2024-04-25T01:15:27.769 INFO:teuthology.orchestra.run.smithi031.stdout:osd 12 55s ago - 2024-04-25T01:15:27.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.781+0000 7f1f7ac00700 1 -- 172.21.15.31:0/3680632536 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1f580a2ca0 msgr2=0x7f1f580a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:27.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.781+0000 7f1f7ac00700 1 --2- 172.21.15.31:0/3680632536 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1f580a2ca0 0x7f1f580a5160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1f60011490 tx=0x7f1f60009260 comp rx=0 tx=0).stop 2024-04-25T01:15:27.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.781+0000 7f1f7ac00700 1 -- 172.21.15.31:0/3680632536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f741044d0 msgr2=0x7f1f7419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:27.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.781+0000 7f1f7ac00700 1 --2- 172.21.15.31:0/3680632536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f741044d0 0x7f1f7419a640 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400e9a0 tx=0x7f1f6400ecb0 comp rx=0 tx=0).stop 2024-04-25T01:15:27.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.782+0000 7f1f7ac00700 1 -- 172.21.15.31:0/3680632536 shutdown_connections 2024-04-25T01:15:27.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.782+0000 7f1f7ac00700 1 --2- 172.21.15.31:0/3680632536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f741044d0 0x7f1f7419a640 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:27.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.782+0000 7f1f7ac00700 1 --2- 172.21.15.31:0/3680632536 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1f580a2ca0 0x7f1f580a5160 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:27.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.782+0000 7f1f7ac00700 1 --2- 172.21.15.31:0/3680632536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f74106070 0x7f1f741a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:27.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.782+0000 7f1f7ac00700 1 --2- 172.21.15.31:0/3680632536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f741056d0 0x7f1f7419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:27.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.782+0000 7f1f7ac00700 1 -- 172.21.15.31:0/3680632536 >> 172.21.15.31:0/3680632536 conn(0x7f1f740ffa90 msgr2=0x7f1f74108900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:27.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.782+0000 7f1f7ac00700 1 -- 172.21.15.31:0/3680632536 shutdown_connections 2024-04-25T01:15:27.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:27.782+0000 7f1f7ac00700 1 -- 172.21.15.31:0/3680632536 wait complete. 2024-04-25T01:15:28.447 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:28 smithi040 ceph-mon[32518]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:15:28.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:28 smithi031 ceph-mon[28053]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:15:28.638 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T01:15:28.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:28 smithi169 ceph-mon[32759]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:15:29.094 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:15:29.455 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:29 smithi031 ceph-mon[28053]: pgmap v377: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:29.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:29 smithi169 ceph-mon[32759]: pgmap v377: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:29.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:29 smithi040 ceph-mon[32518]: pgmap v377: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:31.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.258+0000 7faa5c000700 1 -- 172.21.15.31:0/2331879387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa5410a470 msgr2=0x7faa54110d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:31.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.258+0000 7faa5c000700 1 --2- 172.21.15.31:0/2331879387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa5410a470 0x7faa54110d20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00b0d0 tx=0x7faa4c00b3e0 comp rx=0 tx=0).stop 2024-04-25T01:15:31.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.259+0000 7faa5c000700 1 -- 172.21.15.31:0/2331879387 shutdown_connections 2024-04-25T01:15:31.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.259+0000 7faa5c000700 1 --2- 172.21.15.31:0/2331879387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa5410a470 0x7faa54110d20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:31.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.259+0000 7faa5c000700 1 --2- 172.21.15.31:0/2331879387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa541088d0 0x7faa54108cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:31.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.259+0000 7faa5c000700 1 --2- 172.21.15.31:0/2331879387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa54109ad0 0x7faa54109f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:31.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.259+0000 7faa5c000700 1 -- 172.21.15.31:0/2331879387 >> 172.21.15.31:0/2331879387 conn(0x7faa540778a0 msgr2=0x7faa54079ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:31.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.259+0000 7faa5c000700 1 -- 172.21.15.31:0/2331879387 shutdown_connections 2024-04-25T01:15:31.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.259+0000 7faa5c000700 1 -- 172.21.15.31:0/2331879387 wait complete. 2024-04-25T01:15:31.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.260+0000 7faa5c000700 1 Processor -- start 2024-04-25T01:15:31.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.260+0000 7faa5c000700 1 -- start start 2024-04-25T01:15:31.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.261+0000 7faa5c000700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa541088d0 0x7faa5419ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:31.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.261+0000 7faa5c000700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa54109ad0 0x7faa5419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:31.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.261+0000 7faa5c000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa5410a470 0x7faa541a5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:31.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.261+0000 7faa5c000700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faa54114510 con 0x7faa54109ad0 2024-04-25T01:15:31.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.261+0000 7faa5c000700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faa54114390 con 0x7faa541088d0 2024-04-25T01:15:31.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.261+0000 7faa5c000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faa54114690 con 0x7faa5410a470 2024-04-25T01:15:31.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.261+0000 7faa58c00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa54109ad0 0x7faa5419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:31.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.261+0000 7faa59600700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa541088d0 0x7faa5419ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:31.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.261+0000 7faa59600700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa541088d0 0x7faa5419ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:40312/0 (socket says 172.21.15.31:40312) 2024-04-25T01:15:31.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.261+0000 7faa59600700 1 -- 172.21.15.31:0/2100321383 learned_addr learned my addr 172.21.15.31:0/2100321383 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:31.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.261+0000 7faa5a000700 1 --2- 172.21.15.31:0/2100321383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa5410a470 0x7faa541a5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:31.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.262+0000 7faa59600700 1 -- 172.21.15.31:0/2100321383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa5410a470 msgr2=0x7faa541a5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:31.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.262+0000 7faa59600700 1 --2- 172.21.15.31:0/2100321383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa5410a470 0x7faa541a5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:31.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.262+0000 7faa59600700 1 -- 172.21.15.31:0/2100321383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa54109ad0 msgr2=0x7faa5419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:31.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.262+0000 7faa59600700 1 --2- 172.21.15.31:0/2100321383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa54109ad0 0x7faa5419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:31.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.262+0000 7faa59600700 1 -- 172.21.15.31:0/2100321383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa541a6690 con 0x7faa541088d0 2024-04-25T01:15:31.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.262+0000 7faa59600700 1 --2- 172.21.15.31:0/2100321383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa541088d0 0x7faa5419ea30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7faa4800e970 tx=0x7faa4800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:31.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.262+0000 7faa52000700 1 -- 172.21.15.31:0/2100321383 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48004220 con 0x7faa541088d0 2024-04-25T01:15:31.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.262+0000 7faa52000700 1 -- 172.21.15.31:0/2100321383 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa480043c0 con 0x7faa541088d0 2024-04-25T01:15:31.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.262+0000 7faa52000700 1 -- 172.21.15.31:0/2100321383 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48003df0 con 0x7faa541088d0 2024-04-25T01:15:31.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.263+0000 7faa5c000700 1 -- 172.21.15.31:0/2100321383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa541a6980 con 0x7faa541088d0 2024-04-25T01:15:31.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.263+0000 7faa5c000700 1 -- 172.21.15.31:0/2100321383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa541a6e70 con 0x7faa541088d0 2024-04-25T01:15:31.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.264+0000 7faa5c000700 1 -- 172.21.15.31:0/2100321383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa54068f20 con 0x7faa541088d0 2024-04-25T01:15:31.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.266+0000 7faa52000700 1 -- 172.21.15.31:0/2100321383 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faa48043ae0 con 0x7faa541088d0 2024-04-25T01:15:31.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.267+0000 7faa52000700 1 --2- 172.21.15.31:0/2100321383 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7faa3c0a2ca0 0x7faa3c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:31.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.268+0000 7faa52000700 1 -- 172.21.15.31:0/2100321383 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7faa480cf110 con 0x7faa541088d0 2024-04-25T01:15:31.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.268+0000 7faa58c00700 1 --2- 172.21.15.31:0/2100321383 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7faa3c0a2ca0 0x7faa3c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:31.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.268+0000 7faa58c00700 1 --2- 172.21.15.31:0/2100321383 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7faa3c0a2ca0 0x7faa3c0a5160 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faa40009c60 tx=0x7faa40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:31.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.270+0000 7faa52000700 1 -- 172.21.15.31:0/2100321383 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7faa4809c060 con 0x7faa541088d0 2024-04-25T01:15:31.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.514+0000 7faa5c000700 1 -- 172.21.15.31:0/2100321383 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7faa54062670 con 0x7faa3c0a2ca0 2024-04-25T01:15:31.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:31 smithi031 ceph-mon[28053]: pgmap v378: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:31.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.517+0000 7faa52000700 1 -- 172.21.15.31:0/2100321383 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7faa54062670 con 0x7faa3c0a2ca0 2024-04-25T01:15:31.503 INFO:teuthology.orchestra.run.smithi031.stdout:HOST ADDR LABELS STATUS 2024-04-25T01:15:31.503 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031 172.21.15.31 2024-04-25T01:15:31.503 INFO:teuthology.orchestra.run.smithi031.stdout:smithi040 172.21.15.40 2024-04-25T01:15:31.503 INFO:teuthology.orchestra.run.smithi031.stdout:smithi169 172.21.15.169 2024-04-25T01:15:31.503 INFO:teuthology.orchestra.run.smithi031.stdout:3 hosts in cluster 2024-04-25T01:15:31.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.521+0000 7faa5c000700 1 -- 172.21.15.31:0/2100321383 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7faa3c0a2ca0 msgr2=0x7faa3c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:31.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.521+0000 7faa5c000700 1 --2- 172.21.15.31:0/2100321383 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7faa3c0a2ca0 0x7faa3c0a5160 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faa40009c60 tx=0x7faa40009260 comp rx=0 tx=0).stop 2024-04-25T01:15:31.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.521+0000 7faa5c000700 1 -- 172.21.15.31:0/2100321383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa541088d0 msgr2=0x7faa5419ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:31.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.521+0000 7faa5c000700 1 --2- 172.21.15.31:0/2100321383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa541088d0 0x7faa5419ea30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7faa4800e970 tx=0x7faa4800ec80 comp rx=0 tx=0).stop 2024-04-25T01:15:31.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.521+0000 7faa5c000700 1 -- 172.21.15.31:0/2100321383 shutdown_connections 2024-04-25T01:15:31.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.521+0000 7faa5c000700 1 --2- 172.21.15.31:0/2100321383 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7faa3c0a2ca0 0x7faa3c0a5160 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:31.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.521+0000 7faa5c000700 1 --2- 172.21.15.31:0/2100321383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa5410a470 0x7faa541a5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:31.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.521+0000 7faa5c000700 1 --2- 172.21.15.31:0/2100321383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa541088d0 0x7faa5419ea30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:31.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.521+0000 7faa5c000700 1 --2- 172.21.15.31:0/2100321383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa54109ad0 0x7faa5419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:31.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.521+0000 7faa5c000700 1 -- 172.21.15.31:0/2100321383 >> 172.21.15.31:0/2100321383 conn(0x7faa540778a0 msgr2=0x7faa5410cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:31.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.521+0000 7faa5c000700 1 -- 172.21.15.31:0/2100321383 shutdown_connections 2024-04-25T01:15:31.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:31.521+0000 7faa5c000700 1 -- 172.21.15.31:0/2100321383 wait complete. 2024-04-25T01:15:31.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:31 smithi169 ceph-mon[32759]: pgmap v378: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:31.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:31 smithi040 ceph-mon[32518]: pgmap v378: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:32.427 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T01:15:32.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:32 smithi031 ceph-mon[28053]: from='client.25073 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:15:32.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:32 smithi169 ceph-mon[32759]: from='client.25073 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:15:32.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:32 smithi040 ceph-mon[32518]: from='client.25073 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:15:32.890 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:15:33.365 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:33 smithi031 ceph-mon[28053]: pgmap v379: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:33.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:33 smithi169 ceph-mon[32759]: pgmap v379: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:33.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:33 smithi040 ceph-mon[32518]: pgmap v379: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:35.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.082+0000 7f973ca00700 1 -- 172.21.15.31:0/64343930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9738102d20 msgr2=0x7f9738105110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:35.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.082+0000 7f973ca00700 1 --2- 172.21.15.31:0/64343930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9738102d20 0x7f9738105110 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9728009990 tx=0x7f9728009ca0 comp rx=0 tx=0).stop 2024-04-25T01:15:35.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.082+0000 7f973ca00700 1 -- 172.21.15.31:0/64343930 shutdown_connections 2024-04-25T01:15:35.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.082+0000 7f973ca00700 1 --2- 172.21.15.31:0/64343930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9738107f80 0x7f973810a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:35.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.082+0000 7f973ca00700 1 --2- 172.21.15.31:0/64343930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9738102d20 0x7f9738105110 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:35.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.082+0000 7f973ca00700 1 --2- 172.21.15.31:0/64343930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9738105650 0x7f9738107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:35.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.082+0000 7f973ca00700 1 -- 172.21.15.31:0/64343930 >> 172.21.15.31:0/64343930 conn(0x7f97380fc730 msgr2=0x7f97380feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:35.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.082+0000 7f973ca00700 1 -- 172.21.15.31:0/64343930 shutdown_connections 2024-04-25T01:15:35.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.082+0000 7f973ca00700 1 -- 172.21.15.31:0/64343930 wait complete. 2024-04-25T01:15:35.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.084+0000 7f973ca00700 1 Processor -- start 2024-04-25T01:15:35.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.084+0000 7f973ca00700 1 -- start start 2024-04-25T01:15:35.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.084+0000 7f973ca00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9738102d20 0x7f973819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:35.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.084+0000 7f973ca00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9738105650 0x7f973819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:35.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.084+0000 7f973ca00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9738107f80 0x7f97381a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:35.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.084+0000 7f973ca00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f973810deb0 con 0x7f9738107f80 2024-04-25T01:15:35.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.084+0000 7f973ca00700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f973810dd30 con 0x7f9738102d20 2024-04-25T01:15:35.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.084+0000 7f973ca00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f973810e030 con 0x7f9738105650 2024-04-25T01:15:35.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.085+0000 7f9735e00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9738102d20 0x7f973819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:35.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.085+0000 7f9735e00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9738102d20 0x7f973819a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:40324/0 (socket says 172.21.15.31:40324) 2024-04-25T01:15:35.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.085+0000 7f9735e00700 1 -- 172.21.15.31:0/340369219 learned_addr learned my addr 172.21.15.31:0/340369219 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:35.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.085+0000 7f9735400700 1 --2- 172.21.15.31:0/340369219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9738105650 0x7f973819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:35.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.085+0000 7f9736800700 1 --2- 172.21.15.31:0/340369219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9738107f80 0x7f97381a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:35.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.085+0000 7f9735e00700 1 -- 172.21.15.31:0/340369219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9738105650 msgr2=0x7f973819acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:35.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.085+0000 7f9735e00700 1 --2- 172.21.15.31:0/340369219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9738105650 0x7f973819acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:35.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.085+0000 7f9735e00700 1 -- 172.21.15.31:0/340369219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9738107f80 msgr2=0x7f97381a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:35.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.085+0000 7f9735e00700 1 --2- 172.21.15.31:0/340369219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9738107f80 0x7f97381a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:35.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.085+0000 7f9735e00700 1 -- 172.21.15.31:0/340369219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97381a23d0 con 0x7f9738102d20 2024-04-25T01:15:35.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.086+0000 7f9735e00700 1 --2- 172.21.15.31:0/340369219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9738102d20 0x7f973819a770 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f972800b2e0 tx=0x7f972800bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:35.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.086+0000 7f972ea00700 1 -- 172.21.15.31:0/340369219 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728026070 con 0x7f9738102d20 2024-04-25T01:15:35.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.086+0000 7f973ca00700 1 -- 172.21.15.31:0/340369219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97381a2660 con 0x7f9738102d20 2024-04-25T01:15:35.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.086+0000 7f972ea00700 1 -- 172.21.15.31:0/340369219 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9728003d70 con 0x7f9738102d20 2024-04-25T01:15:35.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.086+0000 7f972ea00700 1 -- 172.21.15.31:0/340369219 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728021410 con 0x7f9738102d20 2024-04-25T01:15:35.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.086+0000 7f973ca00700 1 -- 172.21.15.31:0/340369219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97381a2b10 con 0x7f9738102d20 2024-04-25T01:15:35.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.087+0000 7f973ca00700 1 -- 172.21.15.31:0/340369219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9738068f20 con 0x7f9738102d20 2024-04-25T01:15:35.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.089+0000 7f972ea00700 1 -- 172.21.15.31:0/340369219 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9728029070 con 0x7f9738102d20 2024-04-25T01:15:35.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.091+0000 7f972ea00700 1 --2- 172.21.15.31:0/340369219 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f971c0a2c20 0x7f971c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:35.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.091+0000 7f972ea00700 1 -- 172.21.15.31:0/340369219 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f97280d03b0 con 0x7f9738102d20 2024-04-25T01:15:35.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.091+0000 7f9735400700 1 --2- 172.21.15.31:0/340369219 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f971c0a2c20 0x7f971c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:35.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.092+0000 7f9735400700 1 --2- 172.21.15.31:0/340369219 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f971c0a2c20 0x7f971c0a50e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9724009c60 tx=0x7f9724009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:35.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.093+0000 7f972ea00700 1 -- 172.21.15.31:0/340369219 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9728030080 con 0x7f9738102d20 2024-04-25T01:15:35.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.332+0000 7f973ca00700 1 -- 172.21.15.31:0/340369219 --> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f9738062670 con 0x7f971c0a2c20 2024-04-25T01:15:35.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.336+0000 7f972ea00700 1 -- 172.21.15.31:0/340369219 <== mgr.14152 v2:172.21.15.31:6800/2874594057 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f9738062670 con 0x7f971c0a2c20 2024-04-25T01:15:35.322 INFO:teuthology.orchestra.run.smithi031.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T01:15:35.322 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53310008400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T01:15:35.322 INFO:teuthology.orchestra.run.smithi031.stdout:smithi040 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330009J400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T01:15:35.322 INFO:teuthology.orchestra.run.smithi031.stdout:smithi169 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300LN400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T01:15:35.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.339+0000 7f973ca00700 1 -- 172.21.15.31:0/340369219 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f971c0a2c20 msgr2=0x7f971c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:35.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.339+0000 7f973ca00700 1 --2- 172.21.15.31:0/340369219 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f971c0a2c20 0x7f971c0a50e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9724009c60 tx=0x7f9724009260 comp rx=0 tx=0).stop 2024-04-25T01:15:35.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.339+0000 7f973ca00700 1 -- 172.21.15.31:0/340369219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9738102d20 msgr2=0x7f973819a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:35.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.339+0000 7f973ca00700 1 --2- 172.21.15.31:0/340369219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9738102d20 0x7f973819a770 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f972800b2e0 tx=0x7f972800bbb0 comp rx=0 tx=0).stop 2024-04-25T01:15:35.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.339+0000 7f973ca00700 1 -- 172.21.15.31:0/340369219 shutdown_connections 2024-04-25T01:15:35.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.339+0000 7f973ca00700 1 --2- 172.21.15.31:0/340369219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9738107f80 0x7f97381a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:35.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.339+0000 7f973ca00700 1 --2- 172.21.15.31:0/340369219 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f971c0a2c20 0x7f971c0a50e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:35.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.339+0000 7f973ca00700 1 --2- 172.21.15.31:0/340369219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9738102d20 0x7f973819a770 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:35.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.339+0000 7f973ca00700 1 --2- 172.21.15.31:0/340369219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9738105650 0x7f973819acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:35.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.339+0000 7f973ca00700 1 -- 172.21.15.31:0/340369219 >> 172.21.15.31:0/340369219 conn(0x7f97380fc730 msgr2=0x7f97381024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:35.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.340+0000 7f973ca00700 1 -- 172.21.15.31:0/340369219 shutdown_connections 2024-04-25T01:15:35.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:35.340+0000 7f973ca00700 1 -- 172.21.15.31:0/340369219 wait complete. 2024-04-25T01:15:35.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:35 smithi031 ceph-mon[28053]: pgmap v380: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:35.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:35 smithi169 ceph-mon[32759]: pgmap v380: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:35.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:35 smithi040 ceph-mon[32518]: pgmap v380: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:36.154 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T01:15:36.166 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T01:15:36.166 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T01:15:36.503 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:36 smithi031 ceph-mon[28053]: from='client.25079 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:15:36.639 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:15:36.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:36 smithi169 ceph-mon[32759]: from='client.25079 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:15:36.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:36 smithi040 ceph-mon[32518]: from='client.25079 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:15:37.354 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:37 smithi031 ceph-mon[28053]: pgmap v381: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T01:15:37.694 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:37 smithi169 ceph-mon[32759]: pgmap v381: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T01:15:37.697 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:37 smithi040 ceph-mon[32518]: pgmap v381: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T01:15:38.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.934+0000 7f1e1fe00700 1 -- 172.21.15.31:0/474569696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e201043d0 msgr2=0x7f1e201047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:38.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.934+0000 7f1e1fe00700 1 --2- 172.21.15.31:0/474569696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e201043d0 0x7f1e201047f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1e100099c0 tx=0x7f1e10009cd0 comp rx=0 tx=0).stop 2024-04-25T01:15:38.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.934+0000 7f1e1fe00700 1 -- 172.21.15.31:0/474569696 shutdown_connections 2024-04-25T01:15:38.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.934+0000 7f1e1fe00700 1 --2- 172.21.15.31:0/474569696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e201043d0 0x7f1e201047f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:38.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.934+0000 7f1e1fe00700 1 --2- 172.21.15.31:0/474569696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e20105f70 0x7f1e2010c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:38.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.934+0000 7f1e1fe00700 1 --2- 172.21.15.31:0/474569696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e201055d0 0x7f1e20105a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:38.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.934+0000 7f1e1fe00700 1 -- 172.21.15.31:0/474569696 >> 172.21.15.31:0/474569696 conn(0x7f1e200ff970 msgr2=0x7f1e20101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:38.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.934+0000 7f1e1fe00700 1 -- 172.21.15.31:0/474569696 shutdown_connections 2024-04-25T01:15:38.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.934+0000 7f1e1fe00700 1 -- 172.21.15.31:0/474569696 wait complete. 2024-04-25T01:15:38.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.935+0000 7f1e1fe00700 1 Processor -- start 2024-04-25T01:15:38.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.935+0000 7f1e1fe00700 1 -- start start 2024-04-25T01:15:38.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.936+0000 7f1e1fe00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e201043d0 0x7f1e2019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:38.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.936+0000 7f1e1fe00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e201055d0 0x7f1e2019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:38.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.936+0000 7f1e1fe00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e20105f70 0x7f1e201a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:38.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.936+0000 7f1e1fe00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2010ffe0 con 0x7f1e20105f70 2024-04-25T01:15:38.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.936+0000 7f1e1fe00700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2010fe60 con 0x7f1e201043d0 2024-04-25T01:15:38.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.936+0000 7f1e1fe00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1e20110160 con 0x7f1e201055d0 2024-04-25T01:15:38.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.936+0000 7f1e1ea00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e201043d0 0x7f1e2019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:38.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.936+0000 7f1e1f400700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e20105f70 0x7f1e201a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:38.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.936+0000 7f1e1ea00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e201043d0 0x7f1e2019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:48062/0 (socket says 172.21.15.31:48062) 2024-04-25T01:15:38.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.936+0000 7f1e1ea00700 1 -- 172.21.15.31:0/3685080098 learned_addr learned my addr 172.21.15.31:0/3685080098 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:38.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.936+0000 7f1e16000700 1 --2- 172.21.15.31:0/3685080098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e201055d0 0x7f1e2019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:38.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.936+0000 7f1e1ea00700 1 -- 172.21.15.31:0/3685080098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e201055d0 msgr2=0x7f1e2019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:38.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.936+0000 7f1e1ea00700 1 --2- 172.21.15.31:0/3685080098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e201055d0 0x7f1e2019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:38.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.936+0000 7f1e1ea00700 1 -- 172.21.15.31:0/3685080098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e20105f70 msgr2=0x7f1e201a1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:38.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.937+0000 7f1e1ea00700 1 --2- 172.21.15.31:0/3685080098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e20105f70 0x7f1e201a1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:38.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.937+0000 7f1e1ea00700 1 -- 172.21.15.31:0/3685080098 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e201a2250 con 0x7f1e201043d0 2024-04-25T01:15:38.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.937+0000 7f1e1f400700 1 --2- 172.21.15.31:0/3685080098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e20105f70 0x7f1e201a1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:38.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.937+0000 7f1e1ea00700 1 --2- 172.21.15.31:0/3685080098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e201043d0 0x7f1e2019a5f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000b580 tx=0x7f1e1000bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:38.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.937+0000 7f1e17e00700 1 -- 172.21.15.31:0/3685080098 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10009e60 con 0x7f1e201043d0 2024-04-25T01:15:38.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.937+0000 7f1e17e00700 1 -- 172.21.15.31:0/3685080098 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e10003e30 con 0x7f1e201043d0 2024-04-25T01:15:38.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.937+0000 7f1e1fe00700 1 -- 172.21.15.31:0/3685080098 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e201a24e0 con 0x7f1e201043d0 2024-04-25T01:15:38.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.937+0000 7f1e17e00700 1 -- 172.21.15.31:0/3685080098 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e100173f0 con 0x7f1e201043d0 2024-04-25T01:15:38.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.937+0000 7f1e1fe00700 1 -- 172.21.15.31:0/3685080098 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e201a2930 con 0x7f1e201043d0 2024-04-25T01:15:38.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.938+0000 7f1e1fe00700 1 -- 172.21.15.31:0/3685080098 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e20068f20 con 0x7f1e201043d0 2024-04-25T01:15:38.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.940+0000 7f1e17e00700 1 -- 172.21.15.31:0/3685080098 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1e1000f680 con 0x7f1e201043d0 2024-04-25T01:15:38.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.942+0000 7f1e17e00700 1 --2- 172.21.15.31:0/3685080098 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1e040a2c20 0x7f1e040a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:38.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.942+0000 7f1e17e00700 1 -- 172.21.15.31:0/3685080098 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f1e100cfda0 con 0x7f1e201043d0 2024-04-25T01:15:38.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.942+0000 7f1e16000700 1 --2- 172.21.15.31:0/3685080098 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1e040a2c20 0x7f1e040a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:38.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.943+0000 7f1e16000700 1 --2- 172.21.15.31:0/3685080098 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1e040a2c20 0x7f1e040a50e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1e08009da0 tx=0x7f1e0800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:38.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:38.946+0000 7f1e17e00700 1 -- 172.21.15.31:0/3685080098 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1e10013070 con 0x7f1e201043d0 2024-04-25T01:15:39.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:39.223+0000 7f1e1fe00700 1 -- 172.21.15.31:0/3685080098 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f1e20062e40 con 0x7f1e201043d0 2024-04-25T01:15:39.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:39.224+0000 7f1e17e00700 1 -- 172.21.15.31:0/3685080098 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) ==== 76+0+3038 (secure 0 0 0) 0x7f1e1009c730 con 0x7f1e201043d0 2024-04-25T01:15:39.214 INFO:teuthology.orchestra.run.smithi031.stdout:e23 2024-04-25T01:15:39.215 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:15:39.215 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T01:15:39.215 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-25T01:15:39.215 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:15:39.215 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:15:39.216 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-25T01:15:39.216 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 23 2024-04-25T01:15:39.216 INFO:teuthology.orchestra.run.smithi031.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T01:15:39.216 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-25T01:13:30.975848+0000 2024-04-25T01:15:39.216 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-25T01:13:36.545846+0000 2024-04-25T01:15:39.216 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-25T01:15:39.216 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-25T01:15:39.217 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 300 2024-04-25T01:15:39.217 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-25T01:15:39.217 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-25T01:15:39.217 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-25T01:15:39.217 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-25T01:15:39.217 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-25T01:15:39.217 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 0 2024-04-25T01:15:39.217 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T01:15:39.217 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 3 2024-04-25T01:15:39.218 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1,2 2024-04-25T01:15:39.218 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=24596,1=24565,2=24532} 2024-04-25T01:15:39.218 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-25T01:15:39.218 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-25T01:15:39.218 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-25T01:15:39.218 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-25T01:15:39.218 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-25T01:15:39.218 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-04-25T01:15:39.218 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-25T01:15:39.218 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-25T01:15:39.219 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 0 2024-04-25T01:15:39.219 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 24596 members: 24532,24565,24596 2024-04-25T01:15:39.219 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.b{0:24596} state up:active seq 3 addr [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:15:39.219 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.h{0:24548} state up:standby-replay seq 1 addr [v2:172.21.15.40:6836/590109562,v1:172.21.15.40:6837/590109562] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:15:39.219 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.i{1:24565} state up:active seq 7 addr [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:15:39.219 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.c{1:24544} state up:standby-replay seq 1 addr [v2:172.21.15.169:6836/2883308641,v1:172.21.15.169:6837/2883308641] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:15:39.219 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.f{2:24532} state up:active seq 9 addr [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:15:39.219 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.e{2:24518} state up:standby-replay seq 1 addr [v2:172.21.15.40:6834/2916305801,v1:172.21.15.40:6835/2916305801] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:15:39.220 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:15:39.220 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:15:39.220 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-25T01:15:39.220 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:15:39.220 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.j{-1:14391} state up:standby seq 1 addr [v2:172.21.15.31:6834/498218714,v1:172.21.15.31:6835/498218714] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:15:39.220 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.g{-1:14457} state up:standby seq 1 addr [v2:172.21.15.31:6838/2647199475,v1:172.21.15.31:6839/2647199475] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:15:39.220 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.a{-1:14469} state up:standby seq 1 addr [v2:172.21.15.31:6840/30518311,v1:172.21.15.31:6841/30518311] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:15:39.221 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.l{-1:24469} state up:standby seq 1 addr [v2:172.21.15.169:6832/420362823,v1:172.21.15.169:6833/420362823] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:15:39.221 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.k{-1:24473} state up:standby seq 1 addr [v2:172.21.15.40:6832/1943710438,v1:172.21.15.40:6833/1943710438] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:15:39.221 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.d{-1:24506} state up:standby seq 1 addr [v2:172.21.15.31:6836/4191344115,v1:172.21.15.31:6837/4191344115] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:15:39.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:39.228+0000 7f1e1fe00700 1 -- 172.21.15.31:0/3685080098 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1e040a2c20 msgr2=0x7f1e040a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:39.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:39.228+0000 7f1e1fe00700 1 --2- 172.21.15.31:0/3685080098 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1e040a2c20 0x7f1e040a50e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1e08009da0 tx=0x7f1e0800b040 comp rx=0 tx=0).stop 2024-04-25T01:15:39.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:39.228+0000 7f1e1fe00700 1 -- 172.21.15.31:0/3685080098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e201043d0 msgr2=0x7f1e2019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:39.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:39.228+0000 7f1e1fe00700 1 --2- 172.21.15.31:0/3685080098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e201043d0 0x7f1e2019a5f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000b580 tx=0x7f1e1000bee0 comp rx=0 tx=0).stop 2024-04-25T01:15:39.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:39.229+0000 7f1e1fe00700 1 -- 172.21.15.31:0/3685080098 shutdown_connections 2024-04-25T01:15:39.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:39.229+0000 7f1e1fe00700 1 --2- 172.21.15.31:0/3685080098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e20105f70 0x7f1e201a1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:39.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:39.229+0000 7f1e1fe00700 1 --2- 172.21.15.31:0/3685080098 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1e040a2c20 0x7f1e040a50e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:39.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:39.229+0000 7f1e1fe00700 1 --2- 172.21.15.31:0/3685080098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e201043d0 0x7f1e2019a5f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:39.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:39.229+0000 7f1e1fe00700 1 --2- 172.21.15.31:0/3685080098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e201055d0 0x7f1e2019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:39.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:39.229+0000 7f1e1fe00700 1 -- 172.21.15.31:0/3685080098 >> 172.21.15.31:0/3685080098 conn(0x7f1e200ff970 msgr2=0x7f1e201089a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:39.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:39.229+0000 7f1e1fe00700 1 -- 172.21.15.31:0/3685080098 shutdown_connections 2024-04-25T01:15:39.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:39.229+0000 7f1e1fe00700 1 -- 172.21.15.31:0/3685080098 wait complete. 2024-04-25T01:15:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 23 2024-04-25T01:15:39.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:39 smithi031 ceph-mon[28053]: pgmap v382: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:39.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:39 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3685080098' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:15:39.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:39 smithi169 ceph-mon[32759]: pgmap v382: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:39.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:39 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3685080098' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:15:39.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:39 smithi040 ceph-mon[32518]: pgmap v382: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:39.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:39 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3685080098' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:15:40.015 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T01:15:40.504 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:15:41.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:41 smithi031 ceph-mon[28053]: pgmap v383: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:41.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:41 smithi169 ceph-mon[32759]: pgmap v383: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:41.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:41 smithi040 ceph-mon[32518]: pgmap v383: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:42.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.820+0000 7f40bf800700 1 -- 172.21.15.31:0/3406492376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b81056d0 msgr2=0x7f40b8105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:42.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.820+0000 7f40bf800700 1 --2- 172.21.15.31:0/3406492376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b81056d0 0x7f40b8105b30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f40a40099c0 tx=0x7f40a4009cd0 comp rx=0 tx=0).stop 2024-04-25T01:15:42.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.820+0000 7f40bf800700 1 -- 172.21.15.31:0/3406492376 shutdown_connections 2024-04-25T01:15:42.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.820+0000 7f40bf800700 1 --2- 172.21.15.31:0/3406492376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40b8106070 0x7f40b810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:42.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.820+0000 7f40bf800700 1 --2- 172.21.15.31:0/3406492376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40b81044d0 0x7f40b81048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:42.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.820+0000 7f40bf800700 1 --2- 172.21.15.31:0/3406492376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b81056d0 0x7f40b8105b30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:42.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.820+0000 7f40bf800700 1 -- 172.21.15.31:0/3406492376 >> 172.21.15.31:0/3406492376 conn(0x7f40b80ffa90 msgr2=0x7f40b8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:42.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.820+0000 7f40bf800700 1 -- 172.21.15.31:0/3406492376 shutdown_connections 2024-04-25T01:15:42.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.820+0000 7f40bf800700 1 -- 172.21.15.31:0/3406492376 wait complete. 2024-04-25T01:15:42.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.821+0000 7f40bf800700 1 Processor -- start 2024-04-25T01:15:42.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.821+0000 7f40bf800700 1 -- start start 2024-04-25T01:15:42.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.822+0000 7f40bf800700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b81044d0 0x7f40b819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:42.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.822+0000 7f40bf800700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40b81056d0 0x7f40b819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:42.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.822+0000 7f40bf800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40b8106070 0x7f40b81a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:42.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.822+0000 7f40bf800700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f40b8110110 con 0x7f40b81044d0 2024-04-25T01:15:42.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.822+0000 7f40bf800700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f40b810ff90 con 0x7f40b81056d0 2024-04-25T01:15:42.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.822+0000 7f40bf800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f40b8110290 con 0x7f40b8106070 2024-04-25T01:15:42.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.822+0000 7f40bce00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b81044d0 0x7f40b819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:42.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.822+0000 7f40bce00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b81044d0 0x7f40b819a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57994/0 (socket says 172.21.15.31:57994) 2024-04-25T01:15:42.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.822+0000 7f40bce00700 1 -- 172.21.15.31:0/1742025068 learned_addr learned my addr 172.21.15.31:0/1742025068 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:42.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.822+0000 7f40bd800700 1 --2- 172.21.15.31:0/1742025068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40b8106070 0x7f40b81a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:42.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.822+0000 7f40afe00700 1 --2- 172.21.15.31:0/1742025068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40b81056d0 0x7f40b819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:42.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.823+0000 7f40bce00700 1 -- 172.21.15.31:0/1742025068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40b8106070 msgr2=0x7f40b81a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:42.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.823+0000 7f40bce00700 1 --2- 172.21.15.31:0/1742025068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40b8106070 0x7f40b81a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:42.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.823+0000 7f40bce00700 1 -- 172.21.15.31:0/1742025068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40b81056d0 msgr2=0x7f40b819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:42.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.823+0000 7f40bce00700 1 --2- 172.21.15.31:0/1742025068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40b81056d0 0x7f40b819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:42.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.823+0000 7f40bce00700 1 -- 172.21.15.31:0/1742025068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b81a2300 con 0x7f40b81044d0 2024-04-25T01:15:42.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.823+0000 7f40bce00700 1 --2- 172.21.15.31:0/1742025068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b81044d0 0x7f40b819a6a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f409c00e9a0 tx=0x7f409c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:42.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.823+0000 7f40b6000700 1 -- 172.21.15.31:0/1742025068 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c004220 con 0x7f40b81044d0 2024-04-25T01:15:42.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.823+0000 7f40b6000700 1 -- 172.21.15.31:0/1742025068 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f409c0043c0 con 0x7f40b81044d0 2024-04-25T01:15:42.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.823+0000 7f40bf800700 1 -- 172.21.15.31:0/1742025068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b81a25f0 con 0x7f40b81044d0 2024-04-25T01:15:42.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.823+0000 7f40b6000700 1 -- 172.21.15.31:0/1742025068 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c003df0 con 0x7f40b81044d0 2024-04-25T01:15:42.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.824+0000 7f40bf800700 1 -- 172.21.15.31:0/1742025068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b81a2b80 con 0x7f40b81044d0 2024-04-25T01:15:42.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.825+0000 7f40bf800700 1 -- 172.21.15.31:0/1742025068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40b8068f20 con 0x7f40b81044d0 2024-04-25T01:15:42.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.827+0000 7f40b6000700 1 -- 172.21.15.31:0/1742025068 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f409c00c810 con 0x7f40b81044d0 2024-04-25T01:15:42.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.828+0000 7f40b6000700 1 --2- 172.21.15.31:0/1742025068 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f40940a2ca0 0x7f40940a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:42.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.828+0000 7f40b6000700 1 -- 172.21.15.31:0/1742025068 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f409c0cf1a0 con 0x7f40b81044d0 2024-04-25T01:15:42.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.829+0000 7f40afe00700 1 --2- 172.21.15.31:0/1742025068 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f40940a2ca0 0x7f40940a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:42.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.829+0000 7f40afe00700 1 --2- 172.21.15.31:0/1742025068 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f40940a2ca0 0x7f40940a5160 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f40a4000c00 tx=0x7f40a4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:42.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:42.833+0000 7f40b6000700 1 -- 172.21.15.31:0/1742025068 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f409c09c0f0 con 0x7f40b81044d0 2024-04-25T01:15:43.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:43.065+0000 7f40bf800700 1 -- 172.21.15.31:0/1742025068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f40b8062e40 con 0x7f40b81044d0 2024-04-25T01:15:43.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:43.067+0000 7f40b6000700 1 -- 172.21.15.31:0/1742025068 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v89) ==== 56+0+5221 (secure 0 0 0) 0x7f409c09b9f0 con 0x7f40b81044d0 2024-04-25T01:15:43.054 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 89 2024-04-25T01:15:43.054 INFO:teuthology.orchestra.run.smithi031.stdout:fsid 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:15:43.054 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-25T01:04:23.310856+0000 2024-04-25T01:15:43.054 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-25T01:14:33.793638+0000 2024-04-25T01:15:43.054 INFO:teuthology.orchestra.run.smithi031.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T01:15:43.054 INFO:teuthology.orchestra.run.smithi031.stdout:crush_version 26 2024-04-25T01:15:43.054 INFO:teuthology.orchestra.run.smithi031.stdout:full_ratio 0.95 2024-04-25T01:15:43.054 INFO:teuthology.orchestra.run.smithi031.stdout:backfillfull_ratio 0.9 2024-04-25T01:15:43.054 INFO:teuthology.orchestra.run.smithi031.stdout:nearfull_ratio 0.85 2024-04-25T01:15:43.054 INFO:teuthology.orchestra.run.smithi031.stdout:require_min_compat_client luminous 2024-04-25T01:15:43.054 INFO:teuthology.orchestra.run.smithi031.stdout:min_compat_client luminous 2024-04-25T01:15:43.055 INFO:teuthology.orchestra.run.smithi031.stdout:require_osd_release squid 2024-04-25T01:15:43.055 INFO:teuthology.orchestra.run.smithi031.stdout:stretch_mode_enabled false 2024-04-25T01:15:43.055 INFO:teuthology.orchestra.run.smithi031.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T01:15:43.055 INFO:teuthology.orchestra.run.smithi031.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-25T01:15:43.055 INFO:teuthology.orchestra.run.smithi031.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T01:15:43.055 INFO:teuthology.orchestra.run.smithi031.stdout:max_osd 12 2024-04-25T01:15:43.055 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 up in weight 1 up_from 8 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6802/788174091,v1:172.21.15.31:6803/788174091] [v2:172.21.15.31:6804/788174091,v1:172.21.15.31:6805/788174091] exists,up caa08e4c-fe6b-4116-bdf5-1adf5b465d43 2024-04-25T01:15:43.055 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 up in weight 1 up_from 13 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.40:6800/3196394509,v1:172.21.15.40:6801/3196394509] [v2:172.21.15.40:6802/3196394509,v1:172.21.15.40:6803/3196394509] exists,up 85339fe7-0dbc-4485-9334-2b039993c1f1 2024-04-25T01:15:43.055 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 up in weight 1 up_from 18 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6800/1955211197,v1:172.21.15.169:6801/1955211197] [v2:172.21.15.169:6802/1955211197,v1:172.21.15.169:6803/1955211197] exists,up ab62ecb2-8aae-4987-9e2e-589e0f2fee5c 2024-04-25T01:15:43.055 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 up in weight 1 up_from 27 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6810/1228688643,v1:172.21.15.31:6811/1228688643] [v2:172.21.15.31:6812/1228688643,v1:172.21.15.31:6813/1228688643] exists,up 8b5a7881-3401-495c-8ca4-c90685919d47 2024-04-25T01:15:43.055 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 up in weight 1 up_from 35 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.40:6808/1913779091,v1:172.21.15.40:6809/1913779091] [v2:172.21.15.40:6810/1913779091,v1:172.21.15.40:6811/1913779091] exists,up 56deb61d-a334-4cf2-8ac7-fbb6e6947092 2024-04-25T01:15:43.055 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 up in weight 1 up_from 40 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6808/2549650781,v1:172.21.15.169:6809/2549650781] [v2:172.21.15.169:6810/2549650781,v1:172.21.15.169:6811/2549650781] exists,up 8da55693-f9c7-49a7-99e0-165078f8001b 2024-04-25T01:15:43.055 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6 up in weight 1 up_from 48 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6818/611507624,v1:172.21.15.31:6819/611507624] [v2:172.21.15.31:6820/611507624,v1:172.21.15.31:6821/611507624] exists,up fc8350d2-1ad6-43cb-89f4-d336afc99ee3 2024-04-25T01:15:43.055 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7 up in weight 1 up_from 53 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.40:6816/3776085849,v1:172.21.15.40:6817/3776085849] [v2:172.21.15.40:6818/3776085849,v1:172.21.15.40:6819/3776085849] exists,up 3121b58b-8c94-40b5-9d48-f56552d7c39b 2024-04-25T01:15:43.056 INFO:teuthology.orchestra.run.smithi031.stdout:osd.8 up in weight 1 up_from 61 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6816/2736546122,v1:172.21.15.169:6817/2736546122] [v2:172.21.15.169:6818/2736546122,v1:172.21.15.169:6819/2736546122] exists,up f60407df-91f5-477a-9702-ddaef24faf82 2024-04-25T01:15:43.056 INFO:teuthology.orchestra.run.smithi031.stdout:osd.9 up in weight 1 up_from 69 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6826/1187207880,v1:172.21.15.31:6827/1187207880] [v2:172.21.15.31:6828/1187207880,v1:172.21.15.31:6829/1187207880] exists,up 7506a407-1960-49bc-9e38-7f2a8bf996ed 2024-04-25T01:15:43.056 INFO:teuthology.orchestra.run.smithi031.stdout:osd.10 up in weight 1 up_from 74 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.40:6824/3050148001,v1:172.21.15.40:6825/3050148001] [v2:172.21.15.40:6826/3050148001,v1:172.21.15.40:6827/3050148001] exists,up 831c48e2-b0d0-451f-959b-228e927542cd 2024-04-25T01:15:43.056 INFO:teuthology.orchestra.run.smithi031.stdout:osd.11 up in weight 1 up_from 82 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6824/262401944,v1:172.21.15.169:6825/262401944] [v2:172.21.15.169:6826/262401944,v1:172.21.15.169:6827/262401944] exists,up 144fe0aa-f757-4b3c-90ad-99775e17d57f 2024-04-25T01:15:43.056 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.a [2,11] 2024-04-25T01:15:43.056 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.b [2,4,8,11] 2024-04-25T01:15:43.056 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.20 [5,9] 2024-04-25T01:15:43.056 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.24 [4,9] 2024-04-25T01:15:43.056 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.38 [3,9] 2024-04-25T01:15:43.056 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.3a [4,8] 2024-04-25T01:15:43.056 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6801/1901973432 expires 2024-04-26T01:05:31.932309+0000 2024-04-25T01:15:43.057 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/4004014066 expires 2024-04-26T01:05:31.932309+0000 2024-04-25T01:15:43.057 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6800/1901973432 expires 2024-04-26T01:05:31.932309+0000 2024-04-25T01:15:43.057 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/1930101646 expires 2024-04-26T01:05:31.932309+0000 2024-04-25T01:15:43.057 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6800/2638309002 expires 2024-04-26T01:04:57.149272+0000 2024-04-25T01:15:43.057 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/2238638505 expires 2024-04-26T01:04:57.149272+0000 2024-04-25T01:15:43.057 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/2902841144 expires 2024-04-26T01:05:31.932309+0000 2024-04-25T01:15:43.057 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/4190696323 expires 2024-04-26T01:04:57.149272+0000 2024-04-25T01:15:43.057 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/480558521 expires 2024-04-26T01:04:57.149272+0000 2024-04-25T01:15:43.057 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6801/2638309002 expires 2024-04-26T01:04:57.149272+0000 2024-04-25T01:15:43.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:43.072+0000 7f40bf800700 1 -- 172.21.15.31:0/1742025068 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f40940a2ca0 msgr2=0x7f40940a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:43.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:43.072+0000 7f40bf800700 1 --2- 172.21.15.31:0/1742025068 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f40940a2ca0 0x7f40940a5160 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f40a4000c00 tx=0x7f40a4005fb0 comp rx=0 tx=0).stop 2024-04-25T01:15:43.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:43.072+0000 7f40bf800700 1 -- 172.21.15.31:0/1742025068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b81044d0 msgr2=0x7f40b819a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:43.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:43.072+0000 7f40bf800700 1 --2- 172.21.15.31:0/1742025068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b81044d0 0x7f40b819a6a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f409c00e9a0 tx=0x7f409c00ecb0 comp rx=0 tx=0).stop 2024-04-25T01:15:43.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:43.072+0000 7f40bf800700 1 -- 172.21.15.31:0/1742025068 shutdown_connections 2024-04-25T01:15:43.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:43.072+0000 7f40bf800700 1 --2- 172.21.15.31:0/1742025068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b81044d0 0x7f40b819a6a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:43.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:43.072+0000 7f40bf800700 1 --2- 172.21.15.31:0/1742025068 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f40940a2ca0 0x7f40940a5160 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:43.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:43.072+0000 7f40bf800700 1 --2- 172.21.15.31:0/1742025068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40b8106070 0x7f40b81a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:43.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:43.072+0000 7f40bf800700 1 --2- 172.21.15.31:0/1742025068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40b81056d0 0x7f40b819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:43.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:43.072+0000 7f40bf800700 1 -- 172.21.15.31:0/1742025068 >> 172.21.15.31:0/1742025068 conn(0x7f40b80ffa90 msgr2=0x7f40b8108970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:43.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:43.072+0000 7f40bf800700 1 -- 172.21.15.31:0/1742025068 shutdown_connections 2024-04-25T01:15:43.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:43.072+0000 7f40bf800700 1 -- 172.21.15.31:0/1742025068 wait complete. 2024-04-25T01:15:43.425 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:43 smithi031 ceph-mon[28053]: pgmap v384: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:43.425 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:43 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1742025068' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T01:15:43.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:43 smithi169 ceph-mon[32759]: pgmap v384: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:43.694 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:43 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1742025068' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T01:15:43.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:43 smithi040 ceph-mon[32518]: pgmap v384: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:43.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:43 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1742025068' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T01:15:43.790 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T01:15:43.802 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:15:44.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.049+0000 7f45fc600640 1 -- 172.21.15.31:0/3980715087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45f4109650 msgr2=0x7f45f410bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:44.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.049+0000 7f45fc600640 1 --2- 172.21.15.31:0/3980715087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45f4109650 0x7f45f410bae0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009ca0 tx=0x7f45ec02a8c0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.049+0000 7f45fc600640 1 -- 172.21.15.31:0/3980715087 shutdown_connections 2024-04-25T01:15:44.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.049+0000 7f45fc600640 1 --2- 172.21.15.31:0/3980715087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45f4109650 0x7f45f410bae0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.049+0000 7f45fc600640 1 --2- 172.21.15.31:0/3980715087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45f40741a0 0x7f45f4074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.049+0000 7f45fc600640 1 --2- 172.21.15.31:0/3980715087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45f4075780 0x7f45f4073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.049+0000 7f45fc600640 1 -- 172.21.15.31:0/3980715087 >> 172.21.15.31:0/3980715087 conn(0x7f45f40fcd40 msgr2=0x7f45f40ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:44.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.050+0000 7f45fc600640 1 -- 172.21.15.31:0/3980715087 shutdown_connections 2024-04-25T01:15:44.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.050+0000 7f45fc600640 1 -- 172.21.15.31:0/3980715087 wait complete. 2024-04-25T01:15:44.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.051+0000 7f45fc600640 1 Processor -- start 2024-04-25T01:15:44.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.051+0000 7f45fc600640 1 -- start start 2024-04-25T01:15:44.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.051+0000 7f45fc600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45f40741a0 0x7f45f4199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:44.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.051+0000 7f45fc600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45f4075780 0x7f45f419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:44.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.051+0000 7f45fc600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45f4109650 0x7f45f41a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:44.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.051+0000 7f45fc600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f45f410f5f0 con 0x7f45f4109650 2024-04-25T01:15:44.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.051+0000 7f45fc600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f45f410f470 con 0x7f45f40741a0 2024-04-25T01:15:44.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.051+0000 7f45fc600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f45f410f770 con 0x7f45f4075780 2024-04-25T01:15:44.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.051+0000 7f45f9200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45f4075780 0x7f45f419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:44.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.052+0000 7f45fa600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45f4109650 0x7f45f41a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:44.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.052+0000 7f45f9200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45f4075780 0x7f45f419a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:57558/0 (socket says 172.21.15.31:57558) 2024-04-25T01:15:44.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.052+0000 7f45f9200640 1 -- 172.21.15.31:0/207365743 learned_addr learned my addr 172.21.15.31:0/207365743 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:44.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.052+0000 7f45f9c00640 1 --2- 172.21.15.31:0/207365743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45f40741a0 0x7f45f4199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:44.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.052+0000 7f45f9200640 1 -- 172.21.15.31:0/207365743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45f40741a0 msgr2=0x7f45f4199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.052+0000 7f45f9200640 1 --2- 172.21.15.31:0/207365743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45f40741a0 0x7f45f4199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.052+0000 7f45f9200640 1 -- 172.21.15.31:0/207365743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45f4109650 msgr2=0x7f45f41a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.052+0000 7f45f9200640 1 --2- 172.21.15.31:0/207365743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45f4109650 0x7f45f41a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.052+0000 7f45f9200640 1 -- 172.21.15.31:0/207365743 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f41a1850 con 0x7f45f4075780 2024-04-25T01:15:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.052+0000 7f45f9200640 1 --2- 172.21.15.31:0/207365743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45f4075780 0x7f45f419a180 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f45e000ea60 tx=0x7f45e000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.052+0000 7f45f2000640 1 -- 172.21.15.31:0/207365743 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e000cde0 con 0x7f45f4075780 2024-04-25T01:15:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.053+0000 7f45f2000640 1 -- 172.21.15.31:0/207365743 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45e00035f0 con 0x7f45f4075780 2024-04-25T01:15:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.053+0000 7f45f2000640 1 -- 172.21.15.31:0/207365743 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e0010670 con 0x7f45f4075780 2024-04-25T01:15:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.053+0000 7f45fc600640 1 -- 172.21.15.31:0/207365743 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45f41a1b40 con 0x7f45f4075780 2024-04-25T01:15:44.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.053+0000 7f45fc600640 1 -- 172.21.15.31:0/207365743 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45f41a2040 con 0x7f45f4075780 2024-04-25T01:15:44.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.054+0000 7f45fc600640 1 -- 172.21.15.31:0/207365743 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45f4108380 con 0x7f45f4075780 2024-04-25T01:15:44.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.056+0000 7f45f2000640 1 -- 172.21.15.31:0/207365743 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f45e00037a0 con 0x7f45f4075780 2024-04-25T01:15:44.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.057+0000 7f45f2000640 1 --2- 172.21.15.31:0/207365743 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f45dc0a2c10 0x7f45dc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:44.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.058+0000 7f45f2000640 1 -- 172.21.15.31:0/207365743 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f45e00cf320 con 0x7f45f4075780 2024-04-25T01:15:44.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.058+0000 7f45f9c00640 1 --2- 172.21.15.31:0/207365743 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f45dc0a2c10 0x7f45dc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:44.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.058+0000 7f45f9c00640 1 --2- 172.21.15.31:0/207365743 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f45dc0a2c10 0x7f45dc0a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f45f419b110 tx=0x7f45e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:44.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.060+0000 7f45f2000640 1 -- 172.21.15.31:0/207365743 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f45e009c270 con 0x7f45f4075780 2024-04-25T01:15:44.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.266+0000 7f45fc600640 1 -- 172.21.15.31:0/207365743 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45f402d3e0 con 0x7f45f4075780 2024-04-25T01:15:44.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.268+0000 7f45f2000640 1 -- 172.21.15.31:0/207365743 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22530 (secure 0 0 0) 0x7f45e009bb70 con 0x7f45f4075780 2024-04-25T01:15:44.253 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:15:44.254 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":89,"fsid":"95849b94-029f-11ef-bc93-c7b262605968","created":"2024-04-25T01:04:23.310856+0000","modified":"2024-04-25T01:14:33.793638+0000","last_up_change":"2024-04-25T01:12:29.393907+0000","last_in_change":"2024-04-25T01:12:08.978148+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:18.662391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:13:29.258771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:13:30.480996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa08e4c-fe6b-4116-bdf5-1adf5b465d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6803","nonce":788174091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6805","nonce":788174091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6809","nonce":788174091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6807","nonce":788174091}]},"public_addr":"172.21.15.31:6803/788174091","cluster_addr":"172.21.15.31:6805/788174091","heartbeat_back_addr":"172.21.15.31:6809/788174091","heartbeat_front_addr":"172.21.15.31:6807/788174091","state":["exists","up"]},{"osd":1,"uuid":"85339fe7-0dbc-4485-9334-2b039993c1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6801","nonce":3196394509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6803","nonce":3196394509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6807","nonce":3196394509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6805","nonce":3196394509}]},"public_addr":"172.21.15.40:6801/3196394509","cluster_addr":"172.21.15.40:6803/3196394509","heartbeat_back_addr":"172.21.15.40:6807/3196394509","heartbeat_front_addr":"172.21.15.40:6805/3196394509","state":["exists","up"]},{"osd":2,"uuid":"ab62ecb2-8aae-4987-9e2e-589e0f2fee5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6801","nonce":1955211197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6803","nonce":1955211197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6807","nonce":1955211197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6805","nonce":1955211197}]},"public_addr":"172.21.15.169:6801/1955211197","cluster_addr":"172.21.15.169:6803/1955211197","heartbeat_back_addr":"172.21.15.169:6807/1955211197","heartbeat_front_addr":"172.21.15.169:6805/1955211197","state":["exists","up"]},{"osd":3,"uuid":"8b5a7881-3401-495c-8ca4-c90685919d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6811","nonce":1228688643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6813","nonce":1228688643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6817","nonce":1228688643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6815","nonce":1228688643}]},"public_addr":"172.21.15.31:6811/1228688643","cluster_addr":"172.21.15.31:6813/1228688643","heartbeat_back_addr":"172.21.15.31:6817/1228688643","heartbeat_front_addr":"172.21.15.31:6815/1228688643","state":["exists","up"]},{"osd":4,"uuid":"56deb61d-a334-4cf2-8ac7-fbb6e6947092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6809","nonce":1913779091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6811","nonce":1913779091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6815","nonce":1913779091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6813","nonce":1913779091}]},"public_addr":"172.21.15.40:6809/1913779091","cluster_addr":"172.21.15.40:6811/1913779091","heartbeat_back_addr":"172.21.15.40:6815/1913779091","heartbeat_front_addr":"172.21.15.40:6813/1913779091","state":["exists","up"]},{"osd":5,"uuid":"8da55693-f9c7-49a7-99e0-165078f8001b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6809","nonce":2549650781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6811","nonce":2549650781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6815","nonce":2549650781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6813","nonce":2549650781}]},"public_addr":"172.21.15.169:6809/2549650781","cluster_addr":"172.21.15.169:6811/2549650781","heartbeat_back_addr":"172.21.15.169:6815/2549650781","heartbeat_front_addr":"172.21.15.169:6813/2549650781","state":["exists","up"]},{"osd":6,"uuid":"fc8350d2-1ad6-43cb-89f4-d336afc99ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6819","nonce":611507624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6821","nonce":611507624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6825","nonce":611507624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6823","nonce":611507624}]},"public_addr":"172.21.15.31:6819/611507624","cluster_addr":"172.21.15.31:6821/611507624","heartbeat_back_addr":"172.21.15.31:6825/611507624","heartbeat_front_addr":"172.21.15.31:6823/611507624","state":["exists","up"]},{"osd":7,"uuid":"3121b58b-8c94-40b5-9d48-f56552d7c39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6817","nonce":3776085849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6819","nonce":3776085849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6823","nonce":3776085849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6821","nonce":3776085849}]},"public_addr":"172.21.15.40:6817/3776085849","cluster_addr":"172.21.15.40:6819/3776085849","heartbeat_back_addr":"172.21.15.40:6823/3776085849","heartbeat_front_addr":"172.21.15.40:6821/3776085849","state":["exists","up"]},{"osd":8,"uuid":"f60407df-91f5-477a-9702-ddaef24faf82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6817","nonce":2736546122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6819","nonce":2736546122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6823","nonce":2736546122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6821","nonce":2736546122}]},"public_addr":"172.21.15.169:6817/2736546122","cluster_addr":"172.21.15.169:6819/2736546122","heartbeat_back_addr":"172.21.15.169:6823/2736546122","heartbeat_front_addr":"172.21.15.169:6821/2736546122","state":["exists","up"]},{"osd":9,"uuid":"7506a407-1960-49bc-9e38-7f2a8bf996ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6827","nonce":1187207880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6829","nonce":1187207880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6833","nonce":1187207880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6831","nonce":1187207880}]},"public_addr":"172.21.15.31:6827/1187207880","cluster_addr":"172.21.15.31:6829/1187207880","heartbeat_back_addr":"172.21.15.31:6833/1187207880","heartbeat_front_addr":"172.21.15.31:6831/1187207880","state":["exists","up"]},{"osd":10,"uuid":"831c48e2-b0d0-451f-959b-228e927542cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6825","nonce":3050148001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6827","nonce":3050148001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6831","nonce":3050148001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6829","nonce":3050148001}]},"public_addr":"172.21.15.40:6825/3050148001","cluster_addr":"172.21.15.40:6827/3050148001","heartbeat_back_addr":"172.21.15.40:6831/3050148001","heartbeat_front_addr":"172.21.15.40:6829/3050148001","state":["exists","up"]},{"osd":11,"uuid":"144fe0aa-f757-4b3c-90ad-99775e17d57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6825","nonce":262401944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6827","nonce":262401944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6831","nonce":262401944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6829","nonce":262401944}]},"public_addr":"172.21.15.169:6825/262401944","cluster_addr":"172.21.15.169:6827/262401944","heartbeat_back_addr":"172.21.15.169:6831/262401944","heartbeat_front_addr":"172.21.15.169:6829/262401944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:25.640995+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:51.036909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.635633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:48.467912+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:14.454971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:37.609315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:12.441807+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:38.239215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:02.230758+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:35.762553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:02.051405+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:26.569593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6801/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4004014066":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/1930101646":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/2638309002":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2238638505":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2902841144":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4190696323":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/480558521":"2024-04-26T01:04:57.149272+0000","172.21.15.31:6801/2638309002":"2024-04-26T01:04:57.149272+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:15:44.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.270+0000 7f45fc600640 1 -- 172.21.15.31:0/207365743 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f45dc0a2c10 msgr2=0x7f45dc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:44.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.270+0000 7f45fc600640 1 --2- 172.21.15.31:0/207365743 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f45dc0a2c10 0x7f45dc0a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f45f419b110 tx=0x7f45e8005f90 comp rx=0 tx=0).stop 2024-04-25T01:15:44.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.270+0000 7f45fc600640 1 -- 172.21.15.31:0/207365743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45f4075780 msgr2=0x7f45f419a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:44.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.270+0000 7f45fc600640 1 --2- 172.21.15.31:0/207365743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45f4075780 0x7f45f419a180 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f45e000ea60 tx=0x7f45e000ef20 comp rx=0 tx=0).stop 2024-04-25T01:15:44.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.270+0000 7f45fc600640 1 -- 172.21.15.31:0/207365743 shutdown_connections 2024-04-25T01:15:44.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.270+0000 7f45fc600640 1 --2- 172.21.15.31:0/207365743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45f4109650 0x7f45f41a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.270+0000 7f45fc600640 1 --2- 172.21.15.31:0/207365743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45f4075780 0x7f45f419a180 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.270+0000 7f45fc600640 1 --2- 172.21.15.31:0/207365743 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f45dc0a2c10 0x7f45dc0a50d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.270+0000 7f45fc600640 1 --2- 172.21.15.31:0/207365743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45f40741a0 0x7f45f4199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.270+0000 7f45fc600640 1 -- 172.21.15.31:0/207365743 >> 172.21.15.31:0/207365743 conn(0x7f45f40fcd40 msgr2=0x7f45f4105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:44.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.271+0000 7f45fc600640 1 -- 172.21.15.31:0/207365743 shutdown_connections 2024-04-25T01:15:44.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.271+0000 7f45fc600640 1 -- 172.21.15.31:0/207365743 wait complete. 2024-04-25T01:15:44.275 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:18.662391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:13:29.258771+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:13:30.480996+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:15:44.276 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:15:44.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.508+0000 7fa27aa00640 1 -- 172.21.15.31:0/1902319140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa274076100 msgr2=0x7fa274076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:44.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.508+0000 7fa27aa00640 1 --2- 172.21.15.31:0/1902319140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa274076100 0x7fa274076520 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa264009900 tx=0x7fa26402a3f0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.508+0000 7fa27aa00640 1 -- 172.21.15.31:0/1902319140 shutdown_connections 2024-04-25T01:15:44.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.508+0000 7fa27aa00640 1 --2- 172.21.15.31:0/1902319140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa274077ce0 0x7fa274078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.508+0000 7fa27aa00640 1 --2- 172.21.15.31:0/1902319140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa274077340 0x7fa2740777a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.508+0000 7fa27aa00640 1 --2- 172.21.15.31:0/1902319140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa274076100 0x7fa274076520 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.508+0000 7fa27aa00640 1 -- 172.21.15.31:0/1902319140 >> 172.21.15.31:0/1902319140 conn(0x7fa2740fef30 msgr2=0x7fa274101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:44.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.508+0000 7fa27aa00640 1 -- 172.21.15.31:0/1902319140 shutdown_connections 2024-04-25T01:15:44.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.508+0000 7fa27aa00640 1 -- 172.21.15.31:0/1902319140 wait complete. 2024-04-25T01:15:44.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.509+0000 7fa27aa00640 1 Processor -- start 2024-04-25T01:15:44.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.509+0000 7fa27aa00640 1 -- start start 2024-04-25T01:15:44.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.510+0000 7fa27aa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa274076100 0x7fa27419e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:44.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.510+0000 7fa27aa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa274077340 0x7fa27419e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:44.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.510+0000 7fa27aa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa274077ce0 0x7fa2741a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:44.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.510+0000 7fa27aa00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa274113a30 con 0x7fa274076100 2024-04-25T01:15:44.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.510+0000 7fa27aa00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa2741138b0 con 0x7fa274077ce0 2024-04-25T01:15:44.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.510+0000 7fa27aa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa274113bb0 con 0x7fa274077340 2024-04-25T01:15:44.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.510+0000 7fa273e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa274076100 0x7fa27419e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:44.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.510+0000 7fa273e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa274076100 0x7fa27419e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58030/0 (socket says 172.21.15.31:58030) 2024-04-25T01:15:44.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.510+0000 7fa273e00640 1 -- 172.21.15.31:0/828542101 learned_addr learned my addr 172.21.15.31:0/828542101 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:44.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.510+0000 7fa26be00640 1 --2- 172.21.15.31:0/828542101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa274077340 0x7fa27419e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:44.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.510+0000 7fa278a00640 1 --2- 172.21.15.31:0/828542101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa274077ce0 0x7fa2741a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:44.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.511+0000 7fa273e00640 1 -- 172.21.15.31:0/828542101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa274077340 msgr2=0x7fa27419e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:44.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.511+0000 7fa273e00640 1 --2- 172.21.15.31:0/828542101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa274077340 0x7fa27419e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.511+0000 7fa273e00640 1 -- 172.21.15.31:0/828542101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa274077ce0 msgr2=0x7fa2741a56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:44.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.511+0000 7fa273e00640 1 --2- 172.21.15.31:0/828542101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa274077ce0 0x7fa2741a56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.511+0000 7fa273e00640 1 -- 172.21.15.31:0/828542101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2741a5d20 con 0x7fa274076100 2024-04-25T01:15:44.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.511+0000 7fa26be00640 1 --2- 172.21.15.31:0/828542101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa274077340 0x7fa27419e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:44.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.511+0000 7fa273e00640 1 --2- 172.21.15.31:0/828542101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa274076100 0x7fa27419e100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa264009230 tx=0x7fa2640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:44.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.511+0000 7fa271600640 1 -- 172.21.15.31:0/828542101 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26403f030 con 0x7fa274076100 2024-04-25T01:15:44.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.511+0000 7fa271600640 1 -- 172.21.15.31:0/828542101 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa264003ce0 con 0x7fa274076100 2024-04-25T01:15:44.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.511+0000 7fa271600640 1 -- 172.21.15.31:0/828542101 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa264035070 con 0x7fa274076100 2024-04-25T01:15:44.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.511+0000 7fa27aa00640 1 -- 172.21.15.31:0/828542101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2741a5fb0 con 0x7fa274076100 2024-04-25T01:15:44.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.511+0000 7fa27aa00640 1 -- 172.21.15.31:0/828542101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2741a6360 con 0x7fa274076100 2024-04-25T01:15:44.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.513+0000 7fa27aa00640 1 -- 172.21.15.31:0/828542101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa274110310 con 0x7fa274076100 2024-04-25T01:15:44.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.515+0000 7fa271600640 1 -- 172.21.15.31:0/828542101 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa264042030 con 0x7fa274076100 2024-04-25T01:15:44.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.516+0000 7fa271600640 1 --2- 172.21.15.31:0/828542101 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa2540a2c10 0x7fa2540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:44.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.516+0000 7fa271600640 1 -- 172.21.15.31:0/828542101 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fa2640ef7c0 con 0x7fa274076100 2024-04-25T01:15:44.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.517+0000 7fa26be00640 1 --2- 172.21.15.31:0/828542101 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa2540a2c10 0x7fa2540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:44.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.517+0000 7fa26be00640 1 --2- 172.21.15.31:0/828542101 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa2540a2c10 0x7fa2540a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa27419f610 tx=0x7fa25c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:44.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.518+0000 7fa271600640 1 -- 172.21.15.31:0/828542101 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa2640bc710 con 0x7fa274076100 2024-04-25T01:15:44.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.723+0000 7fa27aa00640 1 -- 172.21.15.31:0/828542101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa274062440 con 0x7fa274076100 2024-04-25T01:15:44.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.724+0000 7fa271600640 1 -- 172.21.15.31:0/828542101 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fa2640bc010 con 0x7fa274076100 2024-04-25T01:15:44.709 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T01:15:44.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.726+0000 7fa27aa00640 1 -- 172.21.15.31:0/828542101 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa2540a2c10 msgr2=0x7fa2540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:44.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.726+0000 7fa27aa00640 1 --2- 172.21.15.31:0/828542101 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa2540a2c10 0x7fa2540a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa27419f610 tx=0x7fa25c005f90 comp rx=0 tx=0).stop 2024-04-25T01:15:44.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.726+0000 7fa27aa00640 1 -- 172.21.15.31:0/828542101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa274076100 msgr2=0x7fa27419e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:44.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.726+0000 7fa27aa00640 1 --2- 172.21.15.31:0/828542101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa274076100 0x7fa27419e100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa264009230 tx=0x7fa2640042f0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.727+0000 7fa27aa00640 1 -- 172.21.15.31:0/828542101 shutdown_connections 2024-04-25T01:15:44.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.727+0000 7fa27aa00640 1 --2- 172.21.15.31:0/828542101 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa2540a2c10 0x7fa2540a50d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.727+0000 7fa27aa00640 1 --2- 172.21.15.31:0/828542101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa274077ce0 0x7fa2741a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.727+0000 7fa27aa00640 1 --2- 172.21.15.31:0/828542101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa274077340 0x7fa27419e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.727+0000 7fa27aa00640 1 --2- 172.21.15.31:0/828542101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa274076100 0x7fa27419e100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.727+0000 7fa27aa00640 1 -- 172.21.15.31:0/828542101 >> 172.21.15.31:0/828542101 conn(0x7fa2740fef30 msgr2=0x7fa274100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:44.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.727+0000 7fa27aa00640 1 -- 172.21.15.31:0/828542101 shutdown_connections 2024-04-25T01:15:44.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.727+0000 7fa27aa00640 1 -- 172.21.15.31:0/828542101 wait complete. 2024-04-25T01:15:44.732 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:15:44.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.960+0000 7f097fe00640 1 -- 172.21.15.31:0/2204862220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0980102100 msgr2=0x7f0980108990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:44.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.960+0000 7f097fe00640 1 --2- 172.21.15.31:0/2204862220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0980102100 0x7f0980108990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0978009860 tx=0x7f097802a340 comp rx=0 tx=0).stop 2024-04-25T01:15:44.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.960+0000 7f097fe00640 1 -- 172.21.15.31:0/2204862220 shutdown_connections 2024-04-25T01:15:44.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.960+0000 7f097fe00640 1 --2- 172.21.15.31:0/2204862220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0980102100 0x7f0980108990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.960+0000 7f097fe00640 1 --2- 172.21.15.31:0/2204862220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0980101740 0x7f0980101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.960+0000 7f097fe00640 1 --2- 172.21.15.31:0/2204862220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0980100550 0x7f0980100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.960+0000 7f097fe00640 1 -- 172.21.15.31:0/2204862220 >> 172.21.15.31:0/2204862220 conn(0x7f09800fbae0 msgr2=0x7f09800fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:44.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.961+0000 7f097fe00640 1 -- 172.21.15.31:0/2204862220 shutdown_connections 2024-04-25T01:15:44.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.961+0000 7f097fe00640 1 -- 172.21.15.31:0/2204862220 wait complete. 2024-04-25T01:15:44.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.962+0000 7f097fe00640 1 Processor -- start 2024-04-25T01:15:44.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.962+0000 7f097fe00640 1 -- start start 2024-04-25T01:15:44.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.962+0000 7f097fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0980100550 0x7f0980199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:44.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.962+0000 7f097fe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0980101740 0x7f098019a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:44.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.962+0000 7f097fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0980102100 0x7f09801a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:44.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.962+0000 7f097fe00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f098010c1e0 con 0x7f0980102100 2024-04-25T01:15:44.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.962+0000 7f097fe00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f098010c060 con 0x7f0980101740 2024-04-25T01:15:44.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.962+0000 7f097fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f098010c360 con 0x7f0980100550 2024-04-25T01:15:44.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.963+0000 7f097f400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0980102100 0x7f09801a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:44.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.963+0000 7f097e000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0980101740 0x7f098019a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:44.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.963+0000 7f097f400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0980102100 0x7f09801a10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58042/0 (socket says 172.21.15.31:58042) 2024-04-25T01:15:44.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.963+0000 7f097ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0980100550 0x7f0980199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:44.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.963+0000 7f097f400640 1 -- 172.21.15.31:0/3923662421 learned_addr learned my addr 172.21.15.31:0/3923662421 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:44.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.963+0000 7f097e000640 1 -- 172.21.15.31:0/3923662421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0980100550 msgr2=0x7f0980199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:44.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.963+0000 7f097e000640 1 --2- 172.21.15.31:0/3923662421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0980100550 0x7f0980199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.963+0000 7f097e000640 1 -- 172.21.15.31:0/3923662421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0980102100 msgr2=0x7f09801a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:44.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.963+0000 7f097e000640 1 --2- 172.21.15.31:0/3923662421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0980102100 0x7f09801a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:44.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.963+0000 7f097e000640 1 -- 172.21.15.31:0/3923662421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09801a1720 con 0x7f0980101740 2024-04-25T01:15:44.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.963+0000 7f097f400640 1 --2- 172.21.15.31:0/3923662421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0980102100 0x7f09801a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:44.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.963+0000 7f097ea00640 1 --2- 172.21.15.31:0/3923662421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0980100550 0x7f0980199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:44.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.963+0000 7f097e000640 1 --2- 172.21.15.31:0/3923662421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0980101740 0x7f098019a040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f096800e9f0 tx=0x7f096800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:44.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.964+0000 7f0984a00640 1 -- 172.21.15.31:0/3923662421 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096800ccc0 con 0x7f0980101740 2024-04-25T01:15:44.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.964+0000 7f097fe00640 1 -- 172.21.15.31:0/3923662421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09801a1a10 con 0x7f0980101740 2024-04-25T01:15:44.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.964+0000 7f0984a00640 1 -- 172.21.15.31:0/3923662421 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0968010910 con 0x7f0980101740 2024-04-25T01:15:44.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.964+0000 7f0984a00640 1 -- 172.21.15.31:0/3923662421 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968018960 con 0x7f0980101740 2024-04-25T01:15:44.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.964+0000 7f097fe00640 1 -- 172.21.15.31:0/3923662421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09801a1fb0 con 0x7f0980101740 2024-04-25T01:15:44.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.965+0000 7f097fe00640 1 -- 172.21.15.31:0/3923662421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09801a21e0 con 0x7f0980101740 2024-04-25T01:15:44.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.967+0000 7f0984a00640 1 -- 172.21.15.31:0/3923662421 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0968010ac0 con 0x7f0980101740 2024-04-25T01:15:44.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.968+0000 7f0984a00640 1 --2- 172.21.15.31:0/3923662421 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f09640a2c10 0x7f09640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:44.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.969+0000 7f0984a00640 1 -- 172.21.15.31:0/3923662421 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f09680cf360 con 0x7f0980101740 2024-04-25T01:15:44.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.969+0000 7f097ea00640 1 --2- 172.21.15.31:0/3923662421 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f09640a2c10 0x7f09640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:44.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.969+0000 7f097ea00640 1 --2- 172.21.15.31:0/3923662421 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f09640a2c10 0x7f09640a50d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f09801015a0 tx=0x7f0970010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:44.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:44.973+0000 7f0984a00640 1 -- 172.21.15.31:0/3923662421 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f096809bc60 con 0x7f0980101740 2024-04-25T01:15:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.178+0000 7f097fe00640 1 -- 172.21.15.31:0/3923662421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f09801a2450 con 0x7f0980101740 2024-04-25T01:15:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.179+0000 7f0984a00640 1 -- 172.21.15.31:0/3923662421 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f096809ff60 con 0x7f0980101740 2024-04-25T01:15:45.164 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-25T01:15:45.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.182+0000 7f097fe00640 1 -- 172.21.15.31:0/3923662421 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f09640a2c10 msgr2=0x7f09640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:45.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.182+0000 7f097fe00640 1 --2- 172.21.15.31:0/3923662421 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f09640a2c10 0x7f09640a50d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f09801015a0 tx=0x7f0970010040 comp rx=0 tx=0).stop 2024-04-25T01:15:45.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.182+0000 7f097fe00640 1 -- 172.21.15.31:0/3923662421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0980101740 msgr2=0x7f098019a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:45.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.182+0000 7f097fe00640 1 --2- 172.21.15.31:0/3923662421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0980101740 0x7f098019a040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f096800e9f0 tx=0x7f096800eeb0 comp rx=0 tx=0).stop 2024-04-25T01:15:45.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.182+0000 7f097fe00640 1 -- 172.21.15.31:0/3923662421 shutdown_connections 2024-04-25T01:15:45.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.182+0000 7f097fe00640 1 --2- 172.21.15.31:0/3923662421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0980102100 0x7f09801a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:45.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.182+0000 7f097fe00640 1 --2- 172.21.15.31:0/3923662421 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f09640a2c10 0x7f09640a50d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:45.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.182+0000 7f097fe00640 1 --2- 172.21.15.31:0/3923662421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0980101740 0x7f098019a040 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:45.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.182+0000 7f097fe00640 1 --2- 172.21.15.31:0/3923662421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0980100550 0x7f0980199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:45.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.182+0000 7f097fe00640 1 -- 172.21.15.31:0/3923662421 >> 172.21.15.31:0/3923662421 conn(0x7f09800fbae0 msgr2=0x7f09800fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:45.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.182+0000 7f097fe00640 1 -- 172.21.15.31:0/3923662421 shutdown_connections 2024-04-25T01:15:45.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.182+0000 7f097fe00640 1 -- 172.21.15.31:0/3923662421 wait complete. 2024-04-25T01:15:45.188 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:15:45.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.423+0000 7f71b7200640 1 -- 172.21.15.31:0/1530801670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b00741a0 msgr2=0x7f71b0074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:45.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.423+0000 7f71b7200640 1 --2- 172.21.15.31:0/1530801670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b00741a0 0x7f71b0074620 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009900 tx=0x7f71a002a3f0 comp rx=0 tx=0).stop 2024-04-25T01:15:45.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.423+0000 7f71b7200640 1 -- 172.21.15.31:0/1530801670 shutdown_connections 2024-04-25T01:15:45.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.423+0000 7f71b7200640 1 --2- 172.21.15.31:0/1530801670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71b0109590 0x7f71b010ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:45.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.423+0000 7f71b7200640 1 --2- 172.21.15.31:0/1530801670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b00741a0 0x7f71b0074620 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:45.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.423+0000 7f71b7200640 1 --2- 172.21.15.31:0/1530801670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71b0075780 0x7f71b0073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:45.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.423+0000 7f71b7200640 1 -- 172.21.15.31:0/1530801670 >> 172.21.15.31:0/1530801670 conn(0x7f71b00fcc40 msgr2=0x7f71b00ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:45.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.423+0000 7f71b7200640 1 -- 172.21.15.31:0/1530801670 shutdown_connections 2024-04-25T01:15:45.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.423+0000 7f71b7200640 1 -- 172.21.15.31:0/1530801670 wait complete. 2024-04-25T01:15:45.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.424+0000 7f71b7200640 1 Processor -- start 2024-04-25T01:15:45.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.425+0000 7f71b7200640 1 -- start start 2024-04-25T01:15:45.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.425+0000 7f71b7200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71b00741a0 0x7f71b0199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:45.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.425+0000 7f71b7200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b0075780 0x7f71b019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:45.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.425+0000 7f71b7200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71b0109590 0x7f71b01a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:45.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.425+0000 7f71b7200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f71b010f510 con 0x7f71b0075780 2024-04-25T01:15:45.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.425+0000 7f71b7200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f71b010f390 con 0x7f71b0109590 2024-04-25T01:15:45.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.425+0000 7f71b7200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f71b010f690 con 0x7f71b00741a0 2024-04-25T01:15:45.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.425+0000 7f71b5400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b0075780 0x7f71b019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:45.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.425+0000 7f71b5400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b0075780 0x7f71b019a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58074/0 (socket says 172.21.15.31:58074) 2024-04-25T01:15:45.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.425+0000 7f71b5400640 1 -- 172.21.15.31:0/1118821178 learned_addr learned my addr 172.21.15.31:0/1118821178 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:45.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.425+0000 7f71b5e00640 1 --2- 172.21.15.31:0/1118821178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71b00741a0 0x7f71b0199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:45.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.425+0000 7f71b6800640 1 --2- 172.21.15.31:0/1118821178 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71b0109590 0x7f71b01a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:45.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.426+0000 7f71b5400640 1 -- 172.21.15.31:0/1118821178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71b00741a0 msgr2=0x7f71b0199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:45.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.426+0000 7f71b5400640 1 --2- 172.21.15.31:0/1118821178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71b00741a0 0x7f71b0199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:45.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.426+0000 7f71b5400640 1 -- 172.21.15.31:0/1118821178 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71b0109590 msgr2=0x7f71b01a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:45.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.426+0000 7f71b5400640 1 --2- 172.21.15.31:0/1118821178 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71b0109590 0x7f71b01a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:45.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.426+0000 7f71b5400640 1 -- 172.21.15.31:0/1118821178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b01a17a0 con 0x7f71b0075780 2024-04-25T01:15:45.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.426+0000 7f71b5400640 1 --2- 172.21.15.31:0/1118821178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b0075780 0x7f71b019a0c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f71a0031440 tx=0x7f71a0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:45.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.426+0000 7f71a6a00640 1 -- 172.21.15.31:0/1118821178 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a003f030 con 0x7f71b0075780 2024-04-25T01:15:45.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.426+0000 7f71b7200640 1 -- 172.21.15.31:0/1118821178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b01a1a30 con 0x7f71b0075780 2024-04-25T01:15:45.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.426+0000 7f71a6a00640 1 -- 172.21.15.31:0/1118821178 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71a0003ce0 con 0x7f71b0075780 2024-04-25T01:15:45.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.426+0000 7f71a6a00640 1 -- 172.21.15.31:0/1118821178 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a0035070 con 0x7f71b0075780 2024-04-25T01:15:45.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.427+0000 7f71b7200640 1 -- 172.21.15.31:0/1118821178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b01a1f70 con 0x7f71b0075780 2024-04-25T01:15:45.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.428+0000 7f71b7200640 1 -- 172.21.15.31:0/1118821178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71b01082c0 con 0x7f71b0075780 2024-04-25T01:15:45.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.430+0000 7f71a6a00640 1 -- 172.21.15.31:0/1118821178 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f71a0042030 con 0x7f71b0075780 2024-04-25T01:15:45.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.431+0000 7f71a6a00640 1 --2- 172.21.15.31:0/1118821178 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f719c0a2c10 0x7f719c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:45.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.431+0000 7f71a6a00640 1 -- 172.21.15.31:0/1118821178 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f71a00ef800 con 0x7f71b0075780 2024-04-25T01:15:45.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.431+0000 7f71b5e00640 1 --2- 172.21.15.31:0/1118821178 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f719c0a2c10 0x7f719c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:45.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.432+0000 7f71b5e00640 1 --2- 172.21.15.31:0/1118821178 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f719c0a2c10 0x7f719c0a50d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f71a8005fd0 tx=0x7f71a8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:45.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.435+0000 7f71a6a00640 1 -- 172.21.15.31:0/1118821178 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f71a00bc750 con 0x7f71b0075780 2024-04-25T01:15:45.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:45 smithi031 ceph-mon[28053]: pgmap v385: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:45.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:45 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/207365743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:15:45.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:45 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/828542101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:15:45.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:45 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3923662421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:15:45.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.641+0000 7f71b7200640 1 -- 172.21.15.31:0/1118821178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f71b0062340 con 0x7f71b0075780 2024-04-25T01:15:45.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.642+0000 7f71a6a00640 1 -- 172.21.15.31:0/1118821178 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f71a00bc050 con 0x7f71b0075780 2024-04-25T01:15:45.627 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-04-25T01:15:45.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.645+0000 7f71b7200640 1 -- 172.21.15.31:0/1118821178 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f719c0a2c10 msgr2=0x7f719c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:45.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.645+0000 7f71b7200640 1 --2- 172.21.15.31:0/1118821178 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f719c0a2c10 0x7f719c0a50d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f71a8005fd0 tx=0x7f71a8009500 comp rx=0 tx=0).stop 2024-04-25T01:15:45.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.645+0000 7f71b7200640 1 -- 172.21.15.31:0/1118821178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b0075780 msgr2=0x7f71b019a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:45.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.645+0000 7f71b7200640 1 --2- 172.21.15.31:0/1118821178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b0075780 0x7f71b019a0c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f71a0031440 tx=0x7f71a0032ce0 comp rx=0 tx=0).stop 2024-04-25T01:15:45.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.645+0000 7f71b7200640 1 -- 172.21.15.31:0/1118821178 shutdown_connections 2024-04-25T01:15:45.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.645+0000 7f71b7200640 1 --2- 172.21.15.31:0/1118821178 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f719c0a2c10 0x7f719c0a50d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:45.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.645+0000 7f71b7200640 1 --2- 172.21.15.31:0/1118821178 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71b0109590 0x7f71b01a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:45.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.645+0000 7f71b7200640 1 --2- 172.21.15.31:0/1118821178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71b0075780 0x7f71b019a0c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:45.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.645+0000 7f71b7200640 1 --2- 172.21.15.31:0/1118821178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71b00741a0 0x7f71b0199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:45.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.645+0000 7f71b7200640 1 -- 172.21.15.31:0/1118821178 >> 172.21.15.31:0/1118821178 conn(0x7f71b00fcc40 msgr2=0x7f71b0105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:45.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.645+0000 7f71b7200640 1 -- 172.21.15.31:0/1118821178 shutdown_connections 2024-04-25T01:15:45.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.645+0000 7f71b7200640 1 -- 172.21.15.31:0/1118821178 wait complete. 2024-04-25T01:15:45.649 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:15:45.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:45 smithi169 ceph-mon[32759]: pgmap v385: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:45.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:45 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/207365743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:15:45.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:45 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/828542101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:15:45.694 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:45 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3923662421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:15:45.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:45 smithi040 ceph-mon[32518]: pgmap v385: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:45.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:45 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/207365743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:15:45.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:45 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/828542101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:15:45.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:45 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3923662421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:15:45.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.879+0000 7f2ccfe00640 1 -- 172.21.15.31:0/3146600886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cd0077340 msgr2=0x7f2cd00777a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:45.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.879+0000 7f2ccfe00640 1 --2- 172.21.15.31:0/3146600886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cd0077340 0x7f2cd00777a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8009860 tx=0x7f2cb802a340 comp rx=0 tx=0).stop 2024-04-25T01:15:45.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.879+0000 7f2ccfe00640 1 -- 172.21.15.31:0/3146600886 shutdown_connections 2024-04-25T01:15:45.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.879+0000 7f2ccfe00640 1 --2- 172.21.15.31:0/3146600886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cd0077ce0 0x7f2cd0078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:45.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.879+0000 7f2ccfe00640 1 --2- 172.21.15.31:0/3146600886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cd0077340 0x7f2cd00777a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:45.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.879+0000 7f2ccfe00640 1 --2- 172.21.15.31:0/3146600886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cd0076100 0x7f2cd0076520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:45.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.879+0000 7f2ccfe00640 1 -- 172.21.15.31:0/3146600886 >> 172.21.15.31:0/3146600886 conn(0x7f2cd00fed50 msgr2=0x7f2cd0101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:45.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.880+0000 7f2ccfe00640 1 -- 172.21.15.31:0/3146600886 shutdown_connections 2024-04-25T01:15:45.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.880+0000 7f2ccfe00640 1 -- 172.21.15.31:0/3146600886 wait complete. 2024-04-25T01:15:45.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.881+0000 7f2ccfe00640 1 Processor -- start 2024-04-25T01:15:45.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.881+0000 7f2ccfe00640 1 -- start start 2024-04-25T01:15:45.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.881+0000 7f2ccfe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cd0076100 0x7f2cd019e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:45.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.881+0000 7f2ccfe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cd0077ce0 0x7f2cd019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:45.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.881+0000 7f2ccfe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cd01a3680 0x7f2cd01a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:45.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.881+0000 7f2ccfe00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd0113850 con 0x7f2cd0077ce0 2024-04-25T01:15:45.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.881+0000 7f2ccfe00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd01136d0 con 0x7f2cd01a3680 2024-04-25T01:15:45.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.881+0000 7f2ccfe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd01139d0 con 0x7f2cd0076100 2024-04-25T01:15:45.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.881+0000 7f2cce000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cd0077ce0 0x7f2cd019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:45.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.881+0000 7f2ccea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cd0076100 0x7f2cd019e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:45.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.881+0000 7f2ccea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cd0076100 0x7f2cd019e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:57632/0 (socket says 172.21.15.31:57632) 2024-04-25T01:15:45.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.881+0000 7f2ccea00640 1 -- 172.21.15.31:0/2138828784 learned_addr learned my addr 172.21.15.31:0/2138828784 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:45.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.882+0000 7f2ccf400640 1 --2- 172.21.15.31:0/2138828784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cd01a3680 0x7f2cd01a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:45.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.882+0000 7f2ccea00640 1 -- 172.21.15.31:0/2138828784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cd01a3680 msgr2=0x7f2cd01a5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:45.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.882+0000 7f2ccea00640 1 --2- 172.21.15.31:0/2138828784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cd01a3680 0x7f2cd01a5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:45.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.882+0000 7f2ccea00640 1 -- 172.21.15.31:0/2138828784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cd0077ce0 msgr2=0x7f2cd019e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:45.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.882+0000 7f2ccea00640 1 --2- 172.21.15.31:0/2138828784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cd0077ce0 0x7f2cd019e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:45.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.882+0000 7f2ccea00640 1 -- 172.21.15.31:0/2138828784 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd01a60e0 con 0x7f2cd0076100 2024-04-25T01:15:45.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.882+0000 7f2ccea00640 1 --2- 172.21.15.31:0/2138828784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cd0076100 0x7f2cd019e0e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000e9f0 tx=0x7f2cc000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:45.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.882+0000 7f2cc7400640 1 -- 172.21.15.31:0/2138828784 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc000cd80 con 0x7f2cd0076100 2024-04-25T01:15:45.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.882+0000 7f2cc7400640 1 -- 172.21.15.31:0/2138828784 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc0005500 con 0x7f2cd0076100 2024-04-25T01:15:45.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.882+0000 7f2ccfe00640 1 -- 172.21.15.31:0/2138828784 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd01a63d0 con 0x7f2cd0076100 2024-04-25T01:15:45.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.882+0000 7f2cc7400640 1 -- 172.21.15.31:0/2138828784 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0010690 con 0x7f2cd0076100 2024-04-25T01:15:45.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.883+0000 7f2ccfe00640 1 -- 172.21.15.31:0/2138828784 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd01a68a0 con 0x7f2cd0076100 2024-04-25T01:15:45.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.884+0000 7f2ccfe00640 1 -- 172.21.15.31:0/2138828784 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cd01103b0 con 0x7f2cd0076100 2024-04-25T01:15:45.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.886+0000 7f2cc7400640 1 -- 172.21.15.31:0/2138828784 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2cc00046c0 con 0x7f2cd0076100 2024-04-25T01:15:45.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.887+0000 7f2cc7400640 1 --2- 172.21.15.31:0/2138828784 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f2cb40a2b90 0x7f2cb40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:45.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.887+0000 7f2cc7400640 1 -- 172.21.15.31:0/2138828784 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f2cc00cf430 con 0x7f2cd0076100 2024-04-25T01:15:45.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.888+0000 7f2cce000640 1 --2- 172.21.15.31:0/2138828784 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f2cb40a2b90 0x7f2cb40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:45.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.888+0000 7f2cce000640 1 --2- 172.21.15.31:0/2138828784 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f2cb40a2b90 0x7f2cb40a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8035040 tx=0x7f2cb8030570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:45.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:45.891+0000 7f2cc7400640 1 -- 172.21.15.31:0/2138828784 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2cc009c380 con 0x7f2cd0076100 2024-04-25T01:15:46.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.139+0000 7f2ccfe00640 1 -- 172.21.15.31:0/2138828784 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2cd002d170 con 0x7f2cd0076100 2024-04-25T01:15:46.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.140+0000 7f2cc7400640 1 -- 172.21.15.31:0/2138828784 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11854 (secure 0 0 0) 0x7f2cc009bc80 con 0x7f2cd0076100 2024-04-25T01:15:46.126 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:15:46.126 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/498218714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":498218714},{"type":"v1","addr":"172.21.15.31:6835","nonce":498218714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2647199475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2647199475},{"type":"v1","addr":"172.21.15.31:6839","nonce":2647199475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/30518311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":30518311},{"type":"v1","addr":"172.21.15.31:6841","nonce":30518311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/420362823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":420362823},{"type":"v1","addr":"172.21.15.169:6833","nonce":420362823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1943710438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1943710438},{"type":"v1","addr":"172.21.15.40:6833","nonce":1943710438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/4191344115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":4191344115},{"type":"v1","addr":"172.21.15.31:6837","nonce":4191344115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:13:30.975848+0000","modified":"2024-04-25T01:13:36.545846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24565,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6835/2916305801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2916305801},{"type":"v1","addr":"172.21.15.40:6835","nonce":2916305801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.169:6835/2253264784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2253264784},{"type":"v1","addr":"172.21.15.169:6835","nonce":2253264784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6837/2883308641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2883308641},{"type":"v1","addr":"172.21.15.169:6837","nonce":2883308641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/590109562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":590109562},{"type":"v1","addr":"172.21.15.40:6837","nonce":590109562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/4185710218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":4185710218},{"type":"v1","addr":"172.21.15.169:6839","nonce":4185710218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6839/927194004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":927194004},{"type":"v1","addr":"172.21.15.40:6839","nonce":927194004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24532,24565,24596]},"id":1}]} 2024-04-25T01:15:46.126 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 23 2024-04-25T01:15:46.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.143+0000 7f2ccfe00640 1 -- 172.21.15.31:0/2138828784 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f2cb40a2b90 msgr2=0x7f2cb40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:46.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.143+0000 7f2ccfe00640 1 --2- 172.21.15.31:0/2138828784 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f2cb40a2b90 0x7f2cb40a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8035040 tx=0x7f2cb8030570 comp rx=0 tx=0).stop 2024-04-25T01:15:46.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.143+0000 7f2ccfe00640 1 -- 172.21.15.31:0/2138828784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cd0076100 msgr2=0x7f2cd019e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:46.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.143+0000 7f2ccfe00640 1 --2- 172.21.15.31:0/2138828784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cd0076100 0x7f2cd019e0e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000e9f0 tx=0x7f2cc000edf0 comp rx=0 tx=0).stop 2024-04-25T01:15:46.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.143+0000 7f2ccfe00640 1 -- 172.21.15.31:0/2138828784 shutdown_connections 2024-04-25T01:15:46.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.143+0000 7f2ccfe00640 1 --2- 172.21.15.31:0/2138828784 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f2cb40a2b90 0x7f2cb40a5050 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:46.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.143+0000 7f2ccfe00640 1 --2- 172.21.15.31:0/2138828784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cd01a3680 0x7f2cd01a5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:46.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.143+0000 7f2ccfe00640 1 --2- 172.21.15.31:0/2138828784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cd0077ce0 0x7f2cd019e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:46.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.143+0000 7f2ccfe00640 1 --2- 172.21.15.31:0/2138828784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cd0076100 0x7f2cd019e0e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:46.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.143+0000 7f2ccfe00640 1 -- 172.21.15.31:0/2138828784 >> 172.21.15.31:0/2138828784 conn(0x7f2cd00fed50 msgr2=0x7f2cd01004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:46.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.143+0000 7f2ccfe00640 1 -- 172.21.15.31:0/2138828784 shutdown_connections 2024-04-25T01:15:46.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.143+0000 7f2ccfe00640 1 -- 172.21.15.31:0/2138828784 wait complete. 2024-04-25T01:15:46.149 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:15:46.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.377+0000 7fcb17200640 1 -- 172.21.15.31:0/454647566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb18108db0 msgr2=0x7fcb18109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:46.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.377+0000 7fcb17200640 1 --2- 172.21.15.31:0/454647566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb18108db0 0x7fcb18109230 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009900 tx=0x7fcb0002a410 comp rx=0 tx=0).stop 2024-04-25T01:15:46.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.377+0000 7fcb17200640 1 -- 172.21.15.31:0/454647566 shutdown_connections 2024-04-25T01:15:46.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.377+0000 7fcb17200640 1 --2- 172.21.15.31:0/454647566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb18109770 0x7fcb18110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:46.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.377+0000 7fcb17200640 1 --2- 172.21.15.31:0/454647566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb18108db0 0x7fcb18109230 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:46.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.377+0000 7fcb17200640 1 --2- 172.21.15.31:0/454647566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb18107bc0 0x7fcb18107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:46.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.377+0000 7fcb17200640 1 -- 172.21.15.31:0/454647566 >> 172.21.15.31:0/454647566 conn(0x7fcb18076b80 msgr2=0x7fcb18078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:46.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.377+0000 7fcb17200640 1 -- 172.21.15.31:0/454647566 shutdown_connections 2024-04-25T01:15:46.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.377+0000 7fcb17200640 1 -- 172.21.15.31:0/454647566 wait complete. 2024-04-25T01:15:46.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.378+0000 7fcb17200640 1 Processor -- start 2024-04-25T01:15:46.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.378+0000 7fcb17200640 1 -- start start 2024-04-25T01:15:46.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.378+0000 7fcb17200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb18107bc0 0x7fcb1819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:46.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.378+0000 7fcb17200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb18108db0 0x7fcb1819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:46.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.378+0000 7fcb17200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb18109770 0x7fcb181a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:46.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.378+0000 7fcb17200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcb18113890 con 0x7fcb18108db0 2024-04-25T01:15:46.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.378+0000 7fcb17200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcb18113710 con 0x7fcb18109770 2024-04-25T01:15:46.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.378+0000 7fcb17200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcb18113a10 con 0x7fcb18107bc0 2024-04-25T01:15:46.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.379+0000 7fcb0fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb18108db0 0x7fcb1819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:46.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.379+0000 7fcb15e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb18107bc0 0x7fcb1819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:46.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.379+0000 7fcb16800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb18109770 0x7fcb181a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:46.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.379+0000 7fcb15e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb18107bc0 0x7fcb1819ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:57644/0 (socket says 172.21.15.31:57644) 2024-04-25T01:15:46.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.379+0000 7fcb0fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb18108db0 0x7fcb1819e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58102/0 (socket says 172.21.15.31:58102) 2024-04-25T01:15:46.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.379+0000 7fcb15e00640 1 -- 172.21.15.31:0/591085330 learned_addr learned my addr 172.21.15.31:0/591085330 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:46.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.379+0000 7fcb16800640 1 --2- 172.21.15.31:0/591085330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb18109770 0x7fcb181a5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:48182/0 (socket says 172.21.15.31:48182) 2024-04-25T01:15:46.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.379+0000 7fcb15e00640 1 -- 172.21.15.31:0/591085330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb18109770 msgr2=0x7fcb181a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:46.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.379+0000 7fcb15e00640 1 --2- 172.21.15.31:0/591085330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb18109770 0x7fcb181a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:46.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.379+0000 7fcb15e00640 1 -- 172.21.15.31:0/591085330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb18108db0 msgr2=0x7fcb1819e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:46.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.379+0000 7fcb15e00640 1 --2- 172.21.15.31:0/591085330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb18108db0 0x7fcb1819e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:46.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.379+0000 7fcb15e00640 1 -- 172.21.15.31:0/591085330 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb181a5af0 con 0x7fcb18107bc0 2024-04-25T01:15:46.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.379+0000 7fcb16800640 1 --2- 172.21.15.31:0/591085330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb18109770 0x7fcb181a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:46.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.379+0000 7fcb15e00640 1 --2- 172.21.15.31:0/591085330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb18107bc0 0x7fcb1819ded0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800e9c0 tx=0x7fcb0800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:46.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.380+0000 7fcb0ea00640 1 -- 172.21.15.31:0/591085330 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0800cd80 con 0x7fcb18107bc0 2024-04-25T01:15:46.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.380+0000 7fcb17200640 1 -- 172.21.15.31:0/591085330 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb181a5de0 con 0x7fcb18107bc0 2024-04-25T01:15:46.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.380+0000 7fcb0ea00640 1 -- 172.21.15.31:0/591085330 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb080035f0 con 0x7fcb18107bc0 2024-04-25T01:15:46.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.380+0000 7fcb0ea00640 1 -- 172.21.15.31:0/591085330 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb08010690 con 0x7fcb18107bc0 2024-04-25T01:15:46.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.380+0000 7fcb17200640 1 -- 172.21.15.31:0/591085330 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb181a6240 con 0x7fcb18107bc0 2024-04-25T01:15:46.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.381+0000 7fcb17200640 1 -- 172.21.15.31:0/591085330 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb1810eee0 con 0x7fcb18107bc0 2024-04-25T01:15:46.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.383+0000 7fcb0ea00640 1 -- 172.21.15.31:0/591085330 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcb080037a0 con 0x7fcb18107bc0 2024-04-25T01:15:46.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.384+0000 7fcb0ea00640 1 --2- 172.21.15.31:0/591085330 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fcafc0a2b90 0x7fcafc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:46.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.385+0000 7fcb0ea00640 1 -- 172.21.15.31:0/591085330 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fcb080cf3a0 con 0x7fcb18107bc0 2024-04-25T01:15:46.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.385+0000 7fcb0fe00640 1 --2- 172.21.15.31:0/591085330 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fcafc0a2b90 0x7fcafc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:46.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.386+0000 7fcb0fe00640 1 --2- 172.21.15.31:0/591085330 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fcafc0a2b90 0x7fcafc0a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcb00031440 tx=0x7fcb000315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:46.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.388+0000 7fcb0ea00640 1 -- 172.21.15.31:0/591085330 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcb0809c2f0 con 0x7fcb18107bc0 2024-04-25T01:15:46.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:46 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1118821178' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:15:46.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:46 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2138828784' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:15:46.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.597+0000 7fcb17200640 1 -- 172.21.15.31:0/591085330 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb180623e0 con 0x7fcb18107bc0 2024-04-25T01:15:46.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.599+0000 7fcb0ea00640 1 -- 172.21.15.31:0/591085330 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22530 (secure 0 0 0) 0x7fcb0809bbf0 con 0x7fcb18107bc0 2024-04-25T01:15:46.585 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:15:46.585 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":89,"fsid":"95849b94-029f-11ef-bc93-c7b262605968","created":"2024-04-25T01:04:23.310856+0000","modified":"2024-04-25T01:14:33.793638+0000","last_up_change":"2024-04-25T01:12:29.393907+0000","last_in_change":"2024-04-25T01:12:08.978148+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:18.662391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:13:29.258771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:13:30.480996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa08e4c-fe6b-4116-bdf5-1adf5b465d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6803","nonce":788174091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6805","nonce":788174091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6809","nonce":788174091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6807","nonce":788174091}]},"public_addr":"172.21.15.31:6803/788174091","cluster_addr":"172.21.15.31:6805/788174091","heartbeat_back_addr":"172.21.15.31:6809/788174091","heartbeat_front_addr":"172.21.15.31:6807/788174091","state":["exists","up"]},{"osd":1,"uuid":"85339fe7-0dbc-4485-9334-2b039993c1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6801","nonce":3196394509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6803","nonce":3196394509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6807","nonce":3196394509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6805","nonce":3196394509}]},"public_addr":"172.21.15.40:6801/3196394509","cluster_addr":"172.21.15.40:6803/3196394509","heartbeat_back_addr":"172.21.15.40:6807/3196394509","heartbeat_front_addr":"172.21.15.40:6805/3196394509","state":["exists","up"]},{"osd":2,"uuid":"ab62ecb2-8aae-4987-9e2e-589e0f2fee5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6801","nonce":1955211197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6803","nonce":1955211197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6807","nonce":1955211197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6805","nonce":1955211197}]},"public_addr":"172.21.15.169:6801/1955211197","cluster_addr":"172.21.15.169:6803/1955211197","heartbeat_back_addr":"172.21.15.169:6807/1955211197","heartbeat_front_addr":"172.21.15.169:6805/1955211197","state":["exists","up"]},{"osd":3,"uuid":"8b5a7881-3401-495c-8ca4-c90685919d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6811","nonce":1228688643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6813","nonce":1228688643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6817","nonce":1228688643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6815","nonce":1228688643}]},"public_addr":"172.21.15.31:6811/1228688643","cluster_addr":"172.21.15.31:6813/1228688643","heartbeat_back_addr":"172.21.15.31:6817/1228688643","heartbeat_front_addr":"172.21.15.31:6815/1228688643","state":["exists","up"]},{"osd":4,"uuid":"56deb61d-a334-4cf2-8ac7-fbb6e6947092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6809","nonce":1913779091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6811","nonce":1913779091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6815","nonce":1913779091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6813","nonce":1913779091}]},"public_addr":"172.21.15.40:6809/1913779091","cluster_addr":"172.21.15.40:6811/1913779091","heartbeat_back_addr":"172.21.15.40:6815/1913779091","heartbeat_front_addr":"172.21.15.40:6813/1913779091","state":["exists","up"]},{"osd":5,"uuid":"8da55693-f9c7-49a7-99e0-165078f8001b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6809","nonce":2549650781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6811","nonce":2549650781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6815","nonce":2549650781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6813","nonce":2549650781}]},"public_addr":"172.21.15.169:6809/2549650781","cluster_addr":"172.21.15.169:6811/2549650781","heartbeat_back_addr":"172.21.15.169:6815/2549650781","heartbeat_front_addr":"172.21.15.169:6813/2549650781","state":["exists","up"]},{"osd":6,"uuid":"fc8350d2-1ad6-43cb-89f4-d336afc99ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6819","nonce":611507624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6821","nonce":611507624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6825","nonce":611507624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6823","nonce":611507624}]},"public_addr":"172.21.15.31:6819/611507624","cluster_addr":"172.21.15.31:6821/611507624","heartbeat_back_addr":"172.21.15.31:6825/611507624","heartbeat_front_addr":"172.21.15.31:6823/611507624","state":["exists","up"]},{"osd":7,"uuid":"3121b58b-8c94-40b5-9d48-f56552d7c39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6817","nonce":3776085849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6819","nonce":3776085849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6823","nonce":3776085849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6821","nonce":3776085849}]},"public_addr":"172.21.15.40:6817/3776085849","cluster_addr":"172.21.15.40:6819/3776085849","heartbeat_back_addr":"172.21.15.40:6823/3776085849","heartbeat_front_addr":"172.21.15.40:6821/3776085849","state":["exists","up"]},{"osd":8,"uuid":"f60407df-91f5-477a-9702-ddaef24faf82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6817","nonce":2736546122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6819","nonce":2736546122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6823","nonce":2736546122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6821","nonce":2736546122}]},"public_addr":"172.21.15.169:6817/2736546122","cluster_addr":"172.21.15.169:6819/2736546122","heartbeat_back_addr":"172.21.15.169:6823/2736546122","heartbeat_front_addr":"172.21.15.169:6821/2736546122","state":["exists","up"]},{"osd":9,"uuid":"7506a407-1960-49bc-9e38-7f2a8bf996ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6827","nonce":1187207880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6829","nonce":1187207880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6833","nonce":1187207880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6831","nonce":1187207880}]},"public_addr":"172.21.15.31:6827/1187207880","cluster_addr":"172.21.15.31:6829/1187207880","heartbeat_back_addr":"172.21.15.31:6833/1187207880","heartbeat_front_addr":"172.21.15.31:6831/1187207880","state":["exists","up"]},{"osd":10,"uuid":"831c48e2-b0d0-451f-959b-228e927542cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6825","nonce":3050148001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6827","nonce":3050148001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6831","nonce":3050148001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6829","nonce":3050148001}]},"public_addr":"172.21.15.40:6825/3050148001","cluster_addr":"172.21.15.40:6827/3050148001","heartbeat_back_addr":"172.21.15.40:6831/3050148001","heartbeat_front_addr":"172.21.15.40:6829/3050148001","state":["exists","up"]},{"osd":11,"uuid":"144fe0aa-f757-4b3c-90ad-99775e17d57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6825","nonce":262401944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6827","nonce":262401944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6831","nonce":262401944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6829","nonce":262401944}]},"public_addr":"172.21.15.169:6825/262401944","cluster_addr":"172.21.15.169:6827/262401944","heartbeat_back_addr":"172.21.15.169:6831/262401944","heartbeat_front_addr":"172.21.15.169:6829/262401944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:25.640995+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:51.036909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.635633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:48.467912+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:14.454971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:37.609315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:12.441807+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:38.239215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:02.230758+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:35.762553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:02.051405+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:26.569593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6801/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4004014066":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/1930101646":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/2638309002":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2238638505":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2902841144":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4190696323":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/480558521":"2024-04-26T01:04:57.149272+0000","172.21.15.31:6801/2638309002":"2024-04-26T01:04:57.149272+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:15:46.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.602+0000 7fcb17200640 1 -- 172.21.15.31:0/591085330 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fcafc0a2b90 msgr2=0x7fcafc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:46.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.602+0000 7fcb17200640 1 --2- 172.21.15.31:0/591085330 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fcafc0a2b90 0x7fcafc0a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcb00031440 tx=0x7fcb000315b0 comp rx=0 tx=0).stop 2024-04-25T01:15:46.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.602+0000 7fcb17200640 1 -- 172.21.15.31:0/591085330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb18107bc0 msgr2=0x7fcb1819ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:46.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.602+0000 7fcb17200640 1 --2- 172.21.15.31:0/591085330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb18107bc0 0x7fcb1819ded0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800e9c0 tx=0x7fcb0800ee80 comp rx=0 tx=0).stop 2024-04-25T01:15:46.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.602+0000 7fcb17200640 1 -- 172.21.15.31:0/591085330 shutdown_connections 2024-04-25T01:15:46.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.602+0000 7fcb17200640 1 --2- 172.21.15.31:0/591085330 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fcafc0a2b90 0x7fcafc0a5050 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:46.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.602+0000 7fcb17200640 1 --2- 172.21.15.31:0/591085330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb18109770 0x7fcb181a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:46.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.602+0000 7fcb17200640 1 --2- 172.21.15.31:0/591085330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb18108db0 0x7fcb1819e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:46.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.602+0000 7fcb17200640 1 --2- 172.21.15.31:0/591085330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb18107bc0 0x7fcb1819ded0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:46.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.602+0000 7fcb17200640 1 -- 172.21.15.31:0/591085330 >> 172.21.15.31:0/591085330 conn(0x7fcb18076b80 msgr2=0x7fcb1810c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:46.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.602+0000 7fcb17200640 1 -- 172.21.15.31:0/591085330 shutdown_connections 2024-04-25T01:15:46.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.602+0000 7fcb17200640 1 -- 172.21.15.31:0/591085330 wait complete. 2024-04-25T01:15:46.608 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:18.662391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:13:29.258771+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:13:30.480996+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:15:46.609 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:15:46.694 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:46 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1118821178' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:15:46.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:46 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2138828784' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:15:46.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:46 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1118821178' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:15:46.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:46 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2138828784' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:15:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.835+0000 7fb1ea400640 1 -- 172.21.15.31:0/2875807447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1e4105550 msgr2=0x7fb1e410bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.835+0000 7fb1ea400640 1 --2- 172.21.15.31:0/2875807447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1e4105550 0x7fb1e410bde0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009860 tx=0x7fb1dc02a340 comp rx=0 tx=0).stop 2024-04-25T01:15:46.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.835+0000 7fb1ea400640 1 -- 172.21.15.31:0/2875807447 shutdown_connections 2024-04-25T01:15:46.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.835+0000 7fb1ea400640 1 --2- 172.21.15.31:0/2875807447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1e4105550 0x7fb1e410bde0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:46.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.835+0000 7fb1ea400640 1 --2- 172.21.15.31:0/2875807447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1e4104b90 0x7fb1e4105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:46.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.835+0000 7fb1ea400640 1 --2- 172.21.15.31:0/2875807447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1e41039a0 0x7fb1e4103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:46.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.835+0000 7fb1ea400640 1 -- 172.21.15.31:0/2875807447 >> 172.21.15.31:0/2875807447 conn(0x7fb1e40fef30 msgr2=0x7fb1e4101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:46.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.836+0000 7fb1ea400640 1 -- 172.21.15.31:0/2875807447 shutdown_connections 2024-04-25T01:15:46.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.836+0000 7fb1ea400640 1 -- 172.21.15.31:0/2875807447 wait complete. 2024-04-25T01:15:46.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.837+0000 7fb1ea400640 1 Processor -- start 2024-04-25T01:15:46.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.837+0000 7fb1ea400640 1 -- start start 2024-04-25T01:15:46.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.837+0000 7fb1ea400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1e41039a0 0x7fb1e4199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:46.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.837+0000 7fb1ea400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1e4104b90 0x7fb1e419a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:46.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.837+0000 7fb1ea400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1e4105550 0x7fb1e41a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:46.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.837+0000 7fb1ea400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e410f620 con 0x7fb1e4104b90 2024-04-25T01:15:46.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.837+0000 7fb1ea400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e410f4a0 con 0x7fb1e41039a0 2024-04-25T01:15:46.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.837+0000 7fb1ea400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e410f7a0 con 0x7fb1e4105550 2024-04-25T01:15:46.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.838+0000 7fb1e3400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1e41039a0 0x7fb1e4199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:46.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.838+0000 7fb1e3400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1e41039a0 0x7fb1e4199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:37344/0 (socket says 172.21.15.31:37344) 2024-04-25T01:15:46.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.838+0000 7fb1e3400640 1 -- 172.21.15.31:0/2731757444 learned_addr learned my addr 172.21.15.31:0/2731757444 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:46.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.838+0000 7fb1e2a00640 1 --2- 172.21.15.31:0/2731757444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1e4104b90 0x7fb1e419a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:46.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.838+0000 7fb1e3e00640 1 --2- 172.21.15.31:0/2731757444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1e4105550 0x7fb1e41a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:46.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.838+0000 7fb1e3400640 1 -- 172.21.15.31:0/2731757444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1e4105550 msgr2=0x7fb1e41a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:46.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.838+0000 7fb1e3400640 1 --2- 172.21.15.31:0/2731757444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1e4105550 0x7fb1e41a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:46.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.838+0000 7fb1e3400640 1 -- 172.21.15.31:0/2731757444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1e4104b90 msgr2=0x7fb1e419a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:46.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.838+0000 7fb1e3400640 1 --2- 172.21.15.31:0/2731757444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1e4104b90 0x7fb1e419a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:46.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.838+0000 7fb1e3400640 1 -- 172.21.15.31:0/2731757444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e41a1860 con 0x7fb1e41039a0 2024-04-25T01:15:46.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.838+0000 7fb1e3400640 1 --2- 172.21.15.31:0/2731757444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1e41039a0 0x7fb1e4199c50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400c3a0 tx=0x7fb1d400c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:46.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.839+0000 7fb1dbe00640 1 -- 172.21.15.31:0/2731757444 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d401b070 con 0x7fb1e41039a0 2024-04-25T01:15:46.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.839+0000 7fb1dbe00640 1 -- 172.21.15.31:0/2731757444 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1d4003e50 con 0x7fb1e41039a0 2024-04-25T01:15:46.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.839+0000 7fb1dbe00640 1 -- 172.21.15.31:0/2731757444 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d400b890 con 0x7fb1e41039a0 2024-04-25T01:15:46.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.839+0000 7fb1ea400640 1 -- 172.21.15.31:0/2731757444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e41a1b50 con 0x7fb1e41039a0 2024-04-25T01:15:46.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.839+0000 7fb1ea400640 1 -- 172.21.15.31:0/2731757444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e41a2050 con 0x7fb1e41039a0 2024-04-25T01:15:46.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.840+0000 7fb1ea400640 1 -- 172.21.15.31:0/2731757444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1e410acc0 con 0x7fb1e41039a0 2024-04-25T01:15:46.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.842+0000 7fb1dbe00640 1 -- 172.21.15.31:0/2731757444 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb1d401e070 con 0x7fb1e41039a0 2024-04-25T01:15:46.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.843+0000 7fb1dbe00640 1 --2- 172.21.15.31:0/2731757444 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb1c80a2b90 0x7fb1c80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:46.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.844+0000 7fb1dbe00640 1 -- 172.21.15.31:0/2731757444 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fb1d40cf4f0 con 0x7fb1e41039a0 2024-04-25T01:15:46.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.844+0000 7fb1e2a00640 1 --2- 172.21.15.31:0/2731757444 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb1c80a2b90 0x7fb1c80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:46.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.844+0000 7fb1e2a00640 1 --2- 172.21.15.31:0/2731757444 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb1c80a2b90 0x7fb1c80a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00bbe0 tx=0x7fb1cc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:46.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:46.846+0000 7fb1dbe00640 1 -- 172.21.15.31:0/2731757444 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb1d409bdf0 con 0x7fb1e41039a0 2024-04-25T01:15:47.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.051+0000 7fb1ea400640 1 -- 172.21.15.31:0/2731757444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb1e402d320 con 0x7fb1e41039a0 2024-04-25T01:15:47.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.052+0000 7fb1dbe00640 1 -- 172.21.15.31:0/2731757444 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fb1d40a00f0 con 0x7fb1e41039a0 2024-04-25T01:15:47.037 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T01:15:47.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.054+0000 7fb1ea400640 1 -- 172.21.15.31:0/2731757444 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb1c80a2b90 msgr2=0x7fb1c80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:47.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.054+0000 7fb1ea400640 1 --2- 172.21.15.31:0/2731757444 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb1c80a2b90 0x7fb1c80a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00bbe0 tx=0x7fb1cc00b390 comp rx=0 tx=0).stop 2024-04-25T01:15:47.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.054+0000 7fb1ea400640 1 -- 172.21.15.31:0/2731757444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1e41039a0 msgr2=0x7fb1e4199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:47.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.054+0000 7fb1ea400640 1 --2- 172.21.15.31:0/2731757444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1e41039a0 0x7fb1e4199c50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400c3a0 tx=0x7fb1d400c860 comp rx=0 tx=0).stop 2024-04-25T01:15:47.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.055+0000 7fb1ea400640 1 -- 172.21.15.31:0/2731757444 shutdown_connections 2024-04-25T01:15:47.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.055+0000 7fb1ea400640 1 --2- 172.21.15.31:0/2731757444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1e4105550 0x7fb1e41a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:47.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.055+0000 7fb1ea400640 1 --2- 172.21.15.31:0/2731757444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1e4104b90 0x7fb1e419a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:47.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.055+0000 7fb1ea400640 1 --2- 172.21.15.31:0/2731757444 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb1c80a2b90 0x7fb1c80a5050 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:47.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.055+0000 7fb1ea400640 1 --2- 172.21.15.31:0/2731757444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1e41039a0 0x7fb1e4199c50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:47.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.055+0000 7fb1ea400640 1 -- 172.21.15.31:0/2731757444 >> 172.21.15.31:0/2731757444 conn(0x7fb1e40fef30 msgr2=0x7fb1e4107e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:47.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.055+0000 7fb1ea400640 1 -- 172.21.15.31:0/2731757444 shutdown_connections 2024-04-25T01:15:47.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.055+0000 7fb1ea400640 1 -- 172.21.15.31:0/2731757444 wait complete. 2024-04-25T01:15:47.062 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:15:47.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.301+0000 7f9746a00640 1 -- 172.21.15.31:0/2662631707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9740102840 msgr2=0x7f9740102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:47.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.301+0000 7f9746a00640 1 --2- 172.21.15.31:0/2662631707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9740102840 0x7f9740102cc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f972c009900 tx=0x7f972c02a3f0 comp rx=0 tx=0).stop 2024-04-25T01:15:47.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.302+0000 7f9746a00640 1 -- 172.21.15.31:0/2662631707 shutdown_connections 2024-04-25T01:15:47.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.302+0000 7f9746a00640 1 --2- 172.21.15.31:0/2662631707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9740103200 0x7f9740109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:47.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.302+0000 7f9746a00640 1 --2- 172.21.15.31:0/2662631707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9740102840 0x7f9740102cc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:47.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.302+0000 7f9746a00640 1 --2- 172.21.15.31:0/2662631707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9740101650 0x7f9740101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:47.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.302+0000 7f9746a00640 1 -- 172.21.15.31:0/2662631707 >> 172.21.15.31:0/2662631707 conn(0x7f97400fcbe0 msgr2=0x7f97400ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.302+0000 7f9746a00640 1 -- 172.21.15.31:0/2662631707 shutdown_connections 2024-04-25T01:15:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.302+0000 7f9746a00640 1 -- 172.21.15.31:0/2662631707 wait complete. 2024-04-25T01:15:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.303+0000 7f9746a00640 1 Processor -- start 2024-04-25T01:15:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.303+0000 7f9746a00640 1 -- start start 2024-04-25T01:15:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.303+0000 7f9746a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9740101650 0x7f9740199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.303+0000 7f9746a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9740102840 0x7f974019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.303+0000 7f9746a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9740103200 0x7f97401a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.303+0000 7f9746a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f974010d320 con 0x7f9740102840 2024-04-25T01:15:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.303+0000 7f9746a00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f974010d1a0 con 0x7f9740103200 2024-04-25T01:15:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.303+0000 7f9746a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f974010d4a0 con 0x7f9740101650 2024-04-25T01:15:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.304+0000 7f9744c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9740102840 0x7f974019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.304+0000 7f9745600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9740101650 0x7f9740199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:47.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.304+0000 7f9745600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9740101650 0x7f9740199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:39844/0 (socket says 172.21.15.31:39844) 2024-04-25T01:15:47.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.304+0000 7f9745600640 1 -- 172.21.15.31:0/640794288 learned_addr learned my addr 172.21.15.31:0/640794288 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:47.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.304+0000 7f9746000640 1 --2- 172.21.15.31:0/640794288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9740103200 0x7f97401a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:47.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.304+0000 7f9745600640 1 -- 172.21.15.31:0/640794288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9740103200 msgr2=0x7f97401a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:47.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.304+0000 7f9745600640 1 --2- 172.21.15.31:0/640794288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9740103200 0x7f97401a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:47.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.304+0000 7f9745600640 1 -- 172.21.15.31:0/640794288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9740102840 msgr2=0x7f974019a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:47.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.304+0000 7f9745600640 1 --2- 172.21.15.31:0/640794288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9740102840 0x7f974019a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:47.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.304+0000 7f9745600640 1 -- 172.21.15.31:0/640794288 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97401a1780 con 0x7f9740101650 2024-04-25T01:15:47.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.304+0000 7f9745600640 1 --2- 172.21.15.31:0/640794288 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9740101650 0x7f9740199b60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f973400e9c0 tx=0x7f973400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:47.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.305+0000 7f973e000640 1 -- 172.21.15.31:0/640794288 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973400cd80 con 0x7f9740101650 2024-04-25T01:15:47.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.305+0000 7f9746a00640 1 -- 172.21.15.31:0/640794288 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97401a1a70 con 0x7f9740101650 2024-04-25T01:15:47.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.305+0000 7f973e000640 1 -- 172.21.15.31:0/640794288 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97340035f0 con 0x7f9740101650 2024-04-25T01:15:47.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.305+0000 7f973e000640 1 -- 172.21.15.31:0/640794288 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9734010690 con 0x7f9740101650 2024-04-25T01:15:47.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.305+0000 7f9746a00640 1 -- 172.21.15.31:0/640794288 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97401a1f90 con 0x7f9740101650 2024-04-25T01:15:47.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.306+0000 7f9746a00640 1 -- 172.21.15.31:0/640794288 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97400688d0 con 0x7f9740101650 2024-04-25T01:15:47.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.308+0000 7f973e000640 1 -- 172.21.15.31:0/640794288 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f97340037a0 con 0x7f9740101650 2024-04-25T01:15:47.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.309+0000 7f973e000640 1 --2- 172.21.15.31:0/640794288 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f97280a2b90 0x7f97280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:47.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.309+0000 7f973e000640 1 -- 172.21.15.31:0/640794288 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f97340cf3a0 con 0x7f9740101650 2024-04-25T01:15:47.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.309+0000 7f9744c00640 1 --2- 172.21.15.31:0/640794288 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f97280a2b90 0x7f97280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:47.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.310+0000 7f9744c00640 1 --2- 172.21.15.31:0/640794288 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f97280a2b90 0x7f97280a5050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f972c031440 tx=0x7f972c0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:47.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.313+0000 7f973e000640 1 -- 172.21.15.31:0/640794288 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f973409c2f0 con 0x7f9740101650 2024-04-25T01:15:47.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:47 smithi031 ceph-mon[28053]: pgmap v386: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T01:15:47.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:47 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/591085330' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:15:47.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:47 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2731757444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:15:47.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.520+0000 7f9746a00640 1 -- 172.21.15.31:0/640794288 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f97401a2220 con 0x7f9740101650 2024-04-25T01:15:47.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.521+0000 7f973e000640 1 -- 172.21.15.31:0/640794288 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f973409bbf0 con 0x7f9740101650 2024-04-25T01:15:47.507 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-25T01:15:47.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.524+0000 7f9746a00640 1 -- 172.21.15.31:0/640794288 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f97280a2b90 msgr2=0x7f97280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:47.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.524+0000 7f9746a00640 1 --2- 172.21.15.31:0/640794288 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f97280a2b90 0x7f97280a5050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f972c031440 tx=0x7f972c0315b0 comp rx=0 tx=0).stop 2024-04-25T01:15:47.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.524+0000 7f9746a00640 1 -- 172.21.15.31:0/640794288 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9740101650 msgr2=0x7f9740199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:47.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.524+0000 7f9746a00640 1 --2- 172.21.15.31:0/640794288 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9740101650 0x7f9740199b60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f973400e9c0 tx=0x7f973400ee80 comp rx=0 tx=0).stop 2024-04-25T01:15:47.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.524+0000 7f9746a00640 1 -- 172.21.15.31:0/640794288 shutdown_connections 2024-04-25T01:15:47.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.524+0000 7f9746a00640 1 --2- 172.21.15.31:0/640794288 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f97280a2b90 0x7f97280a5050 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:47.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.524+0000 7f9746a00640 1 --2- 172.21.15.31:0/640794288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9740103200 0x7f97401a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:47.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.524+0000 7f9746a00640 1 --2- 172.21.15.31:0/640794288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9740102840 0x7f974019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:47.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.524+0000 7f9746a00640 1 --2- 172.21.15.31:0/640794288 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9740101650 0x7f9740199b60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:47.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.524+0000 7f9746a00640 1 -- 172.21.15.31:0/640794288 >> 172.21.15.31:0/640794288 conn(0x7f97400fcbe0 msgr2=0x7f9740073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:47.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.525+0000 7f9746a00640 1 -- 172.21.15.31:0/640794288 shutdown_connections 2024-04-25T01:15:47.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.525+0000 7f9746a00640 1 -- 172.21.15.31:0/640794288 wait complete. 2024-04-25T01:15:47.530 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:15:47.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:47 smithi169 ceph-mon[32759]: pgmap v386: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T01:15:47.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:47 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/591085330' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:15:47.694 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:47 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2731757444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:15:47.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:47 smithi040 ceph-mon[32518]: pgmap v386: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T01:15:47.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:47 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/591085330' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:15:47.697 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:47 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2731757444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:15:47.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.764+0000 7f3ab8e00640 1 -- 172.21.15.31:0/2613023960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab41039a0 msgr2=0x7f3ab4103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:47.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.764+0000 7f3ab8e00640 1 --2- 172.21.15.31:0/2613023960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab41039a0 0x7f3ab4103dc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009900 tx=0x7f3a9c02a410 comp rx=0 tx=0).stop 2024-04-25T01:15:47.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.764+0000 7f3ab8e00640 1 -- 172.21.15.31:0/2613023960 shutdown_connections 2024-04-25T01:15:47.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.764+0000 7f3ab8e00640 1 --2- 172.21.15.31:0/2613023960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ab4105550 0x7f3ab410bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:47.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.764+0000 7f3ab8e00640 1 --2- 172.21.15.31:0/2613023960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ab4104b90 0x7f3ab4105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:47.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.764+0000 7f3ab8e00640 1 --2- 172.21.15.31:0/2613023960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab41039a0 0x7f3ab4103dc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:47.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.764+0000 7f3ab8e00640 1 -- 172.21.15.31:0/2613023960 >> 172.21.15.31:0/2613023960 conn(0x7f3ab40feef0 msgr2=0x7f3ab4101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:47.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.764+0000 7f3ab8e00640 1 -- 172.21.15.31:0/2613023960 shutdown_connections 2024-04-25T01:15:47.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.764+0000 7f3ab8e00640 1 -- 172.21.15.31:0/2613023960 wait complete. 2024-04-25T01:15:47.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.765+0000 7f3ab8e00640 1 Processor -- start 2024-04-25T01:15:47.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.765+0000 7f3ab8e00640 1 -- start start 2024-04-25T01:15:47.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.765+0000 7f3ab8e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ab41039a0 0x7f3ab4199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:47.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.765+0000 7f3ab8e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ab4104b90 0x7f3ab419a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:47.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.765+0000 7f3ab8e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab4105550 0x7f3ab41a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:47.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.765+0000 7f3ab8e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab410f660 con 0x7f3ab41039a0 2024-04-25T01:15:47.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.765+0000 7f3ab8e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab410f4e0 con 0x7f3ab4105550 2024-04-25T01:15:47.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.765+0000 7f3ab8e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab410f7e0 con 0x7f3ab4104b90 2024-04-25T01:15:47.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.766+0000 7f3ab1e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ab41039a0 0x7f3ab4199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:47.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.766+0000 7f3ab1e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ab41039a0 0x7f3ab4199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48654/0 (socket says 172.21.15.31:48654) 2024-04-25T01:15:47.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.766+0000 7f3ab1e00640 1 -- 172.21.15.31:0/2715285086 learned_addr learned my addr 172.21.15.31:0/2715285086 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:47.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.766+0000 7f3ab1400640 1 --2- 172.21.15.31:0/2715285086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ab4104b90 0x7f3ab419a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:47.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.766+0000 7f3ab2800640 1 --2- 172.21.15.31:0/2715285086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab4105550 0x7f3ab41a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:47.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.766+0000 7f3ab1e00640 1 -- 172.21.15.31:0/2715285086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ab4104b90 msgr2=0x7f3ab419a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:47.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.766+0000 7f3ab1e00640 1 --2- 172.21.15.31:0/2715285086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ab4104b90 0x7f3ab419a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:47.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.766+0000 7f3ab1e00640 1 -- 172.21.15.31:0/2715285086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab4105550 msgr2=0x7f3ab41a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:47.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.766+0000 7f3ab1e00640 1 --2- 172.21.15.31:0/2715285086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab4105550 0x7f3ab41a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:47.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.766+0000 7f3ab1e00640 1 -- 172.21.15.31:0/2715285086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab41a1930 con 0x7f3ab41039a0 2024-04-25T01:15:47.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.767+0000 7f3ab1e00640 1 --2- 172.21.15.31:0/2715285086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ab41039a0 0x7f3ab4199d20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c031440 tx=0x7f3a9c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:47.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.767+0000 7f3aaaa00640 1 -- 172.21.15.31:0/2715285086 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c03f030 con 0x7f3ab41039a0 2024-04-25T01:15:47.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.767+0000 7f3ab8e00640 1 -- 172.21.15.31:0/2715285086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab41a1bc0 con 0x7f3ab41039a0 2024-04-25T01:15:47.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.767+0000 7f3aaaa00640 1 -- 172.21.15.31:0/2715285086 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a9c003ce0 con 0x7f3ab41039a0 2024-04-25T01:15:47.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.767+0000 7f3aaaa00640 1 -- 172.21.15.31:0/2715285086 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c035070 con 0x7f3ab41039a0 2024-04-25T01:15:47.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.767+0000 7f3ab8e00640 1 -- 172.21.15.31:0/2715285086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab41a20b0 con 0x7f3ab41039a0 2024-04-25T01:15:47.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.768+0000 7f3ab8e00640 1 -- 172.21.15.31:0/2715285086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ab410acc0 con 0x7f3ab41039a0 2024-04-25T01:15:47.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.770+0000 7f3aaaa00640 1 -- 172.21.15.31:0/2715285086 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3a9c042030 con 0x7f3ab41039a0 2024-04-25T01:15:47.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.771+0000 7f3aaaa00640 1 --2- 172.21.15.31:0/2715285086 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f3a980a2b90 0x7f3a980a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:47.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.772+0000 7f3aaaa00640 1 -- 172.21.15.31:0/2715285086 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f3a9c0ef640 con 0x7f3ab41039a0 2024-04-25T01:15:47.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.772+0000 7f3ab1400640 1 --2- 172.21.15.31:0/2715285086 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f3a980a2b90 0x7f3a980a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:47.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.773+0000 7f3ab1400640 1 --2- 172.21.15.31:0/2715285086 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f3a980a2b90 0x7f3a980a5050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4005fd0 tx=0x7f3aa4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:47.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.777+0000 7f3aaaa00640 1 -- 172.21.15.31:0/2715285086 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3a9c0bbf40 con 0x7f3ab41039a0 2024-04-25T01:15:47.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.983+0000 7f3ab8e00640 1 -- 172.21.15.31:0/2715285086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3ab4062340 con 0x7f3ab41039a0 2024-04-25T01:15:47.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.984+0000 7f3aaaa00640 1 -- 172.21.15.31:0/2715285086 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f3a9c0c0240 con 0x7f3ab41039a0 2024-04-25T01:15:47.969 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-04-25T01:15:47.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.987+0000 7f3ab8e00640 1 -- 172.21.15.31:0/2715285086 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f3a980a2b90 msgr2=0x7f3a980a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:47.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.987+0000 7f3ab8e00640 1 --2- 172.21.15.31:0/2715285086 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f3a980a2b90 0x7f3a980a5050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4005fd0 tx=0x7f3aa4009500 comp rx=0 tx=0).stop 2024-04-25T01:15:47.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.987+0000 7f3ab8e00640 1 -- 172.21.15.31:0/2715285086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ab41039a0 msgr2=0x7f3ab4199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:47.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.987+0000 7f3ab8e00640 1 --2- 172.21.15.31:0/2715285086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ab41039a0 0x7f3ab4199d20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c031440 tx=0x7f3a9c032ce0 comp rx=0 tx=0).stop 2024-04-25T01:15:47.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.987+0000 7f3ab8e00640 1 -- 172.21.15.31:0/2715285086 shutdown_connections 2024-04-25T01:15:47.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.987+0000 7f3ab8e00640 1 --2- 172.21.15.31:0/2715285086 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f3a980a2b90 0x7f3a980a5050 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:47.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.987+0000 7f3ab8e00640 1 --2- 172.21.15.31:0/2715285086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab4105550 0x7f3ab41a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:47.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.987+0000 7f3ab8e00640 1 --2- 172.21.15.31:0/2715285086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ab4104b90 0x7f3ab419a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:47.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.987+0000 7f3ab8e00640 1 --2- 172.21.15.31:0/2715285086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ab41039a0 0x7f3ab4199d20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:47.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.987+0000 7f3ab8e00640 1 -- 172.21.15.31:0/2715285086 >> 172.21.15.31:0/2715285086 conn(0x7f3ab40feef0 msgr2=0x7f3ab4107e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:47.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.987+0000 7f3ab8e00640 1 -- 172.21.15.31:0/2715285086 shutdown_connections 2024-04-25T01:15:47.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:47.987+0000 7f3ab8e00640 1 -- 172.21.15.31:0/2715285086 wait complete. 2024-04-25T01:15:47.991 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:15:48.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.221+0000 7f5f62200640 1 -- 172.21.15.31:0/2414315215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c1015e0 msgr2=0x7f5f5c101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:48.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.221+0000 7f5f62200640 1 --2- 172.21.15.31:0/2414315215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c1015e0 0x7f5f5c101a00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c009900 tx=0x7f5f4c02a3f0 comp rx=0 tx=0).stop 2024-04-25T01:15:48.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.221+0000 7f5f62200640 1 -- 172.21.15.31:0/2414315215 shutdown_connections 2024-04-25T01:15:48.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.221+0000 7f5f62200640 1 --2- 172.21.15.31:0/2414315215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f5c103190 0x7f5f5c109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:48.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.221+0000 7f5f62200640 1 --2- 172.21.15.31:0/2414315215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f5c1027d0 0x7f5f5c102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:48.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.221+0000 7f5f62200640 1 --2- 172.21.15.31:0/2414315215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c1015e0 0x7f5f5c101a00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:48.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.221+0000 7f5f62200640 1 -- 172.21.15.31:0/2414315215 >> 172.21.15.31:0/2414315215 conn(0x7f5f5c0fcb50 msgr2=0x7f5f5c0fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:48.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.221+0000 7f5f62200640 1 -- 172.21.15.31:0/2414315215 shutdown_connections 2024-04-25T01:15:48.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.221+0000 7f5f62200640 1 -- 172.21.15.31:0/2414315215 wait complete. 2024-04-25T01:15:48.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.222+0000 7f5f62200640 1 Processor -- start 2024-04-25T01:15:48.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.222+0000 7f5f62200640 1 -- start start 2024-04-25T01:15:48.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.222+0000 7f5f62200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f5c1015e0 0x7f5f5c199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:48.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.222+0000 7f5f62200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c1027d0 0x7f5f5c19a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:48.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.222+0000 7f5f62200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f5c103190 0x7f5f5c1a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:48.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.222+0000 7f5f62200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c10d2b0 con 0x7f5f5c1027d0 2024-04-25T01:15:48.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.222+0000 7f5f62200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c10d130 con 0x7f5f5c1015e0 2024-04-25T01:15:48.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.222+0000 7f5f62200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c10d430 con 0x7f5f5c103190 2024-04-25T01:15:48.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.223+0000 7f5f60e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f5c1015e0 0x7f5f5c199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:48.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.223+0000 7f5f60e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f5c1015e0 0x7f5f5c199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:37400/0 (socket says 172.21.15.31:37400) 2024-04-25T01:15:48.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.223+0000 7f5f60e00640 1 -- 172.21.15.31:0/1382184806 learned_addr learned my addr 172.21.15.31:0/1382184806 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:48.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.223+0000 7f5f5be00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c1027d0 0x7f5f5c19a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:48.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.223+0000 7f5f61800640 1 --2- 172.21.15.31:0/1382184806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f5c103190 0x7f5f5c1a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:48.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.223+0000 7f5f60e00640 1 -- 172.21.15.31:0/1382184806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f5c103190 msgr2=0x7f5f5c1a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:48.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.223+0000 7f5f60e00640 1 --2- 172.21.15.31:0/1382184806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f5c103190 0x7f5f5c1a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:48.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.223+0000 7f5f60e00640 1 -- 172.21.15.31:0/1382184806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c1027d0 msgr2=0x7f5f5c19a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:48.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.223+0000 7f5f60e00640 1 --2- 172.21.15.31:0/1382184806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c1027d0 0x7f5f5c19a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:48.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.223+0000 7f5f60e00640 1 -- 172.21.15.31:0/1382184806 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f5c1a1760 con 0x7f5f5c1015e0 2024-04-25T01:15:48.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.223+0000 7f5f61800640 1 --2- 172.21.15.31:0/1382184806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f5c103190 0x7f5f5c1a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:15:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.223+0000 7f5f60e00640 1 --2- 172.21.15.31:0/1382184806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f5c1015e0 0x7f5f5c199b90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c031440 tx=0x7f5f4c032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.224+0000 7f5f59600640 1 -- 172.21.15.31:0/1382184806 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c03f030 con 0x7f5f5c1015e0 2024-04-25T01:15:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.224+0000 7f5f59600640 1 -- 172.21.15.31:0/1382184806 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f4c003ce0 con 0x7f5f5c1015e0 2024-04-25T01:15:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.224+0000 7f5f62200640 1 -- 172.21.15.31:0/1382184806 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f5c1a19f0 con 0x7f5f5c1015e0 2024-04-25T01:15:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.224+0000 7f5f59600640 1 -- 172.21.15.31:0/1382184806 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c035070 con 0x7f5f5c1015e0 2024-04-25T01:15:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.224+0000 7f5f62200640 1 -- 172.21.15.31:0/1382184806 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f5c1a1ee0 con 0x7f5f5c1015e0 2024-04-25T01:15:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.225+0000 7f5f62200640 1 -- 172.21.15.31:0/1382184806 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f5c1a2110 con 0x7f5f5c1015e0 2024-04-25T01:15:48.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.227+0000 7f5f59600640 1 -- 172.21.15.31:0/1382184806 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5f4c042030 con 0x7f5f5c1015e0 2024-04-25T01:15:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.229+0000 7f5f59600640 1 --2- 172.21.15.31:0/1382184806 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5f400a2b90 0x7f5f400a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.229+0000 7f5f59600640 1 -- 172.21.15.31:0/1382184806 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f5f4c0ef560 con 0x7f5f5c1015e0 2024-04-25T01:15:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.229+0000 7f5f5be00640 1 --2- 172.21.15.31:0/1382184806 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5f400a2b90 0x7f5f400a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:48.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.230+0000 7f5f5be00640 1 --2- 172.21.15.31:0/1382184806 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5f400a2b90 0x7f5f400a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c19b050 tx=0x7f5f44005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:48.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.233+0000 7f5f59600640 1 -- 172.21.15.31:0/1382184806 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5f4c0bc4b0 con 0x7f5f5c1015e0 2024-04-25T01:15:48.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.478+0000 7f5f62200640 1 -- 172.21.15.31:0/1382184806 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f5c1a22e0 con 0x7f5f5c1015e0 2024-04-25T01:15:48.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.480+0000 7f5f59600640 1 -- 172.21.15.31:0/1382184806 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11854 (secure 0 0 0) 0x7f5f4c0bbdb0 con 0x7f5f5c1015e0 2024-04-25T01:15:48.464 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:15:48.465 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/498218714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":498218714},{"type":"v1","addr":"172.21.15.31:6835","nonce":498218714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2647199475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2647199475},{"type":"v1","addr":"172.21.15.31:6839","nonce":2647199475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/30518311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":30518311},{"type":"v1","addr":"172.21.15.31:6841","nonce":30518311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/420362823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":420362823},{"type":"v1","addr":"172.21.15.169:6833","nonce":420362823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1943710438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1943710438},{"type":"v1","addr":"172.21.15.40:6833","nonce":1943710438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/4191344115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":4191344115},{"type":"v1","addr":"172.21.15.31:6837","nonce":4191344115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:13:30.975848+0000","modified":"2024-04-25T01:13:36.545846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24565,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6835/2916305801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2916305801},{"type":"v1","addr":"172.21.15.40:6835","nonce":2916305801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.169:6835/2253264784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2253264784},{"type":"v1","addr":"172.21.15.169:6835","nonce":2253264784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6837/2883308641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2883308641},{"type":"v1","addr":"172.21.15.169:6837","nonce":2883308641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/590109562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":590109562},{"type":"v1","addr":"172.21.15.40:6837","nonce":590109562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/4185710218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":4185710218},{"type":"v1","addr":"172.21.15.169:6839","nonce":4185710218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6839/927194004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":927194004},{"type":"v1","addr":"172.21.15.40:6839","nonce":927194004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24532,24565,24596]},"id":1}]} 2024-04-25T01:15:48.465 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 23 2024-04-25T01:15:48.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.482+0000 7f5f62200640 1 -- 172.21.15.31:0/1382184806 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5f400a2b90 msgr2=0x7f5f400a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:48.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.482+0000 7f5f62200640 1 --2- 172.21.15.31:0/1382184806 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5f400a2b90 0x7f5f400a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c19b050 tx=0x7f5f44005f90 comp rx=0 tx=0).stop 2024-04-25T01:15:48.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.483+0000 7f5f62200640 1 -- 172.21.15.31:0/1382184806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f5c1015e0 msgr2=0x7f5f5c199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:48.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.483+0000 7f5f62200640 1 --2- 172.21.15.31:0/1382184806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f5c1015e0 0x7f5f5c199b90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c031440 tx=0x7f5f4c032ce0 comp rx=0 tx=0).stop 2024-04-25T01:15:48.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.483+0000 7f5f62200640 1 -- 172.21.15.31:0/1382184806 shutdown_connections 2024-04-25T01:15:48.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.483+0000 7f5f62200640 1 --2- 172.21.15.31:0/1382184806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f5c103190 0x7f5f5c1a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:48.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.483+0000 7f5f62200640 1 --2- 172.21.15.31:0/1382184806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c1027d0 0x7f5f5c19a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:48.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.483+0000 7f5f62200640 1 --2- 172.21.15.31:0/1382184806 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5f400a2b90 0x7f5f400a5050 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:48.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.483+0000 7f5f62200640 1 --2- 172.21.15.31:0/1382184806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f5c1015e0 0x7f5f5c199b90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:48.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.483+0000 7f5f62200640 1 -- 172.21.15.31:0/1382184806 >> 172.21.15.31:0/1382184806 conn(0x7f5f5c0fcb50 msgr2=0x7f5f5c073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:48.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.483+0000 7f5f62200640 1 -- 172.21.15.31:0/1382184806 shutdown_connections 2024-04-25T01:15:48.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.483+0000 7f5f62200640 1 -- 172.21.15.31:0/1382184806 wait complete. 2024-04-25T01:15:48.486 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:15:48.555 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:48 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/640794288' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:15:48.555 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:48 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2715285086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:15:48.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.708+0000 7f58b2800640 1 -- 172.21.15.31:0/596047533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58ac102980 msgr2=0x7f58ac102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:48.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.708+0000 7f58b2800640 1 --2- 172.21.15.31:0/596047533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58ac102980 0x7f58ac102e00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5894009ca0 tx=0x7f589402a8c0 comp rx=0 tx=0).stop 2024-04-25T01:15:48.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.708+0000 7f58b2800640 1 -- 172.21.15.31:0/596047533 shutdown_connections 2024-04-25T01:15:48.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.708+0000 7f58b2800640 1 --2- 172.21.15.31:0/596047533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58ac103340 0x7f58ac109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:48.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.708+0000 7f58b2800640 1 --2- 172.21.15.31:0/596047533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58ac102980 0x7f58ac102e00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:48.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.708+0000 7f58b2800640 1 --2- 172.21.15.31:0/596047533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58ac101790 0x7f58ac101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:48.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.708+0000 7f58b2800640 1 -- 172.21.15.31:0/596047533 >> 172.21.15.31:0/596047533 conn(0x7f58ac0fcd20 msgr2=0x7f58ac0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:48.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.708+0000 7f58b2800640 1 -- 172.21.15.31:0/596047533 shutdown_connections 2024-04-25T01:15:48.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.708+0000 7f58b2800640 1 -- 172.21.15.31:0/596047533 wait complete. 2024-04-25T01:15:48.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:48 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/640794288' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:15:48.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:48 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2715285086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:15:48.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.709+0000 7f58b2800640 1 Processor -- start 2024-04-25T01:15:48.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.709+0000 7f58b2800640 1 -- start start 2024-04-25T01:15:48.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.710+0000 7f58b2800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58ac101790 0x7f58ac0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:48.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.710+0000 7f58b2800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58ac102980 0x7f58ac073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:48.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.710+0000 7f58b2800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58ac103340 0x7f58ac074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:48.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.710+0000 7f58b2800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac10d460 con 0x7f58ac103340 2024-04-25T01:15:48.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.710+0000 7f58b2800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac10d2e0 con 0x7f58ac101790 2024-04-25T01:15:48.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.710+0000 7f58b2800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac10d5e0 con 0x7f58ac102980 2024-04-25T01:15:48.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.710+0000 7f58abe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58ac103340 0x7f58ac074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:48.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.710+0000 7f58aaa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58ac102980 0x7f58ac073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:48.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.710+0000 7f58abe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58ac103340 0x7f58ac074070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48704/0 (socket says 172.21.15.31:48704) 2024-04-25T01:15:48.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.710+0000 7f58abe00640 1 -- 172.21.15.31:0/2038399498 learned_addr learned my addr 172.21.15.31:0/2038399498 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:48.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.710+0000 7f58ab400640 1 --2- 172.21.15.31:0/2038399498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58ac101790 0x7f58ac0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:48.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.710+0000 7f58abe00640 1 -- 172.21.15.31:0/2038399498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58ac102980 msgr2=0x7f58ac073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:48.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.710+0000 7f58abe00640 1 --2- 172.21.15.31:0/2038399498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58ac102980 0x7f58ac073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:48.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.710+0000 7f58abe00640 1 -- 172.21.15.31:0/2038399498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58ac101790 msgr2=0x7f58ac0754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:48.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.710+0000 7f58abe00640 1 --2- 172.21.15.31:0/2038399498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58ac101790 0x7f58ac0754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:48.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.710+0000 7f58abe00640 1 -- 172.21.15.31:0/2038399498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ac074900 con 0x7f58ac103340 2024-04-25T01:15:48.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.711+0000 7f58abe00640 1 --2- 172.21.15.31:0/2038399498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58ac103340 0x7f58ac074070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f58a400e970 tx=0x7f58a400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:48.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.711+0000 7f58a3e00640 1 -- 172.21.15.31:0/2038399498 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a400ccc0 con 0x7f58ac103340 2024-04-25T01:15:48.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.711+0000 7f58b2800640 1 -- 172.21.15.31:0/2038399498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58ac1a1b90 con 0x7f58ac103340 2024-04-25T01:15:48.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.711+0000 7f58a3e00640 1 -- 172.21.15.31:0/2038399498 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58a400ce60 con 0x7f58ac103340 2024-04-25T01:15:48.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.711+0000 7f58a3e00640 1 -- 172.21.15.31:0/2038399498 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4010450 con 0x7f58ac103340 2024-04-25T01:15:48.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.711+0000 7f58b2800640 1 -- 172.21.15.31:0/2038399498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58ac1a20b0 con 0x7f58ac103340 2024-04-25T01:15:48.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.712+0000 7f58b2800640 1 -- 172.21.15.31:0/2038399498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58ac0689b0 con 0x7f58ac103340 2024-04-25T01:15:48.701 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:48 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/640794288' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:15:48.702 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:48 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2715285086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:15:48.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.714+0000 7f58a3e00640 1 -- 172.21.15.31:0/2038399498 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f58a4005020 con 0x7f58ac103340 2024-04-25T01:15:48.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.716+0000 7f58a3e00640 1 --2- 172.21.15.31:0/2038399498 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f588c0a2b90 0x7f588c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:48.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.716+0000 7f58a3e00640 1 -- 172.21.15.31:0/2038399498 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f58a40cf4a0 con 0x7f58ac103340 2024-04-25T01:15:48.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.716+0000 7f58ab400640 1 --2- 172.21.15.31:0/2038399498 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f588c0a2b90 0x7f588c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:48.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.717+0000 7f58ab400640 1 --2- 172.21.15.31:0/2038399498 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f588c0a2b90 0x7f588c0a5050 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f58ac1027e0 tx=0x7f589c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:48.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.720+0000 7f58a3e00640 1 -- 172.21.15.31:0/2038399498 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f58a409c3f0 con 0x7f58ac103340 2024-04-25T01:15:48.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.926+0000 7f58b2800640 1 -- 172.21.15.31:0/2038399498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58ac062400 con 0x7f58ac103340 2024-04-25T01:15:48.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.929+0000 7f58a3e00640 1 -- 172.21.15.31:0/2038399498 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22530 (secure 0 0 0) 0x7f58a409bcf0 con 0x7f58ac103340 2024-04-25T01:15:48.915 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:15:48.915 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":89,"fsid":"95849b94-029f-11ef-bc93-c7b262605968","created":"2024-04-25T01:04:23.310856+0000","modified":"2024-04-25T01:14:33.793638+0000","last_up_change":"2024-04-25T01:12:29.393907+0000","last_in_change":"2024-04-25T01:12:08.978148+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:18.662391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:13:29.258771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:13:30.480996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa08e4c-fe6b-4116-bdf5-1adf5b465d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6803","nonce":788174091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6805","nonce":788174091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6809","nonce":788174091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6807","nonce":788174091}]},"public_addr":"172.21.15.31:6803/788174091","cluster_addr":"172.21.15.31:6805/788174091","heartbeat_back_addr":"172.21.15.31:6809/788174091","heartbeat_front_addr":"172.21.15.31:6807/788174091","state":["exists","up"]},{"osd":1,"uuid":"85339fe7-0dbc-4485-9334-2b039993c1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6801","nonce":3196394509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6803","nonce":3196394509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6807","nonce":3196394509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6805","nonce":3196394509}]},"public_addr":"172.21.15.40:6801/3196394509","cluster_addr":"172.21.15.40:6803/3196394509","heartbeat_back_addr":"172.21.15.40:6807/3196394509","heartbeat_front_addr":"172.21.15.40:6805/3196394509","state":["exists","up"]},{"osd":2,"uuid":"ab62ecb2-8aae-4987-9e2e-589e0f2fee5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6801","nonce":1955211197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6803","nonce":1955211197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6807","nonce":1955211197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6805","nonce":1955211197}]},"public_addr":"172.21.15.169:6801/1955211197","cluster_addr":"172.21.15.169:6803/1955211197","heartbeat_back_addr":"172.21.15.169:6807/1955211197","heartbeat_front_addr":"172.21.15.169:6805/1955211197","state":["exists","up"]},{"osd":3,"uuid":"8b5a7881-3401-495c-8ca4-c90685919d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6811","nonce":1228688643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6813","nonce":1228688643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6817","nonce":1228688643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6815","nonce":1228688643}]},"public_addr":"172.21.15.31:6811/1228688643","cluster_addr":"172.21.15.31:6813/1228688643","heartbeat_back_addr":"172.21.15.31:6817/1228688643","heartbeat_front_addr":"172.21.15.31:6815/1228688643","state":["exists","up"]},{"osd":4,"uuid":"56deb61d-a334-4cf2-8ac7-fbb6e6947092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6809","nonce":1913779091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6811","nonce":1913779091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6815","nonce":1913779091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6813","nonce":1913779091}]},"public_addr":"172.21.15.40:6809/1913779091","cluster_addr":"172.21.15.40:6811/1913779091","heartbeat_back_addr":"172.21.15.40:6815/1913779091","heartbeat_front_addr":"172.21.15.40:6813/1913779091","state":["exists","up"]},{"osd":5,"uuid":"8da55693-f9c7-49a7-99e0-165078f8001b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6809","nonce":2549650781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6811","nonce":2549650781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6815","nonce":2549650781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6813","nonce":2549650781}]},"public_addr":"172.21.15.169:6809/2549650781","cluster_addr":"172.21.15.169:6811/2549650781","heartbeat_back_addr":"172.21.15.169:6815/2549650781","heartbeat_front_addr":"172.21.15.169:6813/2549650781","state":["exists","up"]},{"osd":6,"uuid":"fc8350d2-1ad6-43cb-89f4-d336afc99ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6819","nonce":611507624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6821","nonce":611507624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6825","nonce":611507624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6823","nonce":611507624}]},"public_addr":"172.21.15.31:6819/611507624","cluster_addr":"172.21.15.31:6821/611507624","heartbeat_back_addr":"172.21.15.31:6825/611507624","heartbeat_front_addr":"172.21.15.31:6823/611507624","state":["exists","up"]},{"osd":7,"uuid":"3121b58b-8c94-40b5-9d48-f56552d7c39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6817","nonce":3776085849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6819","nonce":3776085849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6823","nonce":3776085849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6821","nonce":3776085849}]},"public_addr":"172.21.15.40:6817/3776085849","cluster_addr":"172.21.15.40:6819/3776085849","heartbeat_back_addr":"172.21.15.40:6823/3776085849","heartbeat_front_addr":"172.21.15.40:6821/3776085849","state":["exists","up"]},{"osd":8,"uuid":"f60407df-91f5-477a-9702-ddaef24faf82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6817","nonce":2736546122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6819","nonce":2736546122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6823","nonce":2736546122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6821","nonce":2736546122}]},"public_addr":"172.21.15.169:6817/2736546122","cluster_addr":"172.21.15.169:6819/2736546122","heartbeat_back_addr":"172.21.15.169:6823/2736546122","heartbeat_front_addr":"172.21.15.169:6821/2736546122","state":["exists","up"]},{"osd":9,"uuid":"7506a407-1960-49bc-9e38-7f2a8bf996ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6827","nonce":1187207880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6829","nonce":1187207880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6833","nonce":1187207880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6831","nonce":1187207880}]},"public_addr":"172.21.15.31:6827/1187207880","cluster_addr":"172.21.15.31:6829/1187207880","heartbeat_back_addr":"172.21.15.31:6833/1187207880","heartbeat_front_addr":"172.21.15.31:6831/1187207880","state":["exists","up"]},{"osd":10,"uuid":"831c48e2-b0d0-451f-959b-228e927542cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6825","nonce":3050148001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6827","nonce":3050148001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6831","nonce":3050148001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6829","nonce":3050148001}]},"public_addr":"172.21.15.40:6825/3050148001","cluster_addr":"172.21.15.40:6827/3050148001","heartbeat_back_addr":"172.21.15.40:6831/3050148001","heartbeat_front_addr":"172.21.15.40:6829/3050148001","state":["exists","up"]},{"osd":11,"uuid":"144fe0aa-f757-4b3c-90ad-99775e17d57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6825","nonce":262401944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6827","nonce":262401944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6831","nonce":262401944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6829","nonce":262401944}]},"public_addr":"172.21.15.169:6825/262401944","cluster_addr":"172.21.15.169:6827/262401944","heartbeat_back_addr":"172.21.15.169:6831/262401944","heartbeat_front_addr":"172.21.15.169:6829/262401944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:25.640995+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:51.036909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.635633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:48.467912+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:14.454971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:37.609315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:12.441807+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:38.239215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:02.230758+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:35.762553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:02.051405+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:26.569593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6801/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4004014066":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/1930101646":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/2638309002":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2238638505":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2902841144":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4190696323":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/480558521":"2024-04-26T01:04:57.149272+0000","172.21.15.31:6801/2638309002":"2024-04-26T01:04:57.149272+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:15:48.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.931+0000 7f58b2800640 1 -- 172.21.15.31:0/2038399498 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f588c0a2b90 msgr2=0x7f588c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:48.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.931+0000 7f58b2800640 1 --2- 172.21.15.31:0/2038399498 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f588c0a2b90 0x7f588c0a5050 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f58ac1027e0 tx=0x7f589c005f90 comp rx=0 tx=0).stop 2024-04-25T01:15:48.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.932+0000 7f58b2800640 1 -- 172.21.15.31:0/2038399498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58ac103340 msgr2=0x7f58ac074070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:48.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.932+0000 7f58b2800640 1 --2- 172.21.15.31:0/2038399498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58ac103340 0x7f58ac074070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f58a400e970 tx=0x7f58a400ed70 comp rx=0 tx=0).stop 2024-04-25T01:15:48.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.932+0000 7f58b2800640 1 -- 172.21.15.31:0/2038399498 shutdown_connections 2024-04-25T01:15:48.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.932+0000 7f58b2800640 1 --2- 172.21.15.31:0/2038399498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58ac103340 0x7f58ac074070 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:48.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.932+0000 7f58b2800640 1 --2- 172.21.15.31:0/2038399498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58ac102980 0x7f58ac073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:48.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.932+0000 7f58b2800640 1 --2- 172.21.15.31:0/2038399498 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f588c0a2b90 0x7f588c0a5050 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:48.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.932+0000 7f58b2800640 1 --2- 172.21.15.31:0/2038399498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58ac101790 0x7f58ac0754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:48.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.932+0000 7f58b2800640 1 -- 172.21.15.31:0/2038399498 >> 172.21.15.31:0/2038399498 conn(0x7f58ac0fcd20 msgr2=0x7f58ac105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:48.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.932+0000 7f58b2800640 1 -- 172.21.15.31:0/2038399498 shutdown_connections 2024-04-25T01:15:48.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:48.932+0000 7f58b2800640 1 -- 172.21.15.31:0/2038399498 wait complete. 2024-04-25T01:15:48.937 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T01:13:30.975848+0000', 'modified': '2024-04-25T01:13:36.545846+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24596, 'mds_1': 24565, 'mds_2': 24532}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24518': {'gid': 24518, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.40:6835/2916305801', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6834', 'nonce': 2916305801}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 2916305801}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24532': {'gid': 24532, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.169:6835/2253264784', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 2253264784}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 2253264784}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24544': {'gid': 24544, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.169:6837/2883308641', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6836', 'nonce': 2883308641}, {'type': 'v1', 'addr': '172.21.15.169:6837', 'nonce': 2883308641}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24548': {'gid': 24548, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.40:6837/590109562', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6836', 'nonce': 590109562}, {'type': 'v1', 'addr': '172.21.15.40:6837', 'nonce': 590109562}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24565': {'gid': 24565, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.169:6839/4185710218', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6838', 'nonce': 4185710218}, {'type': 'v1', 'addr': '172.21.15.169:6839', 'nonce': 4185710218}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24596': {'gid': 24596, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6839/927194004', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6838', 'nonce': 927194004}, {'type': 'v1', 'addr': '172.21.15.40:6839', 'nonce': 927194004}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24596, 'qdb_cluster': [24532, 24565, 24596]} 2024-04-25T01:15:48.937 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T01:15:48.938 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-25T01:15:49.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.164+0000 7f6d72200640 1 -- 172.21.15.31:0/547023600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d6c107da0 msgr2=0x7f6d6c1081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.164+0000 7f6d72200640 1 --2- 172.21.15.31:0/547023600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d6c107da0 0x7f6d6c1081c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009860 tx=0x7f6d5c02a340 comp rx=0 tx=0).stop 2024-04-25T01:15:49.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.164+0000 7f6d72200640 1 -- 172.21.15.31:0/547023600 shutdown_connections 2024-04-25T01:15:49.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.164+0000 7f6d72200640 1 --2- 172.21.15.31:0/547023600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d6c109950 0x7f6d6c1101e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.164+0000 7f6d72200640 1 --2- 172.21.15.31:0/547023600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d6c108f90 0x7f6d6c109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.164+0000 7f6d72200640 1 --2- 172.21.15.31:0/547023600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d6c107da0 0x7f6d6c1081c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.164+0000 7f6d72200640 1 -- 172.21.15.31:0/547023600 >> 172.21.15.31:0/547023600 conn(0x7f6d6c076d40 msgr2=0x7f6d6c079180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:49.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.164+0000 7f6d72200640 1 -- 172.21.15.31:0/547023600 shutdown_connections 2024-04-25T01:15:49.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.164+0000 7f6d72200640 1 -- 172.21.15.31:0/547023600 wait complete. 2024-04-25T01:15:49.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.165+0000 7f6d72200640 1 Processor -- start 2024-04-25T01:15:49.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.165+0000 7f6d72200640 1 -- start start 2024-04-25T01:15:49.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.166+0000 7f6d72200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d6c107da0 0x7f6d6c19e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.166+0000 7f6d72200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d6c108f90 0x7f6d6c19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.166+0000 7f6d72200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d6c109950 0x7f6d6c1a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.166+0000 7f6d72200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c1139f0 con 0x7f6d6c107da0 2024-04-25T01:15:49.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.166+0000 7f6d72200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c113870 con 0x7f6d6c108f90 2024-04-25T01:15:49.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.166+0000 7f6d72200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c113b70 con 0x7f6d6c109950 2024-04-25T01:15:49.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.166+0000 7f6d6b400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d6c107da0 0x7f6d6c19e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.166+0000 7f6d6aa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d6c108f90 0x7f6d6c19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.166+0000 7f6d6b400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d6c107da0 0x7f6d6c19e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48718/0 (socket says 172.21.15.31:48718) 2024-04-25T01:15:49.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.166+0000 7f6d6b400640 1 -- 172.21.15.31:0/2083561139 learned_addr learned my addr 172.21.15.31:0/2083561139 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:49.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.166+0000 7f6d6be00640 1 --2- 172.21.15.31:0/2083561139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d6c109950 0x7f6d6c1a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.166+0000 7f6d6b400640 1 -- 172.21.15.31:0/2083561139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d6c109950 msgr2=0x7f6d6c1a56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.166+0000 7f6d6b400640 1 --2- 172.21.15.31:0/2083561139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d6c109950 0x7f6d6c1a56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.166+0000 7f6d6b400640 1 -- 172.21.15.31:0/2083561139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d6c108f90 msgr2=0x7f6d6c19e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.166+0000 7f6d6b400640 1 --2- 172.21.15.31:0/2083561139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d6c108f90 0x7f6d6c19e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.166+0000 7f6d6b400640 1 -- 172.21.15.31:0/2083561139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6c1a5d40 con 0x7f6d6c107da0 2024-04-25T01:15:49.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.167+0000 7f6d6b400640 1 --2- 172.21.15.31:0/2083561139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d6c107da0 0x7f6d6c19e120 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c035040 tx=0x7f6d5c033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:49.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.167+0000 7f6d63e00640 1 -- 172.21.15.31:0/2083561139 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c037040 con 0x7f6d6c107da0 2024-04-25T01:15:49.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.167+0000 7f6d63e00640 1 -- 172.21.15.31:0/2083561139 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d5c032770 con 0x7f6d6c107da0 2024-04-25T01:15:49.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.167+0000 7f6d72200640 1 -- 172.21.15.31:0/2083561139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d6c1a5fd0 con 0x7f6d6c107da0 2024-04-25T01:15:49.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.167+0000 7f6d63e00640 1 -- 172.21.15.31:0/2083561139 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c0046c0 con 0x7f6d6c107da0 2024-04-25T01:15:49.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.167+0000 7f6d72200640 1 -- 172.21.15.31:0/2083561139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d6c1a63e0 con 0x7f6d6c107da0 2024-04-25T01:15:49.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.170+0000 7f6d7500d740 1 Processor -- start 2024-04-25T01:15:49.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.170+0000 7f6d7500d740 1 -- start start 2024-04-25T01:15:49.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.170+0000 7f6d63e00640 1 -- 172.21.15.31:0/2083561139 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6d5c004050 con 0x7f6d6c107da0 2024-04-25T01:15:49.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.171+0000 7f6d7500d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a470475760 0x55a470475b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.171+0000 7f6d7500d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 0x55a47047ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.171+0000 7f6d7500d740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47047b2c0 0x55a47047d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.171+0000 7f6d7500d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a470375ed0 con 0x55a470476150 2024-04-25T01:15:49.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.171+0000 7f6d7500d740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a4703d8df0 con 0x55a47047b2c0 2024-04-25T01:15:49.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.171+0000 7f6d7500d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a4703818d0 con 0x55a470475760 2024-04-25T01:15:49.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.171+0000 7f6d6b400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 0x55a47047ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.171+0000 7f6d6b400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 0x55a47047ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48730/0 (socket says 172.21.15.31:48730) 2024-04-25T01:15:49.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.171+0000 7f6d6b400640 1 -- 172.21.15.31:0/2156600530 learned_addr learned my addr 172.21.15.31:0/2156600530 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:49.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.171+0000 7f6d6be00640 1 --2- 172.21.15.31:0/2156600530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a470475760 0x55a470475b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.171+0000 7f6d6aa00640 1 --2- 172.21.15.31:0/2156600530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47047b2c0 0x55a47047d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.171+0000 7f6d6b400640 1 -- 172.21.15.31:0/2156600530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a470475760 msgr2=0x55a470475b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:49.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.171+0000 7f6d6b400640 1 --2- 172.21.15.31:0/2156600530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a470475760 0x55a470475b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.171+0000 7f6d6b400640 1 -- 172.21.15.31:0/2156600530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47047b2c0 msgr2=0x55a47047d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:49.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.171+0000 7f6d6b400640 1 --2- 172.21.15.31:0/2156600530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47047b2c0 0x55a47047d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.171+0000 7f6d6b400640 1 -- 172.21.15.31:0/2156600530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4700ed2d0 con 0x55a470476150 2024-04-25T01:15:49.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.172+0000 7f6d6b400640 1 --2- 172.21.15.31:0/2156600530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 0x55a47047ac50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c0f2dd0 tx=0x7f6d5c0f2ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98211a00ea0b350 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:49.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.172+0000 7f6d5be00640 1 -- 172.21.15.31:0/2156600530 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c0f7bf0 con 0x55a470476150 2024-04-25T01:15:49.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.172+0000 7f6d63e00640 1 --2- 172.21.15.31:0/2083561139 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6d4c0a2b90 0x7f6d4c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.172+0000 7f6d5be00640 1 -- 172.21.15.31:0/2156600530 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d5c0fc070 con 0x55a470476150 2024-04-25T01:15:49.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.172+0000 7f6d5be00640 1 -- 172.21.15.31:0/2156600530 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c0ca1a0 con 0x55a470476150 2024-04-25T01:15:49.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.172+0000 7f6d63e00640 1 -- 172.21.15.31:0/2083561139 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f6d5c0eeec0 con 0x7f6d6c107da0 2024-04-25T01:15:49.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.172+0000 7f6d6be00640 1 --2- 172.21.15.31:0/2083561139 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6d4c0a2b90 0x7f6d4c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.173+0000 7f6d7500d740 1 -- 172.21.15.31:0/2156600530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 msgr2=0x55a47047ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.173+0000 7f6d7500d740 1 --2- 172.21.15.31:0/2156600530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 0x55a47047ac50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c0f2dd0 tx=0x7f6d5c0f2ec0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.173+0000 7f6d6be00640 1 --2- 172.21.15.31:0/2083561139 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6d4c0a2b90 0x7f6d4c0a5050 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55a4704769b0 tx=0x7f6d64005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:49.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.173+0000 7f6d7500d740 1 -- 172.21.15.31:0/2156600530 shutdown_connections 2024-04-25T01:15:49.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.173+0000 7f6d7500d740 1 --2- 172.21.15.31:0/2156600530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47047b2c0 0x55a47047d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.173+0000 7f6d7500d740 1 --2- 172.21.15.31:0/2156600530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 0x55a47047ac50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.173+0000 7f6d7500d740 1 --2- 172.21.15.31:0/2156600530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a470475760 0x55a470475b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.173+0000 7f6d7500d740 1 -- 172.21.15.31:0/2156600530 >> 172.21.15.31:0/2156600530 conn(0x55a470470dd0 msgr2=0x55a4704711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:49.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.173+0000 7f6d7500d740 1 -- 172.21.15.31:0/2156600530 shutdown_connections 2024-04-25T01:15:49.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.173+0000 7f6d7500d740 1 -- 172.21.15.31:0/2156600530 wait complete. 2024-04-25T01:15:49.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.174+0000 7f6d7500d740 1 -- start start 2024-04-25T01:15:49.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.174+0000 7f6d7500d740 1 Processor -- start 2024-04-25T01:15:49.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.174+0000 7f6d7500d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a470475760 0x55a47047f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.174+0000 7f6d7500d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 0x55a47047fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.174+0000 7f6d7500d740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47047b2c0 0x55a470486c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.174+0000 7f6d7500d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a470394d20 con 0x55a470476150 2024-04-25T01:15:49.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.174+0000 7f6d7500d740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a4703d4db0 con 0x55a47047b2c0 2024-04-25T01:15:49.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.174+0000 7f6d7500d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a47047e030 con 0x55a470475760 2024-04-25T01:15:49.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.175+0000 7f6d7500d740 10 client.0 _ll_register_callbacks cb 0x55a4703cff10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:15:49.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.175+0000 7f6d6aa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 0x55a47047fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.175+0000 7f6d6be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47047b2c0 0x55a470486c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.175+0000 7f6d6aa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 0x55a47047fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48738/0 (socket says 172.21.15.31:48738) 2024-04-25T01:15:49.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.175+0000 7f6d6aa00640 1 -- 172.21.15.31:0/1729623662 learned_addr learned my addr 172.21.15.31:0/1729623662 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:49.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.175+0000 7f6d6b400640 1 --2- 172.21.15.31:0/1729623662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a470475760 0x55a47047f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.175+0000 7f6d6be00640 1 -- 172.21.15.31:0/1729623662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a470475760 msgr2=0x55a47047f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:49.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.175+0000 7f6d6be00640 1 --2- 172.21.15.31:0/1729623662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a470475760 0x55a47047f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.175+0000 7f6d6be00640 1 -- 172.21.15.31:0/1729623662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 msgr2=0x55a47047fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:49.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.175+0000 7f6d6be00640 1 --2- 172.21.15.31:0/1729623662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 0x55a47047fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.175+0000 7f6d6be00640 1 -- 172.21.15.31:0/1729623662 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4701b76b0 con 0x55a47047b2c0 2024-04-25T01:15:49.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.175+0000 7f6d6aa00640 1 --2- 172.21.15.31:0/1729623662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 0x55a47047fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:49.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.175+0000 7f6d6be00640 1 --2- 172.21.15.31:0/1729623662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47047b2c0 0x55a470486c80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6d640045e0 tx=0x7f6d64004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a98f81b173086b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:49.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.175+0000 7f6d58c00640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T01:15:49.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.176+0000 7f6d58c00640 1 -- 172.21.15.31:0/1729623662 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6401e470 con 0x55a47047b2c0 2024-04-25T01:15:49.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.176+0000 7f6d7500d740 1 -- 172.21.15.31:0/1729623662 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d5c0fc070 con 0x55a47047b2c0 2024-04-25T01:15:49.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.176+0000 7f6d58c00640 1 -- 172.21.15.31:0/1729623662 <== mon.2 v2:172.21.15.40:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f6d6401e610 con 0x55a47047b2c0 2024-04-25T01:15:49.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.176+0000 7f6d58c00640 1 -- 172.21.15.31:0/1729623662 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6d6401e810 con 0x55a47047b2c0 2024-04-25T01:15:49.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.176+0000 7f6d7500d740 10 client.25169 fetch_fsmap learned FSMap version 23 2024-04-25T01:15:49.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.176+0000 7f6d7500d740 1 -- 172.21.15.31:0/1729623662 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a47048a0b0 con 0x55a47047b2c0 2024-04-25T01:15:49.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.176+0000 7f6d58c00640 1 -- 172.21.15.31:0/1729623662 <== mon.2 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d5c0fc070 con 0x55a47047b2c0 2024-04-25T01:15:49.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.177+0000 7f6d58c00640 1 -- 172.21.15.31:0/1729623662 <== mon.2 v2:172.21.15.40:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f6d64032050 con 0x55a47047b2c0 2024-04-25T01:15:49.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.177+0000 7f6d7500d740 10 client.25169 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T01:15:49.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.177+0000 7f6d7500d740 10 client.25169 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T01:15:49.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.177+0000 7f6d7500d740 20 client.25169 populate_metadata read hostname 'smithi031' 2024-04-25T01:15:49.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.177+0000 7f6d7500d740 1 --2- 172.21.15.31:0/1729623662 >> [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] conn(0x55a47048c740 0x55a47048dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.177+0000 7f6d7500d740 4 client.25169 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:15:49.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.177+0000 7f6d7500d740 1 -- 172.21.15.31:0/1729623662 --> [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4703e6330 con 0x55a47048c740 2024-04-25T01:15:49.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.177+0000 7f6d6b400640 1 --2- 172.21.15.31:0/1729623662 >> [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] conn(0x55a47048c740 0x55a47048dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.178+0000 7f6d6b400640 1 --2- 172.21.15.31:0/1729623662 >> [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] conn(0x55a47048c740 0x55a47048dbb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f98f7cc0906bfaf server_cookie=760a5fa5edf3594f in_seq=0 out_seq=0 2024-04-25T01:15:49.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.178+0000 7f6d58c00640 10 client.25169 ms_handle_connect on v2:172.21.15.169:6834/2253264784 2024-04-25T01:15:49.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.179+0000 7f6d58c00640 1 -- 172.21.15.31:0/1729623662 <== mds.2 v2:172.21.15.169:6834/2253264784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a4703e6330 con 0x55a47048c740 2024-04-25T01:15:49.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.179+0000 7f6d58c00640 10 client.25169 handle_command_reply: tid=0 2024-04-25T01:15:49.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.179+0000 7f6d7500d740 1 client.25169 shutdown 2024-04-25T01:15:49.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.179+0000 7f6d7500d740 1 -- 172.21.15.31:0/1729623662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47047b2c0 msgr2=0x55a470486c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.179+0000 7f6d7500d740 1 --2- 172.21.15.31:0/1729623662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47047b2c0 0x55a470486c80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6d640045e0 tx=0x7f6d64004730 comp rx=0 tx=0).stop 2024-04-25T01:15:49.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.179+0000 7f6d7500d740 1 -- 172.21.15.31:0/1729623662 shutdown_connections 2024-04-25T01:15:49.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.179+0000 7f6d7500d740 1 --2- 172.21.15.31:0/1729623662 >> [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] conn(0x55a47048c740 0x55a47048dbb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.179+0000 7f6d7500d740 1 --2- 172.21.15.31:0/1729623662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47047b2c0 0x55a470486c80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.179+0000 7f6d7500d740 1 --2- 172.21.15.31:0/1729623662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 0x55a47047fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.179+0000 7f6d7500d740 1 --2- 172.21.15.31:0/1729623662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a470475760 0x55a47047f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.179+0000 7f6d58c00640 0 client.25169 ms_handle_reset on v2:172.21.15.169:6834/2253264784 2024-04-25T01:15:49.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.179+0000 7f6d7500d740 1 -- 172.21.15.31:0/1729623662 >> 172.21.15.31:0/1729623662 conn(0x55a470470dd0 msgr2=0x55a47046ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:49.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.179+0000 7f6d7500d740 1 -- 172.21.15.31:0/1729623662 shutdown_connections 2024-04-25T01:15:49.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.179+0000 7f6d7500d740 1 -- 172.21.15.31:0/1729623662 wait complete. 2024-04-25T01:15:49.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.179+0000 7f6d7500d740 20 client.25169 trim_cache size 0 max 16384 2024-04-25T01:15:49.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.199+0000 7f6d7500d740 1 Processor -- start 2024-04-25T01:15:49.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.199+0000 7f6d7500d740 1 -- start start 2024-04-25T01:15:49.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.199+0000 7f6d7500d740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a470475760 0x55a47048b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.199+0000 7f6d7500d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 0x55a470486c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.199+0000 7f6d7500d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a47047b2c0 0x55a47047f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.199+0000 7f6d7500d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a47048bef0 con 0x55a470476150 2024-04-25T01:15:49.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.199+0000 7f6d7500d740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a47048c5c0 con 0x55a470475760 2024-04-25T01:15:49.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.199+0000 7f6d7500d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a47048cc90 con 0x55a47047b2c0 2024-04-25T01:15:49.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.200+0000 7f6d6aa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 0x55a470486c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.200+0000 7f6d6aa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 0x55a470486c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48752/0 (socket says 172.21.15.31:48752) 2024-04-25T01:15:49.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.200+0000 7f6d6aa00640 1 -- 172.21.15.31:0/1815933710 learned_addr learned my addr 172.21.15.31:0/1815933710 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:49.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.200+0000 7f6d6b400640 1 --2- 172.21.15.31:0/1815933710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a470475760 0x55a47048b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.200+0000 7f6d6be00640 1 --2- 172.21.15.31:0/1815933710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a47047b2c0 0x55a47047f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.200+0000 7f6d6aa00640 1 -- 172.21.15.31:0/1815933710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a47047b2c0 msgr2=0x55a47047f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:49.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.200+0000 7f6d6aa00640 1 --2- 172.21.15.31:0/1815933710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a47047b2c0 0x55a47047f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.200+0000 7f6d6aa00640 1 -- 172.21.15.31:0/1815933710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a470475760 msgr2=0x55a47048b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:49.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.200+0000 7f6d6aa00640 1 --2- 172.21.15.31:0/1815933710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a470475760 0x55a47048b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.200+0000 7f6d6aa00640 1 -- 172.21.15.31:0/1815933710 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a470470860 con 0x55a470476150 2024-04-25T01:15:49.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.200+0000 7f6d6b400640 1 --2- 172.21.15.31:0/1815933710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a470475760 0x55a47048b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:15:49.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.200+0000 7f6d6aa00640 1 --2- 172.21.15.31:0/1815933710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 0x55a470486c40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400eae0 tx=0x7f6d540095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec25618763ba4c08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:49.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.200+0000 7f6d5b400640 1 -- 172.21.15.31:0/1815933710 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5400de70 con 0x55a470476150 2024-04-25T01:15:49.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.201+0000 7f6d5b400640 1 -- 172.21.15.31:0/1815933710 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d54005020 con 0x55a470476150 2024-04-25T01:15:49.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.201+0000 7f6d5b400640 1 -- 172.21.15.31:0/1815933710 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d54019630 con 0x55a470476150 2024-04-25T01:15:49.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.201+0000 7f6d7500d740 1 -- 172.21.15.31:0/1815933710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 msgr2=0x55a470486c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.201+0000 7f6d7500d740 1 --2- 172.21.15.31:0/1815933710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 0x55a470486c40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400eae0 tx=0x7f6d540095c0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.201+0000 7f6d7500d740 1 -- 172.21.15.31:0/1815933710 shutdown_connections 2024-04-25T01:15:49.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.201+0000 7f6d7500d740 1 --2- 172.21.15.31:0/1815933710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a47047b2c0 0x55a47047f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.201+0000 7f6d7500d740 1 --2- 172.21.15.31:0/1815933710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 0x55a470486c40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.201+0000 7f6d7500d740 1 --2- 172.21.15.31:0/1815933710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a470475760 0x55a47048b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.201+0000 7f6d7500d740 1 -- 172.21.15.31:0/1815933710 >> 172.21.15.31:0/1815933710 conn(0x55a470470dd0 msgr2=0x55a4704711b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:49.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.201+0000 7f6d7500d740 1 -- 172.21.15.31:0/1815933710 shutdown_connections 2024-04-25T01:15:49.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.201+0000 7f6d7500d740 1 -- 172.21.15.31:0/1815933710 wait complete. 2024-04-25T01:15:49.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.202+0000 7f6d7500d740 1 -- start start 2024-04-25T01:15:49.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.202+0000 7f6d7500d740 1 Processor -- start 2024-04-25T01:15:49.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.202+0000 7f6d7500d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a470475760 0x55a470482410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.202+0000 7f6d7500d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 0x55a470482950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.202+0000 7f6d7500d740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47047b2c0 0x55a4704858b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.202+0000 7f6d7500d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a47048da30 con 0x55a470476150 2024-04-25T01:15:49.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.202+0000 7f6d7500d740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a47048d360 con 0x55a47047b2c0 2024-04-25T01:15:49.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.202+0000 7f6d7500d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a47048e100 con 0x55a470475760 2024-04-25T01:15:49.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.202+0000 7f6d7500d740 10 client.0 _ll_register_callbacks cb 0x7f6d64009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:15:49.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.203+0000 7f6d6aa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 0x55a470482950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.203+0000 7f6d6be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47047b2c0 0x55a4704858b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.203+0000 7f6d6b400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a470475760 0x55a470482410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.203+0000 7f6d6be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47047b2c0 0x55a4704858b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:37472/0 (socket says 172.21.15.31:37472) 2024-04-25T01:15:49.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.203+0000 7f6d6be00640 1 -- 172.21.15.31:0/1064593766 learned_addr learned my addr 172.21.15.31:0/1064593766 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:49.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.203+0000 7f6d6be00640 1 -- 172.21.15.31:0/1064593766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a470475760 msgr2=0x55a470482410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:49.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.203+0000 7f6d6be00640 1 --2- 172.21.15.31:0/1064593766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a470475760 0x55a470482410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.203+0000 7f6d6be00640 1 -- 172.21.15.31:0/1064593766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 msgr2=0x55a470482950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:49.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.203+0000 7f6d6be00640 1 --2- 172.21.15.31:0/1064593766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 0x55a470482950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.203+0000 7f6d6be00640 1 -- 172.21.15.31:0/1064593766 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a470485ea0 con 0x55a47047b2c0 2024-04-25T01:15:49.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.203+0000 7f6d6be00640 1 --2- 172.21.15.31:0/1064593766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47047b2c0 0x55a4704858b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6d6401ec70 tx=0x7f6d6401eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af2e4444d051cbd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:49.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.203+0000 7f6d58c00640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T01:15:49.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.203+0000 7f6d58c00640 1 -- 172.21.15.31:0/1064593766 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64037020 con 0x55a47047b2c0 2024-04-25T01:15:49.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.203+0000 7f6d58c00640 1 -- 172.21.15.31:0/1064593766 <== mon.2 v2:172.21.15.40:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f6d64029650 con 0x55a47047b2c0 2024-04-25T01:15:49.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.203+0000 7f6d7500d740 1 -- 172.21.15.31:0/1064593766 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d54005020 con 0x55a47047b2c0 2024-04-25T01:15:49.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.204+0000 7f6d58c00640 1 -- 172.21.15.31:0/1064593766 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6d64042050 con 0x55a47047b2c0 2024-04-25T01:15:49.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.204+0000 7f6d7500d740 10 client.25175 fetch_fsmap learned FSMap version 23 2024-04-25T01:15:49.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.204+0000 7f6d7500d740 1 -- 172.21.15.31:0/1064593766 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a4704f2750 con 0x55a47047b2c0 2024-04-25T01:15:49.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.204+0000 7f6d58c00640 1 -- 172.21.15.31:0/1064593766 <== mon.2 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d54005020 con 0x55a47047b2c0 2024-04-25T01:15:49.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.204+0000 7f6d58c00640 1 -- 172.21.15.31:0/1064593766 <== mon.2 v2:172.21.15.40:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f6d64032040 con 0x55a47047b2c0 2024-04-25T01:15:49.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.204+0000 7f6d7500d740 10 client.25175 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T01:15:49.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.205+0000 7f6d7500d740 10 client.25175 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T01:15:49.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.205+0000 7f6d7500d740 20 client.25175 populate_metadata read hostname 'smithi031' 2024-04-25T01:15:49.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.205+0000 7f6d7500d740 1 --2- 172.21.15.31:0/1064593766 >> [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] conn(0x55a4704f7350 0x55a4704f3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.205+0000 7f6d7500d740 4 client.25175 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T01:15:49.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.205+0000 7f6d7500d740 1 -- 172.21.15.31:0/1064593766 --> [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] -- command(tid 0: {"prefix": "status"}) -- 0x55a4703e0680 con 0x55a4704f7350 2024-04-25T01:15:49.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.205+0000 7f6d6b400640 1 --2- 172.21.15.31:0/1064593766 >> [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] conn(0x55a4704f7350 0x55a4704f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.205+0000 7f6d6b400640 1 --2- 172.21.15.31:0/1064593766 >> [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] conn(0x55a4704f7350 0x55a4704f3790 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2fcde272ac64c0ee server_cookie=31b4f5ccf817aa8b in_seq=0 out_seq=0 2024-04-25T01:15:49.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.205+0000 7f6d58c00640 10 client.25175 ms_handle_connect on v2:172.21.15.169:6834/2253264784 2024-04-25T01:15:49.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.206+0000 7f6d58c00640 1 -- 172.21.15.31:0/1064593766 <== mds.2 v2:172.21.15.169:6834/2253264784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a4703e0680 con 0x55a4704f7350 2024-04-25T01:15:49.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.206+0000 7f6d58c00640 10 client.25175 handle_command_reply: tid=0 2024-04-25T01:15:49.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.206+0000 7f6d7500d740 1 client.25175 shutdown 2024-04-25T01:15:49.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.206+0000 7f6d7500d740 1 -- 172.21.15.31:0/1064593766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47047b2c0 msgr2=0x55a4704858b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.206+0000 7f6d7500d740 1 --2- 172.21.15.31:0/1064593766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47047b2c0 0x55a4704858b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6d6401ec70 tx=0x7f6d6401eca0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.206+0000 7f6d7500d740 1 -- 172.21.15.31:0/1064593766 shutdown_connections 2024-04-25T01:15:49.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.206+0000 7f6d7500d740 1 --2- 172.21.15.31:0/1064593766 >> [v2:172.21.15.169:6834/2253264784,v1:172.21.15.169:6835/2253264784] conn(0x55a4704f7350 0x55a4704f3790 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.206+0000 7f6d7500d740 1 --2- 172.21.15.31:0/1064593766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a47047b2c0 0x55a4704858b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.206+0000 7f6d7500d740 1 --2- 172.21.15.31:0/1064593766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a470476150 0x55a470482950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.206+0000 7f6d58c00640 0 client.25175 ms_handle_reset on v2:172.21.15.169:6834/2253264784 2024-04-25T01:15:49.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.206+0000 7f6d7500d740 1 --2- 172.21.15.31:0/1064593766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a470475760 0x55a470482410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.206+0000 7f6d7500d740 1 -- 172.21.15.31:0/1064593766 >> 172.21.15.31:0/1064593766 conn(0x55a470470dd0 msgr2=0x55a47048b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:49.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.206+0000 7f6d7500d740 1 -- 172.21.15.31:0/1064593766 shutdown_connections 2024-04-25T01:15:49.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.206+0000 7f6d7500d740 1 -- 172.21.15.31:0/1064593766 wait complete. 2024-04-25T01:15:49.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.206+0000 7f6d7500d740 20 client.25175 trim_cache size 0 max 16384 2024-04-25T01:15:49.197 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T01:15:49.197 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "95849b94-029f-11ef-bc93-c7b262605968", 2024-04-25T01:15:49.197 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 2, 2024-04-25T01:15:49.197 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24532, 2024-04-25T01:15:49.198 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-04-25T01:15:49.198 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-04-25T01:15:49.198 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-04-25T01:15:49.198 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 133.668928569, 2024-04-25T01:15:49.198 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 23, 2024-04-25T01:15:49.198 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 89, 2024-04-25T01:15:49.198 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 87, 2024-04-25T01:15:49.198 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 164.52774939099999 2024-04-25T01:15:49.198 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T01:15:49.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.207+0000 7f6d72200640 1 -- 172.21.15.31:0/2083561139 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6d4c0a2b90 msgr2=0x7f6d4c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.207+0000 7f6d72200640 1 --2- 172.21.15.31:0/2083561139 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6d4c0a2b90 0x7f6d4c0a5050 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55a4704769b0 tx=0x7f6d64005f90 comp rx=0 tx=0).stop 2024-04-25T01:15:49.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.207+0000 7f6d72200640 1 -- 172.21.15.31:0/2083561139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d6c107da0 msgr2=0x7f6d6c19e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.207+0000 7f6d72200640 1 --2- 172.21.15.31:0/2083561139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d6c107da0 0x7f6d6c19e120 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c035040 tx=0x7f6d5c033d90 comp rx=0 tx=0).stop 2024-04-25T01:15:49.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.207+0000 7f6d72200640 1 -- 172.21.15.31:0/2083561139 shutdown_connections 2024-04-25T01:15:49.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.207+0000 7f6d72200640 1 --2- 172.21.15.31:0/2083561139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d6c109950 0x7f6d6c1a56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.207+0000 7f6d72200640 1 --2- 172.21.15.31:0/2083561139 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6d4c0a2b90 0x7f6d4c0a5050 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.207+0000 7f6d72200640 1 --2- 172.21.15.31:0/2083561139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d6c108f90 0x7f6d6c19e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.207+0000 7f6d72200640 1 --2- 172.21.15.31:0/2083561139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d6c107da0 0x7f6d6c19e120 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.207+0000 7f6d72200640 1 -- 172.21.15.31:0/2083561139 >> 172.21.15.31:0/2083561139 conn(0x7f6d6c076d40 msgr2=0x7f6d6c10c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:49.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.207+0000 7f6d72200640 1 -- 172.21.15.31:0/2083561139 shutdown_connections 2024-04-25T01:15:49.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.207+0000 7f6d72200640 1 -- 172.21.15.31:0/2083561139 wait complete. 2024-04-25T01:15:49.209 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T01:15:49.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.436+0000 7f254be00640 1 -- 172.21.15.31:0/4108314674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f254c104a40 msgr2=0x7f254c106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.436+0000 7f254be00640 1 --2- 172.21.15.31:0/4108314674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f254c104a40 0x7f254c106e30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2534009ca0 tx=0x7f253402a8c0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.436+0000 7f254be00640 1 -- 172.21.15.31:0/4108314674 shutdown_connections 2024-04-25T01:15:49.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.436+0000 7f254be00640 1 --2- 172.21.15.31:0/4108314674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f254c107370 0x7f254c109780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.436+0000 7f254be00640 1 --2- 172.21.15.31:0/4108314674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f254c104a40 0x7f254c106e30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.436+0000 7f254be00640 1 --2- 172.21.15.31:0/4108314674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f254c102110 0x7f254c104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.436+0000 7f254be00640 1 -- 172.21.15.31:0/4108314674 >> 172.21.15.31:0/4108314674 conn(0x7f254c0fbae0 msgr2=0x7f254c0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:49.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.436+0000 7f254be00640 1 -- 172.21.15.31:0/4108314674 shutdown_connections 2024-04-25T01:15:49.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.437+0000 7f254be00640 1 -- 172.21.15.31:0/4108314674 wait complete. 2024-04-25T01:15:49.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.438+0000 7f254be00640 1 Processor -- start 2024-04-25T01:15:49.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.438+0000 7f254be00640 1 -- start start 2024-04-25T01:15:49.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.438+0000 7f254be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f254c102110 0x7f254c199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.438+0000 7f254be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f254c104a40 0x7f254c19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.438+0000 7f254be00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f254c107370 0x7f254c1a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.438+0000 7f254be00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f254c10d310 con 0x7f254c107370 2024-04-25T01:15:49.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.438+0000 7f254be00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f254c10d190 con 0x7f254c102110 2024-04-25T01:15:49.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.438+0000 7f254be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f254c10d490 con 0x7f254c104a40 2024-04-25T01:15:49.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.439+0000 7f254aa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f254c102110 0x7f254c199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.439+0000 7f254b400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f254c107370 0x7f254c1a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.439+0000 7f254aa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f254c102110 0x7f254c199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:37498/0 (socket says 172.21.15.31:37498) 2024-04-25T01:15:49.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.439+0000 7f254aa00640 1 -- 172.21.15.31:0/2894546796 learned_addr learned my addr 172.21.15.31:0/2894546796 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:49.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.439+0000 7f2542000640 1 --2- 172.21.15.31:0/2894546796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f254c104a40 0x7f254c19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.439+0000 7f254aa00640 1 -- 172.21.15.31:0/2894546796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f254c104a40 msgr2=0x7f254c19a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.439+0000 7f254aa00640 1 --2- 172.21.15.31:0/2894546796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f254c104a40 0x7f254c19a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.439+0000 7f254aa00640 1 -- 172.21.15.31:0/2894546796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f254c107370 msgr2=0x7f254c1a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.439+0000 7f254aa00640 1 --2- 172.21.15.31:0/2894546796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f254c107370 0x7f254c1a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.439+0000 7f254aa00640 1 -- 172.21.15.31:0/2894546796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254c1a17a0 con 0x7f254c102110 2024-04-25T01:15:49.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.439+0000 7f254aa00640 1 --2- 172.21.15.31:0/2894546796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f254c102110 0x7f254c199b80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f253c00e9f0 tx=0x7f253c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:49.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.440+0000 7f2543e00640 1 -- 172.21.15.31:0/2894546796 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c00cd80 con 0x7f254c102110 2024-04-25T01:15:49.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.440+0000 7f254be00640 1 -- 172.21.15.31:0/2894546796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f254c1a1a90 con 0x7f254c102110 2024-04-25T01:15:49.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.440+0000 7f2543e00640 1 -- 172.21.15.31:0/2894546796 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f253c010910 con 0x7f254c102110 2024-04-25T01:15:49.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.440+0000 7f2543e00640 1 -- 172.21.15.31:0/2894546796 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c0188f0 con 0x7f254c102110 2024-04-25T01:15:49.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.440+0000 7f254be00640 1 -- 172.21.15.31:0/2894546796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f254c1a2000 con 0x7f254c102110 2024-04-25T01:15:49.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.443+0000 7f2543e00640 1 -- 172.21.15.31:0/2894546796 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f253c010ac0 con 0x7f254c102110 2024-04-25T01:15:49.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.443+0000 7f25549c8740 1 Processor -- start 2024-04-25T01:15:49.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.443+0000 7f25549c8740 1 -- start start 2024-04-25T01:15:49.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.444+0000 7f25549c8740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a2c3a970 0x5599a2c3ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.444+0000 7f25549c8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599a2c3b360 0x5599a2c3fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.444+0000 7f25549c8740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a2c404d0 0x5599a2c42960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.444+0000 7f25549c8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5599a2b3aed0 con 0x5599a2c3b360 2024-04-25T01:15:49.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.444+0000 7f25549c8740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5599a2b9ddf0 con 0x5599a2c404d0 2024-04-25T01:15:49.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.444+0000 7f25549c8740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5599a2b468d0 con 0x5599a2c3a970 2024-04-25T01:15:49.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.444+0000 7f254aa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599a2c3b360 0x5599a2c3fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.444+0000 7f254b400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a2c3a970 0x5599a2c3ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.444+0000 7f2542000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a2c404d0 0x5599a2c42960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.444+0000 7f254b400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a2c3a970 0x5599a2c3ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:39988/0 (socket says 172.21.15.31:39988) 2024-04-25T01:15:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.444+0000 7f254b400640 1 -- 172.21.15.31:0/554313362 learned_addr learned my addr 172.21.15.31:0/554313362 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.444+0000 7f2542000640 1 --2- 172.21.15.31:0/554313362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a2c404d0 0x5599a2c42960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:37512/0 (socket says 172.21.15.31:37512) 2024-04-25T01:15:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.445+0000 7f254b400640 1 -- 172.21.15.31:0/554313362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a2c404d0 msgr2=0x5599a2c42960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.445+0000 7f254b400640 1 --2- 172.21.15.31:0/554313362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a2c404d0 0x5599a2c42960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.445+0000 7f254b400640 1 -- 172.21.15.31:0/554313362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599a2c3b360 msgr2=0x5599a2c3fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.445+0000 7f254b400640 1 --2- 172.21.15.31:0/554313362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599a2c3b360 0x5599a2c3fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.445+0000 7f254b400640 1 -- 172.21.15.31:0/554313362 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599a28b22d0 con 0x5599a2c3a970 2024-04-25T01:15:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.445+0000 7f254b400640 1 --2- 172.21.15.31:0/554313362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a2c3a970 0x5599a2c3ad90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2544009ca0 tx=0x7f2544011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ed89768c016e0a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.445+0000 7f2543e00640 1 --2- 172.21.15.31:0/2894546796 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f25300a2b90 0x7f25300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.445+0000 7f253be00640 1 -- 172.21.15.31:0/554313362 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254401e070 con 0x5599a2c3a970 2024-04-25T01:15:49.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.445+0000 7f2543e00640 1 -- 172.21.15.31:0/2894546796 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f253c0cf400 con 0x7f254c102110 2024-04-25T01:15:49.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.445+0000 7f253be00640 1 -- 172.21.15.31:0/554313362 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2544009e50 con 0x5599a2c3a970 2024-04-25T01:15:49.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.445+0000 7f254aa00640 1 --2- 172.21.15.31:0/2894546796 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f25300a2b90 0x7f25300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.445+0000 7f253be00640 1 -- 172.21.15.31:0/554313362 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254400eac0 con 0x5599a2c3a970 2024-04-25T01:15:49.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.446+0000 7f25549c8740 1 -- 172.21.15.31:0/554313362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a2c3a970 msgr2=0x5599a2c3ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.446+0000 7f254aa00640 1 --2- 172.21.15.31:0/2894546796 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f25300a2b90 0x7f25300a5050 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5599a2c3b760 tx=0x7f253c0d2e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:49.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.446+0000 7f25549c8740 1 --2- 172.21.15.31:0/554313362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a2c3a970 0x5599a2c3ad90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2544009ca0 tx=0x7f2544011810 comp rx=0 tx=0).stop 2024-04-25T01:15:49.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.446+0000 7f25549c8740 1 -- 172.21.15.31:0/554313362 shutdown_connections 2024-04-25T01:15:49.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.446+0000 7f25549c8740 1 --2- 172.21.15.31:0/554313362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a2c404d0 0x5599a2c42960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.446+0000 7f25549c8740 1 --2- 172.21.15.31:0/554313362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599a2c3b360 0x5599a2c3fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.446+0000 7f25549c8740 1 --2- 172.21.15.31:0/554313362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a2c3a970 0x5599a2c3ad90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.446+0000 7f25549c8740 1 -- 172.21.15.31:0/554313362 >> 172.21.15.31:0/554313362 conn(0x5599a2c35fe0 msgr2=0x5599a2c363e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:49.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.446+0000 7f25549c8740 1 -- 172.21.15.31:0/554313362 shutdown_connections 2024-04-25T01:15:49.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.446+0000 7f25549c8740 1 -- 172.21.15.31:0/554313362 wait complete. 2024-04-25T01:15:49.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.447+0000 7f25549c8740 1 -- start start 2024-04-25T01:15:49.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.447+0000 7f25549c8740 1 Processor -- start 2024-04-25T01:15:49.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.447+0000 7f25549c8740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a2c3a970 0x5599a2c44c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.447+0000 7f25549c8740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a2c3b360 0x5599a2c451a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.448+0000 7f25549c8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599a2c404d0 0x5599a2c4c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.448+0000 7f25549c8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5599a2b59d20 con 0x5599a2c404d0 2024-04-25T01:15:49.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.448+0000 7f25549c8740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5599a2b99db0 con 0x5599a2c3b360 2024-04-25T01:15:49.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.448+0000 7f25549c8740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5599a2c43480 con 0x5599a2c3a970 2024-04-25T01:15:49.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.448+0000 7f25549c8740 10 client.0 _ll_register_callbacks cb 0x5599a2b94f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:15:49.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.448+0000 7f254b400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599a2c404d0 0x5599a2c4c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.448+0000 7f2542000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a2c3b360 0x5599a2c451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.448+0000 7f254b400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599a2c404d0 0x5599a2c4c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48780/0 (socket says 172.21.15.31:48780) 2024-04-25T01:15:49.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.448+0000 7f254b400640 1 -- 172.21.15.31:0/2589262538 learned_addr learned my addr 172.21.15.31:0/2589262538 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:49.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.448+0000 7f2542000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a2c3b360 0x5599a2c451a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:37522/0 (socket says 172.21.15.31:37522) 2024-04-25T01:15:49.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.448+0000 7f254b400640 1 --2- 172.21.15.31:0/2589262538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a2c3a970 0x5599a2c44c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.448+0000 7f2542000640 1 -- 172.21.15.31:0/2589262538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a2c3a970 msgr2=0x5599a2c44c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:49.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.448+0000 7f2542000640 1 --2- 172.21.15.31:0/2589262538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a2c3a970 0x5599a2c44c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.448+0000 7f2542000640 1 -- 172.21.15.31:0/2589262538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599a2c404d0 msgr2=0x5599a2c4c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:49.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.448+0000 7f2542000640 1 --2- 172.21.15.31:0/2589262538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599a2c404d0 0x5599a2c4c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.448+0000 7f2542000640 1 -- 172.21.15.31:0/2589262538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599a297c6b0 con 0x5599a2c3b360 2024-04-25T01:15:49.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.449+0000 7f2542000640 1 --2- 172.21.15.31:0/2589262538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a2c3b360 0x5599a2c451a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f253402ade0 tx=0x7f2534009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abe1133d5beac5c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:49.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.449+0000 7f2538c00640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T01:15:49.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.449+0000 7f2538c00640 1 -- 172.21.15.31:0/2589262538 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2534003b80 con 0x5599a2c3b360 2024-04-25T01:15:49.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.449+0000 7f2538c00640 1 -- 172.21.15.31:0/2589262538 <== mon.2 v2:172.21.15.40:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f2534003d20 con 0x5599a2c3b360 2024-04-25T01:15:49.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.449+0000 7f25549c8740 1 -- 172.21.15.31:0/2589262538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2544009e50 con 0x5599a2c3b360 2024-04-25T01:15:49.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.449+0000 7f2538c00640 1 -- 172.21.15.31:0/2589262538 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2534004040 con 0x5599a2c3b360 2024-04-25T01:15:49.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.449+0000 7f25549c8740 10 client.25187 fetch_fsmap learned FSMap version 23 2024-04-25T01:15:49.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.449+0000 7f25549c8740 1 -- 172.21.15.31:0/2589262538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5599a2c4f5f0 con 0x5599a2c3b360 2024-04-25T01:15:49.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.450+0000 7f2538c00640 1 -- 172.21.15.31:0/2589262538 <== mon.2 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2544009e50 con 0x5599a2c3b360 2024-04-25T01:15:49.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.450+0000 7f2538c00640 1 -- 172.21.15.31:0/2589262538 <== mon.2 v2:172.21.15.40:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f253402f9e0 con 0x5599a2c3b360 2024-04-25T01:15:49.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.450+0000 7f25549c8740 10 client.25187 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T01:15:49.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.450+0000 7f25549c8740 10 client.25187 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T01:15:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.450+0000 7f25549c8740 20 client.25187 populate_metadata read hostname 'smithi031' 2024-04-25T01:15:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.450+0000 7f25549c8740 1 --2- 172.21.15.31:0/2589262538 >> [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] conn(0x5599a2c51c10 0x5599a2c53080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.450+0000 7f25549c8740 4 client.25187 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:15:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.450+0000 7f25549c8740 1 -- 172.21.15.31:0/2589262538 --> [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599a2bab330 con 0x5599a2c51c10 2024-04-25T01:15:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.451+0000 7f254b400640 1 --2- 172.21.15.31:0/2589262538 >> [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] conn(0x5599a2c51c10 0x5599a2c53080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.451+0000 7f254b400640 1 --2- 172.21.15.31:0/2589262538 >> [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] conn(0x5599a2c51c10 0x5599a2c53080 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d5949f63ca89aa7a server_cookie=3102085c34de3429 in_seq=0 out_seq=0 2024-04-25T01:15:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.451+0000 7f2538c00640 10 client.25187 ms_handle_connect on v2:172.21.15.169:6838/4185710218 2024-04-25T01:15:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.452+0000 7f2538c00640 1 -- 172.21.15.31:0/2589262538 <== mds.1 v2:172.21.15.169:6838/4185710218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5599a2bab330 con 0x5599a2c51c10 2024-04-25T01:15:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.452+0000 7f2538c00640 10 client.25187 handle_command_reply: tid=0 2024-04-25T01:15:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.452+0000 7f25549c8740 1 client.25187 shutdown 2024-04-25T01:15:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.452+0000 7f25549c8740 1 -- 172.21.15.31:0/2589262538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a2c3b360 msgr2=0x5599a2c451a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.452+0000 7f25549c8740 1 --2- 172.21.15.31:0/2589262538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a2c3b360 0x5599a2c451a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f253402ade0 tx=0x7f2534009800 comp rx=0 tx=0).stop 2024-04-25T01:15:49.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.452+0000 7f25549c8740 1 -- 172.21.15.31:0/2589262538 shutdown_connections 2024-04-25T01:15:49.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.452+0000 7f25549c8740 1 --2- 172.21.15.31:0/2589262538 >> [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] conn(0x5599a2c51c10 0x5599a2c53080 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.452+0000 7f25549c8740 1 --2- 172.21.15.31:0/2589262538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599a2c404d0 0x5599a2c4c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.452+0000 7f25549c8740 1 --2- 172.21.15.31:0/2589262538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a2c3b360 0x5599a2c451a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.452+0000 7f2538c00640 0 client.25187 ms_handle_reset on v2:172.21.15.169:6838/4185710218 2024-04-25T01:15:49.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.452+0000 7f25549c8740 1 --2- 172.21.15.31:0/2589262538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a2c3a970 0x5599a2c44c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.452+0000 7f25549c8740 1 -- 172.21.15.31:0/2589262538 >> 172.21.15.31:0/2589262538 conn(0x5599a2c35fe0 msgr2=0x5599a2c35120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:49.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.452+0000 7f25549c8740 1 -- 172.21.15.31:0/2589262538 shutdown_connections 2024-04-25T01:15:49.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.452+0000 7f25549c8740 1 -- 172.21.15.31:0/2589262538 wait complete. 2024-04-25T01:15:49.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.453+0000 7f25549c8740 20 client.25187 trim_cache size 0 max 16384 2024-04-25T01:15:49.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.472+0000 7f25549c8740 1 Processor -- start 2024-04-25T01:15:49.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.473+0000 7f25549c8740 1 -- start start 2024-04-25T01:15:49.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.473+0000 7f25549c8740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a2c3a970 0x5599a2c34890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.473+0000 7f25549c8740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a2c3b360 0x5599a2c4c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.473+0000 7f25549c8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599a2c404d0 0x5599a2c34dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.473+0000 7f25549c8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5599a2c513c0 con 0x5599a2c404d0 2024-04-25T01:15:49.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.473+0000 7f25549c8740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5599a2c51a90 con 0x5599a2c3a970 2024-04-25T01:15:49.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.473+0000 7f25549c8740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5599a2c52160 con 0x5599a2c3b360 2024-04-25T01:15:49.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.473+0000 7f2542000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a2c3b360 0x5599a2c4c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.473+0000 7f254b400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599a2c404d0 0x5599a2c34dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.473+0000 7f2542000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a2c3b360 0x5599a2c4c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:40012/0 (socket says 172.21.15.31:40012) 2024-04-25T01:15:49.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.473+0000 7f2542000640 1 -- 172.21.15.31:0/3783455641 learned_addr learned my addr 172.21.15.31:0/3783455641 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:49.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.473+0000 7f254b400640 1 --2- 172.21.15.31:0/3783455641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a2c3a970 0x5599a2c34890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.474+0000 7f2542000640 1 -- 172.21.15.31:0/3783455641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a2c3a970 msgr2=0x5599a2c34890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:49.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.474+0000 7f2542000640 1 --2- 172.21.15.31:0/3783455641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a2c3a970 0x5599a2c34890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.474+0000 7f2542000640 1 -- 172.21.15.31:0/3783455641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599a2c404d0 msgr2=0x5599a2c34dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:49.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.474+0000 7f2542000640 1 --2- 172.21.15.31:0/3783455641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599a2c404d0 0x5599a2c34dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.474+0000 7f2542000640 1 -- 172.21.15.31:0/3783455641 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599a2c3bb50 con 0x5599a2c3b360 2024-04-25T01:15:49.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.474+0000 7f2542000640 1 --2- 172.21.15.31:0/3783455641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a2c3b360 0x5599a2c4c160 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2534000f80 tx=0x7f2534009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b56842b1ca4724b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:49.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.475+0000 7f253b400640 1 -- 172.21.15.31:0/3783455641 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2534054070 con 0x5599a2c3b360 2024-04-25T01:15:49.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.475+0000 7f253b400640 1 -- 172.21.15.31:0/3783455641 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2534036050 con 0x5599a2c3b360 2024-04-25T01:15:49.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.475+0000 7f253b400640 1 -- 172.21.15.31:0/3783455641 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25340308c0 con 0x5599a2c3b360 2024-04-25T01:15:49.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.475+0000 7f25549c8740 1 -- 172.21.15.31:0/3783455641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a2c3b360 msgr2=0x5599a2c4c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.475+0000 7f25549c8740 1 --2- 172.21.15.31:0/3783455641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a2c3b360 0x5599a2c4c160 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2534000f80 tx=0x7f2534009360 comp rx=0 tx=0).stop 2024-04-25T01:15:49.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.475+0000 7f25549c8740 1 -- 172.21.15.31:0/3783455641 shutdown_connections 2024-04-25T01:15:49.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.475+0000 7f25549c8740 1 --2- 172.21.15.31:0/3783455641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599a2c404d0 0x5599a2c34dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.475+0000 7f25549c8740 1 --2- 172.21.15.31:0/3783455641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a2c3b360 0x5599a2c4c160 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.475+0000 7f25549c8740 1 --2- 172.21.15.31:0/3783455641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a2c3a970 0x5599a2c34890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.475+0000 7f25549c8740 1 -- 172.21.15.31:0/3783455641 >> 172.21.15.31:0/3783455641 conn(0x5599a2c35fe0 msgr2=0x5599a2c363c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:49.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.475+0000 7f25549c8740 1 -- 172.21.15.31:0/3783455641 shutdown_connections 2024-04-25T01:15:49.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.475+0000 7f25549c8740 1 -- 172.21.15.31:0/3783455641 wait complete. 2024-04-25T01:15:49.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.476+0000 7f25549c8740 1 -- start start 2024-04-25T01:15:49.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.476+0000 7f25549c8740 1 Processor -- start 2024-04-25T01:15:49.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.476+0000 7f25549c8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599a2c3a970 0x5599a2c47e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.476+0000 7f25549c8740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a2c3b360 0x5599a2c48370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.476+0000 7f25549c8740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a2c404d0 0x5599a2c4b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.477+0000 7f25549c8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5599a2c52f00 con 0x5599a2c3a970 2024-04-25T01:15:49.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.477+0000 7f25549c8740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5599a2c52830 con 0x5599a2c3b360 2024-04-25T01:15:49.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.477+0000 7f25549c8740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5599a2c535d0 con 0x5599a2c404d0 2024-04-25T01:15:49.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.477+0000 7f25549c8740 10 client.0 _ll_register_callbacks cb 0x7f2534009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:15:49.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.477+0000 7f254b400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599a2c3a970 0x5599a2c47e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.477+0000 7f254b400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599a2c3a970 0x5599a2c47e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48790/0 (socket says 172.21.15.31:48790) 2024-04-25T01:15:49.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.477+0000 7f254b400640 1 -- 172.21.15.31:0/155408555 learned_addr learned my addr 172.21.15.31:0/155408555 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:49.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.477+0000 7f2542000640 1 --2- 172.21.15.31:0/155408555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a2c3b360 0x5599a2c48370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.477+0000 7f254b400640 1 -- 172.21.15.31:0/155408555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a2c404d0 msgr2=0x5599a2c4b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:49.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.477+0000 7f254b400640 1 --2- 172.21.15.31:0/155408555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a2c404d0 0x5599a2c4b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.477+0000 7f254b400640 1 -- 172.21.15.31:0/155408555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a2c3b360 msgr2=0x5599a2c48370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:49.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.477+0000 7f254b400640 1 --2- 172.21.15.31:0/155408555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a2c3b360 0x5599a2c48370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.477+0000 7f254b400640 1 -- 172.21.15.31:0/155408555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599a2c4b980 con 0x5599a2c3a970 2024-04-25T01:15:49.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.477+0000 7f254b400640 1 --2- 172.21.15.31:0/155408555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599a2c3a970 0x5599a2c47e30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2544009ca0 tx=0x7f25440098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78178ae4c31fdf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:49.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.477+0000 7f2538c00640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T01:15:49.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.478+0000 7f2538c00640 1 -- 172.21.15.31:0/155408555 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254402c020 con 0x5599a2c3a970 2024-04-25T01:15:49.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.478+0000 7f2538c00640 1 -- 172.21.15.31:0/155408555 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f254402fb50 con 0x5599a2c3a970 2024-04-25T01:15:49.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.478+0000 7f25549c8740 1 -- 172.21.15.31:0/155408555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2534036050 con 0x5599a2c3a970 2024-04-25T01:15:49.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.478+0000 7f2538c00640 1 -- 172.21.15.31:0/155408555 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f254401e040 con 0x5599a2c3a970 2024-04-25T01:15:49.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.478+0000 7f25549c8740 10 client.15072 fetch_fsmap learned FSMap version 23 2024-04-25T01:15:49.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.478+0000 7f25549c8740 1 -- 172.21.15.31:0/155408555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5599a2cc09c0 con 0x5599a2c3a970 2024-04-25T01:15:49.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.478+0000 7f2538c00640 1 -- 172.21.15.31:0/155408555 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2534036050 con 0x5599a2c3a970 2024-04-25T01:15:49.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.478+0000 7f2538c00640 1 -- 172.21.15.31:0/155408555 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f2544024030 con 0x5599a2c3a970 2024-04-25T01:15:49.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.479+0000 7f25549c8740 10 client.15072 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T01:15:49.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.479+0000 7f25549c8740 10 client.15072 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T01:15:49.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.479+0000 7f25549c8740 20 client.15072 populate_metadata read hostname 'smithi031' 2024-04-25T01:15:49.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.479+0000 7f25549c8740 1 --2- 172.21.15.31:0/155408555 >> [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] conn(0x5599a2cc5350 0x5599a2cc4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.479+0000 7f25549c8740 4 client.15072 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T01:15:49.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.479+0000 7f25549c8740 1 -- 172.21.15.31:0/155408555 --> [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] -- command(tid 0: {"prefix": "status"}) -- 0x5599a2ba5680 con 0x5599a2cc5350 2024-04-25T01:15:49.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.479+0000 7f2542000640 1 --2- 172.21.15.31:0/155408555 >> [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] conn(0x5599a2cc5350 0x5599a2cc4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.480+0000 7f2542000640 1 --2- 172.21.15.31:0/155408555 >> [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] conn(0x5599a2cc5350 0x5599a2cc4060 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a23f9fb775218383 server_cookie=e7da6c29ca804074 in_seq=0 out_seq=0 2024-04-25T01:15:49.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.480+0000 7f2538c00640 10 client.15072 ms_handle_connect on v2:172.21.15.169:6838/4185710218 2024-04-25T01:15:49.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.480+0000 7f2538c00640 1 -- 172.21.15.31:0/155408555 <== mds.1 v2:172.21.15.169:6838/4185710218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5599a2ba5680 con 0x5599a2cc5350 2024-04-25T01:15:49.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.480+0000 7f2538c00640 10 client.15072 handle_command_reply: tid=0 2024-04-25T01:15:49.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.480+0000 7f25549c8740 1 client.15072 shutdown 2024-04-25T01:15:49.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.480+0000 7f25549c8740 1 -- 172.21.15.31:0/155408555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599a2c3a970 msgr2=0x5599a2c47e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.480+0000 7f25549c8740 1 --2- 172.21.15.31:0/155408555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599a2c3a970 0x5599a2c47e30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2544009ca0 tx=0x7f25440098f0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.480+0000 7f25549c8740 1 -- 172.21.15.31:0/155408555 shutdown_connections 2024-04-25T01:15:49.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.480+0000 7f25549c8740 1 --2- 172.21.15.31:0/155408555 >> [v2:172.21.15.169:6838/4185710218,v1:172.21.15.169:6839/4185710218] conn(0x5599a2cc5350 0x5599a2cc4060 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.480+0000 7f25549c8740 1 --2- 172.21.15.31:0/155408555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5599a2c404d0 0x5599a2c4b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.480+0000 7f25549c8740 1 --2- 172.21.15.31:0/155408555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a2c3b360 0x5599a2c48370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.480+0000 7f25549c8740 1 --2- 172.21.15.31:0/155408555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599a2c3a970 0x5599a2c47e30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.480+0000 7f25549c8740 1 -- 172.21.15.31:0/155408555 >> 172.21.15.31:0/155408555 conn(0x5599a2c35fe0 msgr2=0x5599a2c34890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:49.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.480+0000 7f2538c00640 0 client.15072 ms_handle_reset on v2:172.21.15.169:6838/4185710218 2024-04-25T01:15:49.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.480+0000 7f25549c8740 1 -- 172.21.15.31:0/155408555 shutdown_connections 2024-04-25T01:15:49.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.480+0000 7f25549c8740 1 -- 172.21.15.31:0/155408555 wait complete. 2024-04-25T01:15:49.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.481+0000 7f25549c8740 20 client.15072 trim_cache size 0 max 16384 2024-04-25T01:15:49.473 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T01:15:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "95849b94-029f-11ef-bc93-c7b262605968", 2024-04-25T01:15:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 1, 2024-04-25T01:15:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24565, 2024-04-25T01:15:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-04-25T01:15:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-04-25T01:15:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-04-25T01:15:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 134.898161617, 2024-04-25T01:15:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 23, 2024-04-25T01:15:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 89, 2024-04-25T01:15:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 87, 2024-04-25T01:15:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 156.79610094500001 2024-04-25T01:15:49.474 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T01:15:49.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.481+0000 7f254be00640 1 -- 172.21.15.31:0/2894546796 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f25300a2b90 msgr2=0x7f25300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.481+0000 7f254be00640 1 --2- 172.21.15.31:0/2894546796 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f25300a2b90 0x7f25300a5050 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5599a2c3b760 tx=0x7f253c0d2e40 comp rx=0 tx=0).stop 2024-04-25T01:15:49.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.481+0000 7f254be00640 1 -- 172.21.15.31:0/2894546796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f254c102110 msgr2=0x7f254c199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.481+0000 7f254be00640 1 --2- 172.21.15.31:0/2894546796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f254c102110 0x7f254c199b80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f253c00e9f0 tx=0x7f253c00eeb0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.481+0000 7f254be00640 1 -- 172.21.15.31:0/2894546796 shutdown_connections 2024-04-25T01:15:49.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.481+0000 7f254be00640 1 --2- 172.21.15.31:0/2894546796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f254c107370 0x7f254c1a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.481+0000 7f254be00640 1 --2- 172.21.15.31:0/2894546796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f254c104a40 0x7f254c19a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.481+0000 7f254be00640 1 --2- 172.21.15.31:0/2894546796 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f25300a2b90 0x7f25300a5050 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.481+0000 7f254be00640 1 --2- 172.21.15.31:0/2894546796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f254c102110 0x7f254c199b80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.481+0000 7f254be00640 1 -- 172.21.15.31:0/2894546796 >> 172.21.15.31:0/2894546796 conn(0x7f254c0fbae0 msgr2=0x7f254c1018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:49.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.481+0000 7f254be00640 1 -- 172.21.15.31:0/2894546796 shutdown_connections 2024-04-25T01:15:49.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.481+0000 7f254be00640 1 -- 172.21.15.31:0/2894546796 wait complete. 2024-04-25T01:15:49.482 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:49 smithi031 ceph-mon[28053]: pgmap v387: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:49.482 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:49 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1382184806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:15:49.482 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:49 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2038399498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:15:49.486 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T01:15:49.594 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:49 smithi169 ceph-mon[32759]: pgmap v387: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:49.595 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:49 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1382184806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:15:49.595 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:49 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2038399498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:15:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.709+0000 7f40d5a00640 1 -- 172.21.15.31:0/1598408061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40d0104b90 msgr2=0x7f40d0105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.709+0000 7f40d5a00640 1 --2- 172.21.15.31:0/1598408061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40d0104b90 0x7f40d0105010 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f40b8009900 tx=0x7f40b802a3f0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.709+0000 7f40d5a00640 1 -- 172.21.15.31:0/1598408061 shutdown_connections 2024-04-25T01:15:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.709+0000 7f40d5a00640 1 --2- 172.21.15.31:0/1598408061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d0105550 0x7f40d010bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.709+0000 7f40d5a00640 1 --2- 172.21.15.31:0/1598408061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40d0104b90 0x7f40d0105010 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.709+0000 7f40d5a00640 1 --2- 172.21.15.31:0/1598408061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40d01039a0 0x7f40d0103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.709+0000 7f40d5a00640 1 -- 172.21.15.31:0/1598408061 >> 172.21.15.31:0/1598408061 conn(0x7f40d00fef30 msgr2=0x7f40d0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:49.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.709+0000 7f40d5a00640 1 -- 172.21.15.31:0/1598408061 shutdown_connections 2024-04-25T01:15:49.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.709+0000 7f40d5a00640 1 -- 172.21.15.31:0/1598408061 wait complete. 2024-04-25T01:15:49.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:49 smithi040 ceph-mon[32518]: pgmap v387: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:49.697 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:49 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1382184806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:15:49.697 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:49 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2038399498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:15:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.710+0000 7f40d5a00640 1 Processor -- start 2024-04-25T01:15:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.711+0000 7f40d5a00640 1 -- start start 2024-04-25T01:15:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.711+0000 7f40d5a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40d01039a0 0x7f40d0199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.711+0000 7f40d5a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d0104b90 0x7f40d019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.711+0000 7f40d5a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40d0105550 0x7f40d01a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.711+0000 7f40d5a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f40d010f670 con 0x7f40d0105550 2024-04-25T01:15:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.711+0000 7f40d5a00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f40d010f4f0 con 0x7f40d01039a0 2024-04-25T01:15:49.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.711+0000 7f40d5a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f40d010f7f0 con 0x7f40d0104b90 2024-04-25T01:15:49.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.711+0000 7f40cea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40d01039a0 0x7f40d0199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.711+0000 7f40cf400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40d0105550 0x7f40d01a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.711+0000 7f40cea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40d01039a0 0x7f40d0199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:37546/0 (socket says 172.21.15.31:37546) 2024-04-25T01:15:49.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.711+0000 7f40cea00640 1 -- 172.21.15.31:0/3049067468 learned_addr learned my addr 172.21.15.31:0/3049067468 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:49.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.711+0000 7f40ce000640 1 --2- 172.21.15.31:0/3049067468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d0104b90 0x7f40d019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.712+0000 7f40cea00640 1 -- 172.21.15.31:0/3049067468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d0104b90 msgr2=0x7f40d019a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.712+0000 7f40cea00640 1 --2- 172.21.15.31:0/3049067468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d0104b90 0x7f40d019a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.712+0000 7f40cea00640 1 -- 172.21.15.31:0/3049067468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40d0105550 msgr2=0x7f40d01a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.712+0000 7f40cea00640 1 --2- 172.21.15.31:0/3049067468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40d0105550 0x7f40d01a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.712+0000 7f40cea00640 1 -- 172.21.15.31:0/3049067468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d01a1900 con 0x7f40d01039a0 2024-04-25T01:15:49.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.712+0000 7f40ce000640 1 --2- 172.21.15.31:0/3049067468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d0104b90 0x7f40d019a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:15:49.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.712+0000 7f40cea00640 1 --2- 172.21.15.31:0/3049067468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40d01039a0 0x7f40d0199cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009d20 tx=0x7f40c000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:49.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.712+0000 7f40c7400640 1 -- 172.21.15.31:0/3049067468 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c0010630 con 0x7f40d01039a0 2024-04-25T01:15:49.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.712+0000 7f40c7400640 1 -- 172.21.15.31:0/3049067468 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40c0010cb0 con 0x7f40d01039a0 2024-04-25T01:15:49.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.712+0000 7f40c7400640 1 -- 172.21.15.31:0/3049067468 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c0018de0 con 0x7f40d01039a0 2024-04-25T01:15:49.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.712+0000 7f40d5a00640 1 -- 172.21.15.31:0/3049067468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40d01a1bf0 con 0x7f40d01039a0 2024-04-25T01:15:49.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.713+0000 7f40d5a00640 1 -- 172.21.15.31:0/3049067468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40d01a2160 con 0x7f40d01039a0 2024-04-25T01:15:49.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.716+0000 7f40d89aa740 1 Processor -- start 2024-04-25T01:15:49.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.716+0000 7f40d89aa740 1 -- start start 2024-04-25T01:15:49.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.716+0000 7f40c7400640 1 -- 172.21.15.31:0/3049067468 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f40c001f020 con 0x7f40d01039a0 2024-04-25T01:15:49.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.716+0000 7f40d89aa740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cef760 0x5639f2cefb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.716+0000 7f40d89aa740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639f2cf0150 0x5639f2cf4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.716+0000 7f40d89aa740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639f2cf52c0 0x5639f2cf7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.716+0000 7f40d89aa740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5639f2befed0 con 0x5639f2cef760 2024-04-25T01:15:49.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.716+0000 7f40d89aa740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5639f2c52df0 con 0x5639f2cf0150 2024-04-25T01:15:49.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.716+0000 7f40d89aa740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5639f2bfb8d0 con 0x5639f2cf52c0 2024-04-25T01:15:49.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.716+0000 7f40cf400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cef760 0x5639f2cefb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.716+0000 7f40cf400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cef760 0x5639f2cefb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48828/0 (socket says 172.21.15.31:48828) 2024-04-25T01:15:49.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.716+0000 7f40cf400640 1 -- 172.21.15.31:0/1134888719 learned_addr learned my addr 172.21.15.31:0/1134888719 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:49.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.716+0000 7f40cea00640 1 --2- 172.21.15.31:0/1134888719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639f2cf0150 0x5639f2cf4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.717+0000 7f40ce000640 1 --2- 172.21.15.31:0/1134888719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639f2cf52c0 0x5639f2cf7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.717+0000 7f40cf400640 1 -- 172.21.15.31:0/1134888719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639f2cf52c0 msgr2=0x5639f2cf7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:49.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.717+0000 7f40cf400640 1 --2- 172.21.15.31:0/1134888719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639f2cf52c0 0x5639f2cf7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.717+0000 7f40cf400640 1 -- 172.21.15.31:0/1134888719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639f2cf0150 msgr2=0x5639f2cf4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:49.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.717+0000 7f40cf400640 1 --2- 172.21.15.31:0/1134888719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639f2cf0150 0x5639f2cf4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.717+0000 7f40cf400640 1 -- 172.21.15.31:0/1134888719 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639f29672d0 con 0x5639f2cef760 2024-04-25T01:15:49.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.717+0000 7f40cf400640 1 --2- 172.21.15.31:0/1134888719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cef760 0x5639f2cefb80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f40c8009ca0 tx=0x7f40c8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dcd89e14af9a680 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:49.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.717+0000 7f40bfe00640 1 -- 172.21.15.31:0/1134888719 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c801e070 con 0x5639f2cef760 2024-04-25T01:15:49.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.718+0000 7f40bfe00640 1 -- 172.21.15.31:0/1134888719 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40c8009e50 con 0x5639f2cef760 2024-04-25T01:15:49.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.718+0000 7f40bfe00640 1 -- 172.21.15.31:0/1134888719 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c800ea00 con 0x5639f2cef760 2024-04-25T01:15:49.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.718+0000 7f40c7400640 1 --2- 172.21.15.31:0/3049067468 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f40b40a2c10 0x7f40b40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.718+0000 7f40c7400640 1 -- 172.21.15.31:0/3049067468 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f40c00cf500 con 0x7f40d01039a0 2024-04-25T01:15:49.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.718+0000 7f40cea00640 1 --2- 172.21.15.31:0/3049067468 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f40b40a2c10 0x7f40b40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.718+0000 7f40d89aa740 1 -- 172.21.15.31:0/1134888719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cef760 msgr2=0x5639f2cefb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.718+0000 7f40d89aa740 1 --2- 172.21.15.31:0/1134888719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cef760 0x5639f2cefb80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f40c8009ca0 tx=0x7f40c8011810 comp rx=0 tx=0).stop 2024-04-25T01:15:49.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.718+0000 7f40d89aa740 1 -- 172.21.15.31:0/1134888719 shutdown_connections 2024-04-25T01:15:49.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.718+0000 7f40d89aa740 1 --2- 172.21.15.31:0/1134888719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639f2cf52c0 0x5639f2cf7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.718+0000 7f40d89aa740 1 --2- 172.21.15.31:0/1134888719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639f2cf0150 0x5639f2cf4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.718+0000 7f40d89aa740 1 --2- 172.21.15.31:0/1134888719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cef760 0x5639f2cefb80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.718+0000 7f40d89aa740 1 -- 172.21.15.31:0/1134888719 >> 172.21.15.31:0/1134888719 conn(0x5639f2ceadd0 msgr2=0x5639f2ceb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:49.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.719+0000 7f40d89aa740 1 -- 172.21.15.31:0/1134888719 shutdown_connections 2024-04-25T01:15:49.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.719+0000 7f40d89aa740 1 -- 172.21.15.31:0/1134888719 wait complete. 2024-04-25T01:15:49.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.719+0000 7f40cea00640 1 --2- 172.21.15.31:0/3049067468 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f40b40a2c10 0x7f40b40a50d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x5639f2cf0550 tx=0x7f40c00d2f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:49.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.719+0000 7f40d89aa740 1 -- start start 2024-04-25T01:15:49.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.719+0000 7f40d89aa740 1 Processor -- start 2024-04-25T01:15:49.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.720+0000 7f40d89aa740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639f2cef760 0x5639f2cf9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.720+0000 7f40d89aa740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cf0150 0x5639f2cf9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.720+0000 7f40d89aa740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639f2cf52c0 0x5639f2d00fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.720+0000 7f40d89aa740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5639f2c0ed20 con 0x5639f2cf0150 2024-04-25T01:15:49.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.720+0000 7f40d89aa740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5639f2c4edb0 con 0x5639f2cf52c0 2024-04-25T01:15:49.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.720+0000 7f40d89aa740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5639f2cf8270 con 0x5639f2cef760 2024-04-25T01:15:49.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.720+0000 7f40d89aa740 10 client.0 _ll_register_callbacks cb 0x5639f2c49f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:15:49.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.720+0000 7f40ce000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cf0150 0x5639f2cf9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.720+0000 7f40ce000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cf0150 0x5639f2cf9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48838/0 (socket says 172.21.15.31:48838) 2024-04-25T01:15:49.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.720+0000 7f40ce000640 1 -- 172.21.15.31:0/1992928007 learned_addr learned my addr 172.21.15.31:0/1992928007 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:49.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.720+0000 7f40cf400640 1 --2- 172.21.15.31:0/1992928007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639f2cef760 0x5639f2cf9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.720+0000 7f40cf400640 1 --2- 172.21.15.31:0/1992928007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639f2cf52c0 0x5639f2d00fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.721+0000 7f40ce000640 1 -- 172.21.15.31:0/1992928007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639f2cef760 msgr2=0x5639f2cf9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:49.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.721+0000 7f40ce000640 1 --2- 172.21.15.31:0/1992928007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639f2cef760 0x5639f2cf9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.721+0000 7f40ce000640 1 -- 172.21.15.31:0/1992928007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639f2cf52c0 msgr2=0x5639f2d00fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:49.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.721+0000 7f40ce000640 1 --2- 172.21.15.31:0/1992928007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639f2cf52c0 0x5639f2d00fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.721+0000 7f40ce000640 1 -- 172.21.15.31:0/1992928007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639f2a316b0 con 0x5639f2cf0150 2024-04-25T01:15:49.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.721+0000 7f40ce000640 1 --2- 172.21.15.31:0/1992928007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cf0150 0x5639f2cf9f70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f40b8005ab0 tx=0x7f40b8005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd3399395867590 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:49.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.721+0000 7f40bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T01:15:49.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.721+0000 7f40bcc00640 1 -- 172.21.15.31:0/1992928007 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8035040 con 0x5639f2cf0150 2024-04-25T01:15:49.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.721+0000 7f40bcc00640 1 -- 172.21.15.31:0/1992928007 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f40b80318b0 con 0x5639f2cf0150 2024-04-25T01:15:49.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.721+0000 7f40d89aa740 1 -- 172.21.15.31:0/1992928007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40c8009e50 con 0x5639f2cf0150 2024-04-25T01:15:49.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.722+0000 7f40bcc00640 1 -- 172.21.15.31:0/1992928007 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f40b8031ab0 con 0x5639f2cf0150 2024-04-25T01:15:49.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.722+0000 7f40d89aa740 10 client.15084 fetch_fsmap learned FSMap version 23 2024-04-25T01:15:49.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.722+0000 7f40d89aa740 1 -- 172.21.15.31:0/1992928007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5639f2d04390 con 0x5639f2cf0150 2024-04-25T01:15:49.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.722+0000 7f40bcc00640 1 -- 172.21.15.31:0/1992928007 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40c8009e50 con 0x5639f2cf0150 2024-04-25T01:15:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.722+0000 7f40bcc00640 1 -- 172.21.15.31:0/1992928007 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f40b80035f0 con 0x5639f2cf0150 2024-04-25T01:15:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.722+0000 7f40d89aa740 10 client.15084 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T01:15:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.722+0000 7f40d89aa740 10 client.15084 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T01:15:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.722+0000 7f40d89aa740 20 client.15084 populate_metadata read hostname 'smithi031' 2024-04-25T01:15:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.722+0000 7f40d89aa740 1 --2- 172.21.15.31:0/1992928007 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5639f2d06a10 0x5639f2d07e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.722+0000 7f40d89aa740 4 client.15084 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:15:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.722+0000 7f40d89aa740 1 -- 172.21.15.31:0/1992928007 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639f2c60330 con 0x5639f2d06a10 2024-04-25T01:15:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.723+0000 7f40cf400640 1 --2- 172.21.15.31:0/1992928007 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5639f2d06a10 0x5639f2d07e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.723+0000 7f40cf400640 1 --2- 172.21.15.31:0/1992928007 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5639f2d06a10 0x5639f2d07e80 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58c6aa8352aff62c server_cookie=6ddf9f72b13d678b in_seq=0 out_seq=0 2024-04-25T01:15:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.723+0000 7f40bcc00640 10 client.15084 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:15:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.724+0000 7f40bcc00640 1 -- 172.21.15.31:0/1992928007 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5639f2c60330 con 0x5639f2d06a10 2024-04-25T01:15:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.724+0000 7f40bcc00640 10 client.15084 handle_command_reply: tid=0 2024-04-25T01:15:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.724+0000 7f40d89aa740 1 client.15084 shutdown 2024-04-25T01:15:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.724+0000 7f40d89aa740 1 -- 172.21.15.31:0/1992928007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cf0150 msgr2=0x5639f2cf9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.724+0000 7f40d89aa740 1 --2- 172.21.15.31:0/1992928007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cf0150 0x5639f2cf9f70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f40b8005ab0 tx=0x7f40b8005270 comp rx=0 tx=0).stop 2024-04-25T01:15:49.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.724+0000 7f40d89aa740 1 -- 172.21.15.31:0/1992928007 shutdown_connections 2024-04-25T01:15:49.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.724+0000 7f40d89aa740 1 --2- 172.21.15.31:0/1992928007 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5639f2d06a10 0x5639f2d07e80 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.724+0000 7f40d89aa740 1 --2- 172.21.15.31:0/1992928007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639f2cf52c0 0x5639f2d00fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.724+0000 7f40d89aa740 1 --2- 172.21.15.31:0/1992928007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cf0150 0x5639f2cf9f70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.724+0000 7f40d89aa740 1 --2- 172.21.15.31:0/1992928007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639f2cef760 0x5639f2cf9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.724+0000 7f40bcc00640 0 client.15084 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:15:49.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.724+0000 7f40d89aa740 1 -- 172.21.15.31:0/1992928007 >> 172.21.15.31:0/1992928007 conn(0x5639f2ceadd0 msgr2=0x5639f2ce9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:49.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.725+0000 7f40d89aa740 1 -- 172.21.15.31:0/1992928007 shutdown_connections 2024-04-25T01:15:49.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.725+0000 7f40d89aa740 1 -- 172.21.15.31:0/1992928007 wait complete. 2024-04-25T01:15:49.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.725+0000 7f40d89aa740 20 client.15084 trim_cache size 0 max 16384 2024-04-25T01:15:49.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.745+0000 7f40d89aa740 1 Processor -- start 2024-04-25T01:15:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.745+0000 7f40d89aa740 1 -- start start 2024-04-25T01:15:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.745+0000 7f40d89aa740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639f2cef760 0x5639f2cf4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.745+0000 7f40d89aa740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639f2cf0150 0x5639f2d00f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.745+0000 7f40d89aa740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cf52c0 0x5639f2ceb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.745+0000 7f40d89aa740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5639f2d061c0 con 0x5639f2cf52c0 2024-04-25T01:15:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.745+0000 7f40d89aa740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5639f2d06890 con 0x5639f2cef760 2024-04-25T01:15:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.745+0000 7f40d89aa740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5639f2d06f60 con 0x5639f2cf0150 2024-04-25T01:15:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.746+0000 7f40cf400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639f2cef760 0x5639f2cf4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.746+0000 7f40ce000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639f2cf0150 0x5639f2d00f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.746+0000 7f40ce000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639f2cf0150 0x5639f2d00f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:40066/0 (socket says 172.21.15.31:40066) 2024-04-25T01:15:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.746+0000 7f40ce000640 1 -- 172.21.15.31:0/1088045359 learned_addr learned my addr 172.21.15.31:0/1088045359 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.746+0000 7f40cf400640 1 --2- 172.21.15.31:0/1088045359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cf52c0 0x5639f2ceb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.746+0000 7f40ce000640 1 -- 172.21.15.31:0/1088045359 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639f2cef760 msgr2=0x5639f2cf4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:49.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.746+0000 7f40ce000640 1 --2- 172.21.15.31:0/1088045359 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639f2cef760 0x5639f2cf4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.746+0000 7f40ce000640 1 -- 172.21.15.31:0/1088045359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cf52c0 msgr2=0x5639f2ceb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:49.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.746+0000 7f40ce000640 1 --2- 172.21.15.31:0/1088045359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cf52c0 0x5639f2ceb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.746+0000 7f40ce000640 1 -- 172.21.15.31:0/1088045359 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639f2cf09e0 con 0x5639f2cf0150 2024-04-25T01:15:49.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.746+0000 7f40cf400640 1 --2- 172.21.15.31:0/1088045359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cf52c0 0x5639f2ceb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:15:49.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.746+0000 7f40ce000640 1 --2- 172.21.15.31:0/1088045359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639f2cf0150 0x5639f2d00f70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f40b8031440 tx=0x7f40b8005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f099ab49af15e8ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:49.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.747+0000 7f40bf400640 1 -- 172.21.15.31:0/1088045359 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8035040 con 0x5639f2cf0150 2024-04-25T01:15:49.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.747+0000 7f40bf400640 1 -- 172.21.15.31:0/1088045359 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40b8040070 con 0x5639f2cf0150 2024-04-25T01:15:49.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.747+0000 7f40bf400640 1 -- 172.21.15.31:0/1088045359 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b80316b0 con 0x5639f2cf0150 2024-04-25T01:15:49.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.747+0000 7f40d89aa740 1 -- 172.21.15.31:0/1088045359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639f2cf0150 msgr2=0x5639f2d00f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.747+0000 7f40d89aa740 1 --2- 172.21.15.31:0/1088045359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639f2cf0150 0x5639f2d00f70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f40b8031440 tx=0x7f40b8005970 comp rx=0 tx=0).stop 2024-04-25T01:15:49.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.747+0000 7f40d89aa740 1 -- 172.21.15.31:0/1088045359 shutdown_connections 2024-04-25T01:15:49.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.747+0000 7f40d89aa740 1 --2- 172.21.15.31:0/1088045359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cf52c0 0x5639f2ceb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.747+0000 7f40d89aa740 1 --2- 172.21.15.31:0/1088045359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639f2cf0150 0x5639f2d00f70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.747+0000 7f40d89aa740 1 --2- 172.21.15.31:0/1088045359 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639f2cef760 0x5639f2cf4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.747+0000 7f40d89aa740 1 -- 172.21.15.31:0/1088045359 >> 172.21.15.31:0/1088045359 conn(0x5639f2ceadd0 msgr2=0x5639f2ce5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:49.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.747+0000 7f40d89aa740 1 -- 172.21.15.31:0/1088045359 shutdown_connections 2024-04-25T01:15:49.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.747+0000 7f40d89aa740 1 -- 172.21.15.31:0/1088045359 wait complete. 2024-04-25T01:15:49.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.748+0000 7f40d89aa740 1 -- start start 2024-04-25T01:15:49.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.748+0000 7f40d89aa740 1 Processor -- start 2024-04-25T01:15:49.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.749+0000 7f40d89aa740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639f2cef760 0x5639f2cfc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.749+0000 7f40d89aa740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cf0150 0x5639f2cfcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.749+0000 7f40d89aa740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639f2cf52c0 0x5639f2cffc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.749+0000 7f40d89aa740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5639f2d07d00 con 0x5639f2cf0150 2024-04-25T01:15:49.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.749+0000 7f40d89aa740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5639f2d07630 con 0x5639f2cf52c0 2024-04-25T01:15:49.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.749+0000 7f40d89aa740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5639f2d083d0 con 0x5639f2cef760 2024-04-25T01:15:49.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.749+0000 7f40d89aa740 10 client.0 _ll_register_callbacks cb 0x7f40c8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:15:49.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.749+0000 7f40ce000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cf0150 0x5639f2cfcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.749+0000 7f40ce000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cf0150 0x5639f2cfcbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48854/0 (socket says 172.21.15.31:48854) 2024-04-25T01:15:49.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.749+0000 7f40ce000640 1 -- 172.21.15.31:0/60188422 learned_addr learned my addr 172.21.15.31:0/60188422 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:49.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.749+0000 7f40cf400640 1 --2- 172.21.15.31:0/60188422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639f2cef760 0x5639f2cfc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.749+0000 7f40cf400640 1 --2- 172.21.15.31:0/60188422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639f2cf52c0 0x5639f2cffc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.749+0000 7f40ce000640 1 -- 172.21.15.31:0/60188422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639f2cef760 msgr2=0x5639f2cfc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:49.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.749+0000 7f40ce000640 1 --2- 172.21.15.31:0/60188422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639f2cef760 0x5639f2cfc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.749+0000 7f40ce000640 1 -- 172.21.15.31:0/60188422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639f2cf52c0 msgr2=0x5639f2cffc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:15:49.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.749+0000 7f40ce000640 1 --2- 172.21.15.31:0/60188422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639f2cf52c0 0x5639f2cffc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.749+0000 7f40ce000640 1 -- 172.21.15.31:0/60188422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639f2d001f0 con 0x5639f2cf0150 2024-04-25T01:15:49.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.749+0000 7f40cf400640 1 --2- 172.21.15.31:0/60188422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639f2cf52c0 0x5639f2cffc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:49.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.750+0000 7f40ce000640 1 --2- 172.21.15.31:0/60188422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cf0150 0x5639f2cfcbe0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f40b8004650 tx=0x7f40b80044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70779ba0373797db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:49.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.750+0000 7f40bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T01:15:49.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.750+0000 7f40bcc00640 1 -- 172.21.15.31:0/60188422 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8035050 con 0x5639f2cf0150 2024-04-25T01:15:49.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.750+0000 7f40bcc00640 1 -- 172.21.15.31:0/60188422 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f40b80035f0 con 0x5639f2cf0150 2024-04-25T01:15:49.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.750+0000 7f40d89aa740 1 -- 172.21.15.31:0/60188422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40b8040070 con 0x5639f2cf0150 2024-04-25T01:15:49.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.750+0000 7f40bcc00640 1 -- 172.21.15.31:0/60188422 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f40b8003e90 con 0x5639f2cf0150 2024-04-25T01:15:49.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.750+0000 7f40d89aa740 10 client.15090 fetch_fsmap learned FSMap version 23 2024-04-25T01:15:49.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.750+0000 7f40d89aa740 1 -- 172.21.15.31:0/60188422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5639f2d6ca20 con 0x5639f2cf0150 2024-04-25T01:15:49.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.751+0000 7f40bcc00640 1 -- 172.21.15.31:0/60188422 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40b8040070 con 0x5639f2cf0150 2024-04-25T01:15:49.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.751+0000 7f40bcc00640 1 -- 172.21.15.31:0/60188422 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f40b8045590 con 0x5639f2cf0150 2024-04-25T01:15:49.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.751+0000 7f40d89aa740 10 client.15090 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T01:15:49.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.751+0000 7f40d89aa740 10 client.15090 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T01:15:49.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.751+0000 7f40d89aa740 20 client.15090 populate_metadata read hostname 'smithi031' 2024-04-25T01:15:49.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.751+0000 7f40d89aa740 1 --2- 172.21.15.31:0/60188422 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5639f2d71470 0x5639f2d70180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:49.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.751+0000 7f40d89aa740 4 client.15090 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T01:15:49.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.751+0000 7f40d89aa740 1 -- 172.21.15.31:0/60188422 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "status"}) -- 0x5639f2c5a680 con 0x5639f2d71470 2024-04-25T01:15:49.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.751+0000 7f40cf400640 1 --2- 172.21.15.31:0/60188422 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5639f2d71470 0x5639f2d70180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:49.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.752+0000 7f40cf400640 1 --2- 172.21.15.31:0/60188422 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5639f2d71470 0x5639f2d70180 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bb4828c20fc62a8 server_cookie=115927d9a32a24d1 in_seq=0 out_seq=0 2024-04-25T01:15:49.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.752+0000 7f40bcc00640 10 client.15090 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:15:49.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.752+0000 7f40bcc00640 1 -- 172.21.15.31:0/60188422 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5639f2c5a680 con 0x5639f2d71470 2024-04-25T01:15:49.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.752+0000 7f40bcc00640 10 client.15090 handle_command_reply: tid=0 2024-04-25T01:15:49.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.752+0000 7f40d89aa740 1 client.15090 shutdown 2024-04-25T01:15:49.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.752+0000 7f40d89aa740 1 -- 172.21.15.31:0/60188422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cf0150 msgr2=0x5639f2cfcbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.752+0000 7f40d89aa740 1 --2- 172.21.15.31:0/60188422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cf0150 0x5639f2cfcbe0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f40b8004650 tx=0x7f40b80044d0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.752+0000 7f40d89aa740 1 -- 172.21.15.31:0/60188422 shutdown_connections 2024-04-25T01:15:49.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.752+0000 7f40d89aa740 1 --2- 172.21.15.31:0/60188422 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5639f2d71470 0x5639f2d70180 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.752+0000 7f40d89aa740 1 --2- 172.21.15.31:0/60188422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5639f2cf52c0 0x5639f2cffc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.752+0000 7f40d89aa740 1 --2- 172.21.15.31:0/60188422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639f2cf0150 0x5639f2cfcbe0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.752+0000 7f40d89aa740 1 --2- 172.21.15.31:0/60188422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639f2cef760 0x5639f2cfc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.752+0000 7f40bcc00640 0 client.15090 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:15:49.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.752+0000 7f40d89aa740 1 -- 172.21.15.31:0/60188422 >> 172.21.15.31:0/60188422 conn(0x5639f2ceadd0 msgr2=0x5639f2ceb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:49.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.753+0000 7f40d89aa740 1 -- 172.21.15.31:0/60188422 shutdown_connections 2024-04-25T01:15:49.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.753+0000 7f40d89aa740 1 -- 172.21.15.31:0/60188422 wait complete. 2024-04-25T01:15:49.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.753+0000 7f40d89aa740 20 client.15090 trim_cache size 0 max 16384 2024-04-25T01:15:49.744 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T01:15:49.744 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "95849b94-029f-11ef-bc93-c7b262605968", 2024-04-25T01:15:49.744 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2024-04-25T01:15:49.744 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24596, 2024-04-25T01:15:49.744 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-04-25T01:15:49.744 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-04-25T01:15:49.744 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-04-25T01:15:49.745 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 138.20229371299999, 2024-04-25T01:15:49.745 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 23, 2024-04-25T01:15:49.745 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 89, 2024-04-25T01:15:49.745 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 87, 2024-04-25T01:15:49.745 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 145.576551661 2024-04-25T01:15:49.745 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T01:15:49.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.753+0000 7f40d5a00640 1 -- 172.21.15.31:0/3049067468 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f40b40a2c10 msgr2=0x7f40b40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.753+0000 7f40d5a00640 1 --2- 172.21.15.31:0/3049067468 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f40b40a2c10 0x7f40b40a50d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x5639f2cf0550 tx=0x7f40c00d2f40 comp rx=0 tx=0).stop 2024-04-25T01:15:49.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.753+0000 7f40d5a00640 1 -- 172.21.15.31:0/3049067468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40d01039a0 msgr2=0x7f40d0199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:49.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.753+0000 7f40d5a00640 1 --2- 172.21.15.31:0/3049067468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40d01039a0 0x7f40d0199cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009d20 tx=0x7f40c000c6a0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.754+0000 7f40d5a00640 1 -- 172.21.15.31:0/3049067468 shutdown_connections 2024-04-25T01:15:49.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.754+0000 7f40d5a00640 1 --2- 172.21.15.31:0/3049067468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40d0105550 0x7f40d01a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.754+0000 7f40d5a00640 1 --2- 172.21.15.31:0/3049067468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d0104b90 0x7f40d019a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.754+0000 7f40d5a00640 1 --2- 172.21.15.31:0/3049067468 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f40b40a2c10 0x7f40b40a50d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.754+0000 7f40d5a00640 1 --2- 172.21.15.31:0/3049067468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40d01039a0 0x7f40d0199cc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:49.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.754+0000 7f40d5a00640 1 -- 172.21.15.31:0/3049067468 >> 172.21.15.31:0/3049067468 conn(0x7f40d00fef30 msgr2=0x7f40d0107e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:49.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.754+0000 7f40d5a00640 1 -- 172.21.15.31:0/3049067468 shutdown_connections 2024-04-25T01:15:49.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:49.754+0000 7f40d5a00640 1 -- 172.21.15.31:0/3049067468 wait complete. 2024-04-25T01:15:49.758 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T01:15:49.767 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T01:15:49.767 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T01:15:49.768 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T01:15:49.768 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T01:15:49.850 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T01:15:49.850 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T01:15:49.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:15:49.850 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-04-25T01:15:49.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:15:49.912 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2024-04-25T01:15:50.009 INFO:teuthology.orchestra.run.smithi031.stderr:Cannot find device "ceph-brx" 2024-04-25T01:15:50.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:15:50.014 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-25T01:15:50.014 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T01:15:50.014 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T01:15:50.014 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T01:15:50.467 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:15:51.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:51 smithi031 ceph-mon[28053]: pgmap v388: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:51.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:51 smithi169 ceph-mon[32759]: pgmap v388: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:51.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:51 smithi040 ceph-mon[32518]: pgmap v388: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:52.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.844+0000 7f1c93c00700 1 -- 172.21.15.31:0/403264827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c8c074460 msgr2=0x7f1c8c074840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:52.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.844+0000 7f1c93c00700 1 --2- 172.21.15.31:0/403264827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c8c074460 0x7f1c8c074840 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009990 tx=0x7f1c80009ca0 comp rx=0 tx=0).stop 2024-04-25T01:15:52.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.844+0000 7f1c93c00700 1 -- 172.21.15.31:0/403264827 shutdown_connections 2024-04-25T01:15:52.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.844+0000 7f1c93c00700 1 --2- 172.21.15.31:0/403264827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c8c074460 0x7f1c8c074840 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:52.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.844+0000 7f1c93c00700 1 --2- 172.21.15.31:0/403264827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c8c10ec90 0x7f1c8c111080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:52.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.844+0000 7f1c93c00700 1 --2- 172.21.15.31:0/403264827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c8c074d80 0x7f1c8c10e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:52.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.844+0000 7f1c93c00700 1 -- 172.21.15.31:0/403264827 >> 172.21.15.31:0/403264827 conn(0x7f1c8c079390 msgr2=0x7f1c8c0797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:52.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.844+0000 7f1c93c00700 1 -- 172.21.15.31:0/403264827 shutdown_connections 2024-04-25T01:15:52.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.844+0000 7f1c93c00700 1 -- 172.21.15.31:0/403264827 wait complete. 2024-04-25T01:15:52.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.846+0000 7f1c93c00700 1 Processor -- start 2024-04-25T01:15:52.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.846+0000 7f1c93c00700 1 -- start start 2024-04-25T01:15:52.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.846+0000 7f1c93c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c8c074460 0x7f1c8c1a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:52.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.846+0000 7f1c93c00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c8c074d80 0x7f1c8c1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:52.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.847+0000 7f1c93c00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c8c10ec90 0x7f1c8c19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:52.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.847+0000 7f1c93c00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c1147a0 con 0x7f1c8c10ec90 2024-04-25T01:15:52.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.847+0000 7f1c93c00700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c114620 con 0x7f1c8c074d80 2024-04-25T01:15:52.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.847+0000 7f1c93c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c114920 con 0x7f1c8c074460 2024-04-25T01:15:52.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.847+0000 7f1c8be00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c8c074d80 0x7f1c8c1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:52.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.847+0000 7f1c91c00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c8c10ec90 0x7f1c8c19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:52.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.847+0000 7f1c8be00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c8c074d80 0x7f1c8c1a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:37582/0 (socket says 172.21.15.31:37582) 2024-04-25T01:15:52.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.847+0000 7f1c8be00700 1 -- 172.21.15.31:0/3494831 learned_addr learned my addr 172.21.15.31:0/3494831 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:52.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.847+0000 7f1c91200700 1 --2- 172.21.15.31:0/3494831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c8c074460 0x7f1c8c1a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:52.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.847+0000 7f1c8be00700 1 -- 172.21.15.31:0/3494831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c8c074460 msgr2=0x7f1c8c1a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:52.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.847+0000 7f1c8be00700 1 --2- 172.21.15.31:0/3494831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c8c074460 0x7f1c8c1a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:52.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.847+0000 7f1c8be00700 1 -- 172.21.15.31:0/3494831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c8c10ec90 msgr2=0x7f1c8c19a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:52.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.847+0000 7f1c8be00700 1 --2- 172.21.15.31:0/3494831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c8c10ec90 0x7f1c8c19a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:52.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.847+0000 7f1c8be00700 1 -- 172.21.15.31:0/3494831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c8c19b0c0 con 0x7f1c8c074d80 2024-04-25T01:15:52.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.848+0000 7f1c8be00700 1 --2- 172.21.15.31:0/3494831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c8c074d80 0x7f1c8c1a0ce0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1c7800e9c0 tx=0x7f1c7800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:52.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.848+0000 7f1c89600700 1 -- 172.21.15.31:0/3494831 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7800cbd0 con 0x7f1c8c074d80 2024-04-25T01:15:52.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.848+0000 7f1c89600700 1 -- 172.21.15.31:0/3494831 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c7800ee60 con 0x7f1c8c074d80 2024-04-25T01:15:52.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.848+0000 7f1c93c00700 1 -- 172.21.15.31:0/3494831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c8c19b3b0 con 0x7f1c8c074d80 2024-04-25T01:15:52.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.848+0000 7f1c89600700 1 -- 172.21.15.31:0/3494831 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c78005630 con 0x7f1c8c074d80 2024-04-25T01:15:52.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.848+0000 7f1c93c00700 1 -- 172.21.15.31:0/3494831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c8c1a7670 con 0x7f1c8c074d80 2024-04-25T01:15:52.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.849+0000 7f1c93c00700 1 -- 172.21.15.31:0/3494831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c8c068f20 con 0x7f1c8c074d80 2024-04-25T01:15:52.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.851+0000 7f1c89600700 1 -- 172.21.15.31:0/3494831 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1c78004750 con 0x7f1c8c074d80 2024-04-25T01:15:52.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.853+0000 7f1c89600700 1 --2- 172.21.15.31:0/3494831 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1c740a2c20 0x7f1c740a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:52.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.853+0000 7f1c89600700 1 -- 172.21.15.31:0/3494831 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f1c780cf2c0 con 0x7f1c8c074d80 2024-04-25T01:15:52.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.853+0000 7f1c91200700 1 --2- 172.21.15.31:0/3494831 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1c740a2c20 0x7f1c740a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.854+0000 7f1c91200700 1 --2- 172.21.15.31:0/3494831 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1c740a2c20 0x7f1c740a50e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c19baa0 tx=0x7f1c80005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:52.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:52.857+0000 7f1c89600700 1 -- 172.21.15.31:0/3494831 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1c7809c210 con 0x7f1c8c074d80 2024-04-25T01:15:53.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:53.135+0000 7f1c93c00700 1 -- 172.21.15.31:0/3494831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1c8c19be90 con 0x7f1c8c074d80 2024-04-25T01:15:53.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:53.136+0000 7f1c89600700 1 -- 172.21.15.31:0/3494831 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7f1c7809bb10 con 0x7f1c8c074d80 2024-04-25T01:15:53.121 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T01:15:53.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:53.140+0000 7f1c93c00700 1 -- 172.21.15.31:0/3494831 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1c740a2c20 msgr2=0x7f1c740a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:53.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:53.140+0000 7f1c93c00700 1 --2- 172.21.15.31:0/3494831 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1c740a2c20 0x7f1c740a50e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c19baa0 tx=0x7f1c80005960 comp rx=0 tx=0).stop 2024-04-25T01:15:53.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:53.140+0000 7f1c93c00700 1 -- 172.21.15.31:0/3494831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c8c074d80 msgr2=0x7f1c8c1a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:53.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:53.140+0000 7f1c93c00700 1 --2- 172.21.15.31:0/3494831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c8c074d80 0x7f1c8c1a0ce0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1c7800e9c0 tx=0x7f1c7800ecd0 comp rx=0 tx=0).stop 2024-04-25T01:15:53.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:53.141+0000 7f1c93c00700 1 -- 172.21.15.31:0/3494831 shutdown_connections 2024-04-25T01:15:53.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:53.141+0000 7f1c93c00700 1 --2- 172.21.15.31:0/3494831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c8c10ec90 0x7f1c8c19a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:53.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:53.141+0000 7f1c93c00700 1 --2- 172.21.15.31:0/3494831 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1c740a2c20 0x7f1c740a50e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:53.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:53.141+0000 7f1c93c00700 1 --2- 172.21.15.31:0/3494831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c8c074460 0x7f1c8c1a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:53.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:53.141+0000 7f1c93c00700 1 --2- 172.21.15.31:0/3494831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c8c074d80 0x7f1c8c1a0ce0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:53.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:53.141+0000 7f1c93c00700 1 -- 172.21.15.31:0/3494831 >> 172.21.15.31:0/3494831 conn(0x7f1c8c079390 msgr2=0x7f1c8c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:53.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:53.141+0000 7f1c93c00700 1 -- 172.21.15.31:0/3494831 shutdown_connections 2024-04-25T01:15:53.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:53.141+0000 7f1c93c00700 1 -- 172.21.15.31:0/3494831 wait complete. 2024-04-25T01:15:53.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:53 smithi031 ceph-mon[28053]: pgmap v389: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:53.504 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:53 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3494831' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T01:15:53.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:53 smithi169 ceph-mon[32759]: pgmap v389: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:53.767 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:53 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3494831' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T01:15:53.768 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:53 smithi040 ceph-mon[32518]: pgmap v389: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:53.768 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:53 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3494831' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T01:15:53.883 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T01:15:53.883 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T01:15:53.883 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2024-04-25T01:15:53.883 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T01:15:53.883 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T01:15:53.883 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T01:15:53.884 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T01:15:53.884 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T01:15:53.884 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T01:15:53.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:15:53.884 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2024-04-25T01:15:53.941 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T01:15:53.942 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T01:15:53.942 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T01:15:53.942 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:15:53.942 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-04-25T01:15:53.942 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:15:53.942 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:15:53.942 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T01:15:53.942 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2024-04-25T01:15:53.942 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:15:53.942 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T01:15:53.942 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2024-04-25T01:15:53.943 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T01:15:53.943 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2024-04-25T01:15:53.943 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2024-04-25T01:15:53.943 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T01:15:53.943 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 41951sec preferred_lft 41951sec 2024-04-25T01:15:53.943 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link noprefixroute 2024-04-25T01:15:53.943 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:15:53.943 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:15:53.943 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2024-04-25T01:15:53.943 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2024-04-25T01:15:53.944 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T01:15:53.944 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:15:53.944 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-04-25T01:15:53.944 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add name ceph-brx type bridge 2024-04-25T01:15:53.945 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr flush dev ceph-brx 2024-04-25T01:15:53.945 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set ceph-brx up 2024-04-25T01:15:53.945 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T01:15:53.945 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-04-25T01:15:54.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:15:54.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:15:54.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:15:54.432 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T01:15:54.499 INFO:teuthology.orchestra.run.smithi031.stdout:1 2024-04-25T01:15:54.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:15:54.502 DEBUG:teuthology.orchestra.run.smithi031:> route 2024-04-25T01:15:54.554 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2024-04-25T01:15:54.554 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T01:15:54.554 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T01:15:54.554 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T01:15:54.554 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T01:15:54.555 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:15:54.555 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-04-25T01:15:54.555 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T01:15:54.555 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T01:15:54.555 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T01:15:54.555 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-04-25T01:15:54.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:15:55.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:15:55.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:15:55.347 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-04-25T01:15:55.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:15:55.407 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2024-04-25T01:15:55.465 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:15:55.466 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-04-25T01:15:55.466 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:15:55.466 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T01:15:55.466 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-04-25T01:15:55.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:15:55.602 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:55 smithi031 ceph-mon[28053]: pgmap v390: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:55.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:15:55.660 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T01:15:55.660 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:15:55.660 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-04-25T01:15:55.660 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T01:15:55.660 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T01:15:55.660 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T01:15:55.660 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T01:15:55.660 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T01:15:55.661 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-04-25T01:15:55.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:55 smithi169 ceph-mon[32759]: pgmap v390: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:55.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:55 smithi040 ceph-mon[32518]: pgmap v390: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T01:15:55.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:15:55.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:15:55.982 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:15:55.982 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-04-25T01:15:55.982 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2024-04-25T01:15:55.982 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T01:15:55.982 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-04-25T01:15:56.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:15:56.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:15:56.200 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:15:56.200 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T01:15:56.247 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T01:15:56.247 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:15:56.248 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T01:15:56.342 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T01:15:56.342 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T01:15:56.342 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/1c94ca6f-03d3-430b-88f5-fb1cad13c507 2024-04-25T01:15:56.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:15:56.343 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/1c94ca6f-03d3-430b-88f5-fb1cad13c507 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-04-25T01:15:56.483 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-04-25T01:15:56.483 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-04-25T01:15:57.202 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-04-25T01:15:57.202 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-04-25T01:15:57.202 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:6789,172.21.15.40:6789,172.21.15.169:6789" 2024-04-25T01:15:57.202 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@95849b94-029f-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/1c94ca6f-03d3-430b-88f5-fb1cad13c507 2024-04-25T01:15:57.202 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.31:6789/172.21.15.40:6789/172.21.15.169:6789" will pass to kernel 2024-04-25T01:15:57.207 INFO:teuthology.orchestra.run.smithi031.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T01:15:57.207 INFO:teuthology.orchestra.run.smithi031.stdout: You just mounted a file system that supports labels which does not 2024-04-25T01:15:57.207 INFO:teuthology.orchestra.run.smithi031.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T01:15:57.207 INFO:teuthology.orchestra.run.smithi031.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T01:15:57.207 INFO:teuthology.orchestra.run.smithi031.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T01:15:57.212 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T01:15:57.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:15:57.212 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T01:15:57.311 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T01:15:57.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T01:15:57.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:15:57.445 INFO:teuthology.orchestra.run.smithi031.stdout:client25225 2024-04-25T01:15:57.446 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T01:15:57.446 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:15:57.503 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T01:15:57.503 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:57 smithi031 ceph-mon[28053]: pgmap v391: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T01:15:57.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T01:15:57.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:15:57.630 INFO:teuthology.orchestra.run.smithi031.stdout:client25225 2024-04-25T01:15:57.630 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T01:15:57.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:57 smithi169 ceph-mon[32759]: pgmap v391: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T01:15:57.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:57 smithi040 ceph-mon[32518]: pgmap v391: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T01:15:57.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T01:15:57.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:15:57.760 INFO:teuthology.orchestra.run.smithi031.stdout:client25225 2024-04-25T01:15:57.761 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/95849b94-029f-11ef-bc93-c7b262605968.client25225/status') 2024-04-25T01:15:57.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:15:57.911 INFO:teuthology.orchestra.run.smithi031.stdout:instance: client.25225 (3)172.21.15.31:0/3386541189 2024-04-25T01:15:57.911 INFO:teuthology.orchestra.run.smithi031.stdout:blocklisted: false 2024-04-25T01:15:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:0+1 records in 2024-04-25T01:15:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:0+1 records out 2024-04-25T01:15:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:71 bytes copied, 9.0494e-05 s, 785 kB/s 2024-04-25T01:15:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:15:57.927 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T01:15:57.959 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T01:15:57.959 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T01:15:57.959 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-25T01:15:58.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.191+0000 7eff32600640 1 -- 172.21.15.31:0/2064488628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff2c077340 msgr2=0x7eff2c0777a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:58.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.191+0000 7eff32600640 1 --2- 172.21.15.31:0/2064488628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff2c077340 0x7eff2c0777a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7eff14009900 tx=0x7eff1402a3f0 comp rx=0 tx=0).stop 2024-04-25T01:15:58.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.191+0000 7eff32600640 1 -- 172.21.15.31:0/2064488628 shutdown_connections 2024-04-25T01:15:58.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.191+0000 7eff32600640 1 --2- 172.21.15.31:0/2064488628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff2c077ce0 0x7eff2c078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:58.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.191+0000 7eff32600640 1 --2- 172.21.15.31:0/2064488628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff2c077340 0x7eff2c0777a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:58.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.191+0000 7eff32600640 1 --2- 172.21.15.31:0/2064488628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2c076100 0x7eff2c076520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:58.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.191+0000 7eff32600640 1 -- 172.21.15.31:0/2064488628 >> 172.21.15.31:0/2064488628 conn(0x7eff2c0fef60 msgr2=0x7eff2c101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:58.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.191+0000 7eff32600640 1 -- 172.21.15.31:0/2064488628 shutdown_connections 2024-04-25T01:15:58.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.191+0000 7eff32600640 1 -- 172.21.15.31:0/2064488628 wait complete. 2024-04-25T01:15:58.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.192+0000 7eff32600640 1 Processor -- start 2024-04-25T01:15:58.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.192+0000 7eff32600640 1 -- start start 2024-04-25T01:15:58.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.193+0000 7eff32600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff2c076100 0x7eff2c19dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:58.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.193+0000 7eff32600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff2c077340 0x7eff2c19e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:58.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.193+0000 7eff32600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2c077ce0 0x7eff2c1a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:58.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.193+0000 7eff32600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c113a60 con 0x7eff2c077340 2024-04-25T01:15:58.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.193+0000 7eff32600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c1138e0 con 0x7eff2c076100 2024-04-25T01:15:58.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.193+0000 7eff32600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c113be0 con 0x7eff2c077ce0 2024-04-25T01:15:58.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.193+0000 7eff2aa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff2c077340 0x7eff2c19e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:58.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.193+0000 7eff2aa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff2c077340 0x7eff2c19e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59436/0 (socket says 172.21.15.31:59436) 2024-04-25T01:15:58.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.193+0000 7eff2aa00640 1 -- 172.21.15.31:0/1275266562 learned_addr learned my addr 172.21.15.31:0/1275266562 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:15:58.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.193+0000 7eff2b400640 1 --2- 172.21.15.31:0/1275266562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff2c076100 0x7eff2c19dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:58.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.193+0000 7eff2be00640 1 --2- 172.21.15.31:0/1275266562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2c077ce0 0x7eff2c1a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:58.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.193+0000 7eff2aa00640 1 -- 172.21.15.31:0/1275266562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2c077ce0 msgr2=0x7eff2c1a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:58.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.193+0000 7eff2aa00640 1 --2- 172.21.15.31:0/1275266562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2c077ce0 0x7eff2c1a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:58.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.193+0000 7eff2aa00640 1 -- 172.21.15.31:0/1275266562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff2c076100 msgr2=0x7eff2c19dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:58.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.193+0000 7eff2aa00640 1 --2- 172.21.15.31:0/1275266562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff2c076100 0x7eff2c19dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:58.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.194+0000 7eff2aa00640 1 -- 172.21.15.31:0/1275266562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2c1a5be0 con 0x7eff2c077340 2024-04-25T01:15:58.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.194+0000 7eff2b400640 1 --2- 172.21.15.31:0/1275266562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff2c076100 0x7eff2c19dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:15:58.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.194+0000 7eff2aa00640 1 --2- 172.21.15.31:0/1275266562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff2c077340 0x7eff2c19e500 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7eff14031440 tx=0x7eff14032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:58.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.194+0000 7eff23e00640 1 -- 172.21.15.31:0/1275266562 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1403f030 con 0x7eff2c077340 2024-04-25T01:15:58.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.194+0000 7eff23e00640 1 -- 172.21.15.31:0/1275266562 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff14003ce0 con 0x7eff2c077340 2024-04-25T01:15:58.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.194+0000 7eff23e00640 1 -- 172.21.15.31:0/1275266562 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14035070 con 0x7eff2c077340 2024-04-25T01:15:58.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.194+0000 7eff32600640 1 -- 172.21.15.31:0/1275266562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff2c1a5e70 con 0x7eff2c077340 2024-04-25T01:15:58.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.195+0000 7eff32600640 1 -- 172.21.15.31:0/1275266562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff2c1a62d0 con 0x7eff2c077340 2024-04-25T01:15:58.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.196+0000 7eff32600640 1 -- 172.21.15.31:0/1275266562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff2c110340 con 0x7eff2c077340 2024-04-25T01:15:58.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.198+0000 7eff23e00640 1 -- 172.21.15.31:0/1275266562 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7eff14042030 con 0x7eff2c077340 2024-04-25T01:15:58.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.199+0000 7eff23e00640 1 --2- 172.21.15.31:0/1275266562 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7eff100a2c30 0x7eff100a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:15:58.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.199+0000 7eff23e00640 1 -- 172.21.15.31:0/1275266562 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7eff140ef460 con 0x7eff2c077340 2024-04-25T01:15:58.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.199+0000 7eff2b400640 1 --2- 172.21.15.31:0/1275266562 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7eff100a2c30 0x7eff100a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:15:58.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.200+0000 7eff2b400640 1 --2- 172.21.15.31:0/1275266562 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7eff100a2c30 0x7eff100a50f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0099a0 tx=0x7eff1c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:15:58.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.204+0000 7eff23e00640 1 -- 172.21.15.31:0/1275266562 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7eff140bc3b0 con 0x7eff2c077340 2024-04-25T01:15:58.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:58.451+0000 7eff32600640 1 -- 172.21.15.31:0/1275266562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7eff2c0623e0 con 0x7eff2c077340 2024-04-25T01:15:59.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:59.322+0000 7eff23e00640 1 -- 172.21.15.31:0/1275266562 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v24) ==== 117+0+0 (secure 0 0 0) 0x7eff140bbcb0 con 0x7eff2c077340 2024-04-25T01:15:59.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:59.326+0000 7eff32600640 1 -- 172.21.15.31:0/1275266562 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7eff100a2c30 msgr2=0x7eff100a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:59.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:59.326+0000 7eff32600640 1 --2- 172.21.15.31:0/1275266562 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7eff100a2c30 0x7eff100a50f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0099a0 tx=0x7eff1c009500 comp rx=0 tx=0).stop 2024-04-25T01:15:59.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:59.326+0000 7eff32600640 1 -- 172.21.15.31:0/1275266562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff2c077340 msgr2=0x7eff2c19e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:15:59.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:59.326+0000 7eff32600640 1 --2- 172.21.15.31:0/1275266562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff2c077340 0x7eff2c19e500 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7eff14031440 tx=0x7eff14032ce0 comp rx=0 tx=0).stop 2024-04-25T01:15:59.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:59.326+0000 7eff32600640 1 -- 172.21.15.31:0/1275266562 shutdown_connections 2024-04-25T01:15:59.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:59.326+0000 7eff32600640 1 --2- 172.21.15.31:0/1275266562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2c077ce0 0x7eff2c1a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:59.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:59.326+0000 7eff32600640 1 --2- 172.21.15.31:0/1275266562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff2c077340 0x7eff2c19e500 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:59.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:59.326+0000 7eff32600640 1 --2- 172.21.15.31:0/1275266562 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7eff100a2c30 0x7eff100a50f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:59.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:59.326+0000 7eff32600640 1 --2- 172.21.15.31:0/1275266562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff2c076100 0x7eff2c19dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:15:59.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:59.326+0000 7eff32600640 1 -- 172.21.15.31:0/1275266562 >> 172.21.15.31:0/1275266562 conn(0x7eff2c0fef60 msgr2=0x7eff2c100a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:15:59.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:59.326+0000 7eff32600640 1 -- 172.21.15.31:0/1275266562 shutdown_connections 2024-04-25T01:15:59.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:15:59.326+0000 7eff32600640 1 -- 172.21.15.31:0/1275266562 wait complete. 2024-04-25T01:15:59.332 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T01:15:59.344 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T01:15:59.354 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T01:15:59.354 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T01:15:59.362 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T01:15:59.374 INFO:tasks.workunit:Pulling workunits from ref d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:15:59.375 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T01:15:59.375 INFO:tasks.workunit:timeout=3h 2024-04-25T01:15:59.375 INFO:tasks.workunit:cleanup=True 2024-04-25T01:15:59.375 DEBUG:teuthology.orchestra.run.smithi031:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T01:15:59.426 INFO:teuthology.orchestra.run.smithi031.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T01:15:59.426 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-25T01:15:59.426 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-04-25T01:15:59.427 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:15:59.427 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-25T01:15:59.427 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-04-25 01:13:38.561086849 +0000 2024-04-25T01:15:59.427 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-04-25 01:13:38.561086849 +0000 2024-04-25T01:15:59.427 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-04-25 01:15:57.315235555 +0000 2024-04-25T01:15:59.427 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-04-25 01:13:38.561086849 +0000 2024-04-25T01:15:59.427 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T01:15:59.427 DEBUG:teuthology.orchestra.run.smithi031:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T01:15:59.514 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:15:59.564 INFO:tasks.workunit.client.0.smithi031.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T01:15:59.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:59 smithi169 ceph-mon[32759]: pgmap v392: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:59.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:15:59 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1275266562' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T01:15:59.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:59 smithi040 ceph-mon[32518]: pgmap v392: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:59.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:15:59 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1275266562' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T01:15:59.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:59 smithi031 ceph-mon[28053]: pgmap v392: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:15:59.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:15:59 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1275266562' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T01:16:00.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:00 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1275266562' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T01:16:00.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:00 smithi169 ceph-mon[32759]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:16:00.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:00 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1275266562' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T01:16:00.698 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:00 smithi040 ceph-mon[32518]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:16:00.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:00 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1275266562' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T01:16:00.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:00 smithi031 ceph-mon[28053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:16:01.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:01 smithi169 ceph-mon[32759]: pgmap v393: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T01:16:01.694 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:01 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:16:01.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:01 smithi040 ceph-mon[32518]: pgmap v393: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T01:16:01.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:01 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:16:01.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:01 smithi031 ceph-mon[28053]: pgmap v393: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T01:16:01.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:01 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:16:02.946 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:02 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:02.947 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:02 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:02.947 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:02 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:02.947 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:02 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:03.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:02 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:03.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:02 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:03.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:02 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:03.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:02 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:03.253 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:02 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:03.253 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:02 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:03.253 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:02 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:03.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:02 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:04.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:03 smithi169 ceph-mon[32759]: pgmap v394: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T01:16:04.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:03 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:04.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:03 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:04.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:03 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:04.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:03 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:04.196 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:03 smithi040 ceph-mon[32518]: pgmap v394: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T01:16:04.196 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:03 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:04.196 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:03 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:04.196 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:03 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:04.196 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:03 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:04.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:03 smithi031 ceph-mon[28053]: pgmap v394: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T01:16:04.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:03 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:04.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:03 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:04.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:03 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:04.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:03 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:06.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:05 smithi169 ceph-mon[32759]: pgmap v395: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T01:16:06.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:05 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:06.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:05 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:06.195 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:05 smithi031 ceph-mon[28053]: pgmap v395: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T01:16:06.195 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:05 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:06.195 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:05 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:06.196 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:05 smithi040 ceph-mon[32518]: pgmap v395: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T01:16:06.196 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:05 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:06.196 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:05 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:07.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:06 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:16:07.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:06 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:16:07.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:06 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:07.196 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:06 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:16:07.196 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:06 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:16:07.196 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:06 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:07.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:06 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:16:07.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:06 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:16:07.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:06 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:16:08.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:07 smithi169 ceph-mon[32759]: pgmap v396: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T01:16:08.196 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:07 smithi040 ceph-mon[32518]: pgmap v396: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T01:16:08.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:07 smithi031 ceph-mon[28053]: pgmap v396: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T01:16:10.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:09 smithi169 ceph-mon[32759]: pgmap v397: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T01:16:10.196 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:09 smithi040 ceph-mon[32518]: pgmap v397: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T01:16:10.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:09 smithi031 ceph-mon[28053]: pgmap v397: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T01:16:12.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:11 smithi169 ceph-mon[32759]: pgmap v398: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T01:16:12.196 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:11 smithi040 ceph-mon[32518]: pgmap v398: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T01:16:12.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:11 smithi031 ceph-mon[28053]: pgmap v398: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T01:16:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.625+0000 7ffa03e00640 1 -- 172.21.15.31:0/928931064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa041031c0 msgr2=0x7ffa04109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.625+0000 7ffa03e00640 1 --2- 172.21.15.31:0/928931064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa041031c0 0x7ffa04109a50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009be0 tx=0x7ff9fc02a6c0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.625+0000 7ffa03e00640 1 -- 172.21.15.31:0/928931064 shutdown_connections 2024-04-25T01:16:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.625+0000 7ffa03e00640 1 --2- 172.21.15.31:0/928931064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa041031c0 0x7ffa04109a50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.625+0000 7ffa03e00640 1 --2- 172.21.15.31:0/928931064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa04102800 0x7ffa04102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.625+0000 7ffa03e00640 1 --2- 172.21.15.31:0/928931064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa04101610 0x7ffa04101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.625+0000 7ffa03e00640 1 -- 172.21.15.31:0/928931064 >> 172.21.15.31:0/928931064 conn(0x7ffa040fcba0 msgr2=0x7ffa040fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:16:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.625+0000 7ffa03e00640 1 -- 172.21.15.31:0/928931064 shutdown_connections 2024-04-25T01:16:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.625+0000 7ffa03e00640 1 -- 172.21.15.31:0/928931064 wait complete. 2024-04-25T01:16:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.626+0000 7ffa03e00640 1 Processor -- start 2024-04-25T01:16:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.626+0000 7ffa03e00640 1 -- start start 2024-04-25T01:16:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.627+0000 7ffa03e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa04101610 0x7ffa040754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.627+0000 7ffa03e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa04102800 0x7ffa04073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.627+0000 7ffa03e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa041031c0 0x7ffa04074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.627+0000 7ffa03e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0410d270 con 0x7ffa041031c0 2024-04-25T01:16:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.627+0000 7ffa03e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0410d0f0 con 0x7ffa04101610 2024-04-25T01:16:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.627+0000 7ffa03e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0410d3f0 con 0x7ffa04102800 2024-04-25T01:16:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.627+0000 7ffa02a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa04101610 0x7ffa040754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.627+0000 7ffa02a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa04101610 0x7ffa040754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:38870/0 (socket says 172.21.15.31:38870) 2024-04-25T01:16:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.627+0000 7ffa02a00640 1 -- 172.21.15.31:0/713438148 learned_addr learned my addr 172.21.15.31:0/713438148 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:16:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.627+0000 7ff9fbe00640 1 --2- 172.21.15.31:0/713438148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa04102800 0x7ffa04073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.627+0000 7ffa03400640 1 --2- 172.21.15.31:0/713438148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa041031c0 0x7ffa04074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.627+0000 7ffa02a00640 1 -- 172.21.15.31:0/713438148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa04102800 msgr2=0x7ffa04073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.627+0000 7ffa02a00640 1 --2- 172.21.15.31:0/713438148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa04102800 0x7ffa04073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.627+0000 7ffa02a00640 1 -- 172.21.15.31:0/713438148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa041031c0 msgr2=0x7ffa04074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.627+0000 7ffa02a00640 1 --2- 172.21.15.31:0/713438148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa041031c0 0x7ffa04074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.627+0000 7ffa02a00640 1 -- 172.21.15.31:0/713438148 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa04074900 con 0x7ffa04101610 2024-04-25T01:16:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.628+0000 7ff9fbe00640 1 --2- 172.21.15.31:0/713438148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa04102800 0x7ffa04073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:16:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.628+0000 7ffa02a00640 1 --2- 172.21.15.31:0/713438148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa04101610 0x7ffa040754f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400e9c0 tx=0x7ff9f400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:16:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.628+0000 7ffa08a00640 1 -- 172.21.15.31:0/713438148 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f400cd80 con 0x7ffa04101610 2024-04-25T01:16:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.628+0000 7ffa03e00640 1 -- 172.21.15.31:0/713438148 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa04197530 con 0x7ffa04101610 2024-04-25T01:16:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.628+0000 7ffa08a00640 1 -- 172.21.15.31:0/713438148 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9f40035f0 con 0x7ffa04101610 2024-04-25T01:16:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.628+0000 7ffa08a00640 1 -- 172.21.15.31:0/713438148 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4010690 con 0x7ffa04101610 2024-04-25T01:16:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.628+0000 7ffa03e00640 1 -- 172.21.15.31:0/713438148 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa04197ad0 con 0x7ffa04101610 2024-04-25T01:16:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.631+0000 7ffa0cd8d740 1 Processor -- start 2024-04-25T01:16:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.632+0000 7ffa0cd8d740 1 -- start start 2024-04-25T01:16:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.632+0000 7ffa08a00640 1 -- 172.21.15.31:0/713438148 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff9f40037a0 con 0x7ffa04101610 2024-04-25T01:16:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.632+0000 7ffa0cd8d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bff6800e50 0x55bff6801270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.632+0000 7ffa0cd8d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bff6801840 0x55bff6806340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.632+0000 7ffa0cd8d740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bff68069b0 0x55bff6808e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.632+0000 7ffa0cd8d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55bff6763ab0 con 0x55bff6801840 2024-04-25T01:16:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.632+0000 7ffa0cd8d740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55bff670c640 con 0x55bff68069b0 2024-04-25T01:16:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.632+0000 7ffa0cd8d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bff675fa70 con 0x55bff6800e50 2024-04-25T01:16:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.632+0000 7ffa02a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bff6801840 0x55bff6806340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.632+0000 7ffa03400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bff6800e50 0x55bff6801270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.632+0000 7ffa03400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bff6800e50 0x55bff6801270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:45234/0 (socket says 172.21.15.31:45234) 2024-04-25T01:16:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.632+0000 7ffa02a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bff6801840 0x55bff6806340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40604/0 (socket says 172.21.15.31:40604) 2024-04-25T01:16:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.632+0000 7ffa03400640 1 -- 172.21.15.31:0/3424717556 learned_addr learned my addr 172.21.15.31:0/3424717556 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:16:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.632+0000 7ff9fbe00640 1 --2- 172.21.15.31:0/3424717556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bff68069b0 0x55bff6808e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.633+0000 7ffa03400640 1 -- 172.21.15.31:0/3424717556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bff68069b0 msgr2=0x55bff6808e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.633+0000 7ffa03400640 1 --2- 172.21.15.31:0/3424717556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bff68069b0 0x55bff6808e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.633+0000 7ffa03400640 1 -- 172.21.15.31:0/3424717556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bff6801840 msgr2=0x55bff6806340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.633+0000 7ffa03400640 1 --2- 172.21.15.31:0/3424717556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bff6801840 0x55bff6806340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.633+0000 7ffa03400640 1 -- 172.21.15.31:0/3424717556 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bff6478210 con 0x55bff6800e50 2024-04-25T01:16:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.633+0000 7ffa03400640 1 --2- 172.21.15.31:0/3424717556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bff6800e50 0x55bff6801270 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009be0 tx=0x7ff9fc005eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2c0e2121f73f7df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:16:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.633+0000 7ffa08a00640 1 --2- 172.21.15.31:0/713438148 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff9e80a2c00 0x7ff9e80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.633+0000 7ff9f3e00640 1 -- 172.21.15.31:0/3424717556 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc041030 con 0x55bff6800e50 2024-04-25T01:16:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.634+0000 7ff9f3e00640 1 -- 172.21.15.31:0/3424717556 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9fc030c70 con 0x55bff6800e50 2024-04-25T01:16:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.634+0000 7ff9f3e00640 1 -- 172.21.15.31:0/3424717556 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc036040 con 0x55bff6800e50 2024-04-25T01:16:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.634+0000 7ffa08a00640 1 -- 172.21.15.31:0/713438148 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7ff9f40cf400 con 0x7ffa04101610 2024-04-25T01:16:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.634+0000 7ffa02a00640 1 --2- 172.21.15.31:0/713438148 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff9e80a2c00 0x7ff9e80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.634+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/3424717556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bff6800e50 msgr2=0x55bff6801270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.634+0000 7ffa0cd8d740 1 --2- 172.21.15.31:0/3424717556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bff6800e50 0x55bff6801270 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009be0 tx=0x7ff9fc005eb0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.634+0000 7ffa02a00640 1 --2- 172.21.15.31:0/713438148 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff9e80a2c00 0x7ff9e80a50c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55bff6801c40 tx=0x7ff9f40d2e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:16:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.634+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/3424717556 shutdown_connections 2024-04-25T01:16:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.634+0000 7ffa0cd8d740 1 --2- 172.21.15.31:0/3424717556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bff68069b0 0x55bff6808e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.634+0000 7ffa0cd8d740 1 --2- 172.21.15.31:0/3424717556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bff6801840 0x55bff6806340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.634+0000 7ffa0cd8d740 1 --2- 172.21.15.31:0/3424717556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bff6800e50 0x55bff6801270 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.634+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/3424717556 >> 172.21.15.31:0/3424717556 conn(0x55bff67fc4c0 msgr2=0x55bff67fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:16:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.635+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/3424717556 shutdown_connections 2024-04-25T01:16:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.635+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/3424717556 wait complete. 2024-04-25T01:16:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.635+0000 7ffa0cd8d740 1 -- start start 2024-04-25T01:16:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.635+0000 7ffa0cd8d740 1 Processor -- start 2024-04-25T01:16:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.636+0000 7ffa0cd8d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bff6800e50 0x55bff680afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.636+0000 7ffa0cd8d740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bff6801840 0x55bff680b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.636+0000 7ffa0cd8d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bff68069b0 0x55bff6812560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.636+0000 7ffa0cd8d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55bff6809900 con 0x55bff68069b0 2024-04-25T01:16:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.636+0000 7ffa0cd8d740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55bff671f990 con 0x55bff6801840 2024-04-25T01:16:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.636+0000 7ffa0cd8d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bff6809a80 con 0x55bff6800e50 2024-04-25T01:16:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.636+0000 7ffa0cd8d740 10 client.0 _ll_register_callbacks cb 0x55bff6721950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:16:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.636+0000 7ffa03400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bff6800e50 0x55bff680afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.636+0000 7ffa03400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bff68069b0 0x55bff6812560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.636+0000 7ffa03400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bff68069b0 0x55bff6812560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40614/0 (socket says 172.21.15.31:40614) 2024-04-25T01:16:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.636+0000 7ffa03400640 1 -- 172.21.15.31:0/657012879 learned_addr learned my addr 172.21.15.31:0/657012879 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:16:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.636+0000 7ff9fbe00640 1 --2- 172.21.15.31:0/657012879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bff6801840 0x55bff680b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.637+0000 7ffa03400640 1 -- 172.21.15.31:0/657012879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bff6800e50 msgr2=0x55bff680afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.637+0000 7ffa03400640 1 --2- 172.21.15.31:0/657012879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bff6800e50 0x55bff680afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.637+0000 7ffa03400640 1 -- 172.21.15.31:0/657012879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bff6801840 msgr2=0x55bff680b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.637+0000 7ffa03400640 1 --2- 172.21.15.31:0/657012879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bff6801840 0x55bff680b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.637+0000 7ffa03400640 1 -- 172.21.15.31:0/657012879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bff6544620 con 0x55bff68069b0 2024-04-25T01:16:12.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.637+0000 7ffa03400640 1 --2- 172.21.15.31:0/657012879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bff68069b0 0x55bff6812560 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc030960 tx=0x7ff9fc02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68dd3c7b05e8205d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:16:12.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.637+0000 7ff9f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T01:16:12.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.637+0000 7ff9f0c00640 1 -- 172.21.15.31:0/657012879 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc04a020 con 0x55bff68069b0 2024-04-25T01:16:12.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.637+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/657012879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9fc030c70 con 0x55bff68069b0 2024-04-25T01:16:12.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.637+0000 7ff9f0c00640 1 -- 172.21.15.31:0/657012879 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7ff9fc0303a0 con 0x55bff68069b0 2024-04-25T01:16:12.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.637+0000 7ff9f0c00640 1 -- 172.21.15.31:0/657012879 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff9fc041070 con 0x55bff68069b0 2024-04-25T01:16:12.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.638+0000 7ffa0cd8d740 10 client.15114 fetch_fsmap learned FSMap version 24 2024-04-25T01:16:12.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.638+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/657012879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bff6815540 con 0x55bff68069b0 2024-04-25T01:16:12.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.638+0000 7ff9f0c00640 1 -- 172.21.15.31:0/657012879 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9fc030c70 con 0x55bff68069b0 2024-04-25T01:16:12.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.638+0000 7ff9f0c00640 1 -- 172.21.15.31:0/657012879 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ff9fc03d8d0 con 0x55bff68069b0 2024-04-25T01:16:12.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.638+0000 7ffa0cd8d740 10 client.15114 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T01:16:12.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.638+0000 7ffa0cd8d740 10 client.15114 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:16:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.638+0000 7ffa0cd8d740 20 client.15114 populate_metadata read hostname 'smithi031' 2024-04-25T01:16:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.638+0000 7ffa0cd8d740 1 --2- 172.21.15.31:0/657012879 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55bff6816890 0x55bff6818c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.638+0000 7ffa0cd8d740 4 client.15114 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:16:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.638+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/657012879 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bff6770ff0 con 0x55bff6816890 2024-04-25T01:16:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.639+0000 7ff9fbe00640 1 --2- 172.21.15.31:0/657012879 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55bff6816890 0x55bff6818c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.639+0000 7ff9fbe00640 1 --2- 172.21.15.31:0/657012879 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55bff6816890 0x55bff6818c80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54ec8bd517efc847 server_cookie=2ad285b99b494358 in_seq=0 out_seq=0 2024-04-25T01:16:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.639+0000 7ff9f0c00640 10 client.15114 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:16:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.640+0000 7ff9f0c00640 1 -- 172.21.15.31:0/657012879 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bff6770ff0 con 0x55bff6816890 2024-04-25T01:16:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.640+0000 7ff9f0c00640 10 client.15114 handle_command_reply: tid=0 2024-04-25T01:16:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.640+0000 7ffa0cd8d740 1 client.15114 shutdown 2024-04-25T01:16:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.640+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/657012879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bff68069b0 msgr2=0x55bff6812560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.640+0000 7ffa0cd8d740 1 --2- 172.21.15.31:0/657012879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bff68069b0 0x55bff6812560 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc030960 tx=0x7ff9fc02a6c0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.640+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/657012879 shutdown_connections 2024-04-25T01:16:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.640+0000 7ffa0cd8d740 1 --2- 172.21.15.31:0/657012879 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55bff6816890 0x55bff6818c80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.640+0000 7ffa0cd8d740 1 --2- 172.21.15.31:0/657012879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bff68069b0 0x55bff6812560 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.640+0000 7ffa0cd8d740 1 --2- 172.21.15.31:0/657012879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bff6801840 0x55bff680b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.640+0000 7ffa0cd8d740 1 --2- 172.21.15.31:0/657012879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bff6800e50 0x55bff680afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.640+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/657012879 >> 172.21.15.31:0/657012879 conn(0x55bff67fc4c0 msgr2=0x55bff67fb620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:16:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.641+0000 7ff9f0c00640 0 client.15114 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:16:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.641+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/657012879 shutdown_connections 2024-04-25T01:16:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.641+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/657012879 wait complete. 2024-04-25T01:16:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.641+0000 7ffa0cd8d740 20 client.15114 trim_cache size 0 max 16384 2024-04-25T01:16:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.657+0000 7ffa0cd8d740 1 Processor -- start 2024-04-25T01:16:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.657+0000 7ffa0cd8d740 1 -- start start 2024-04-25T01:16:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.658+0000 7ffa0cd8d740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bff6800e50 0x55bff67f7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.658+0000 7ffa0cd8d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bff6801840 0x55bff6812480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.658+0000 7ffa0cd8d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bff68069b0 0x55bff681d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.658+0000 7ffa0cd8d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041b1730 con 0x55bff68069b0 2024-04-25T01:16:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.658+0000 7ffa0cd8d740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041af000 con 0x55bff6800e50 2024-04-25T01:16:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.658+0000 7ffa0cd8d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041ace50 con 0x55bff6801840 2024-04-25T01:16:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.658+0000 7ffa03400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bff68069b0 0x55bff681d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.658+0000 7ffa03400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bff68069b0 0x55bff681d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40630/0 (socket says 172.21.15.31:40630) 2024-04-25T01:16:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.658+0000 7ffa03400640 1 -- 172.21.15.31:0/1265833062 learned_addr learned my addr 172.21.15.31:0/1265833062 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:16:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.658+0000 7ff9fbe00640 1 --2- 172.21.15.31:0/1265833062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bff6801840 0x55bff6812480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.658+0000 7ffa03400640 1 --2- 172.21.15.31:0/1265833062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bff6800e50 0x55bff67f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.659+0000 7ff9fbe00640 1 -- 172.21.15.31:0/1265833062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bff6800e50 msgr2=0x55bff67f7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.659+0000 7ff9fbe00640 1 --2- 172.21.15.31:0/1265833062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bff6800e50 0x55bff67f7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.659+0000 7ff9fbe00640 1 -- 172.21.15.31:0/1265833062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bff68069b0 msgr2=0x55bff681d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.659+0000 7ff9fbe00640 1 --2- 172.21.15.31:0/1265833062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bff68069b0 0x55bff681d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.659+0000 7ff9fbe00640 1 -- 172.21.15.31:0/1265833062 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bff67fc2f0 con 0x55bff6801840 2024-04-25T01:16:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.659+0000 7ff9fbe00640 1 --2- 172.21.15.31:0/1265833062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bff6801840 0x55bff6812480 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55bff67fd390 tx=0x7ff9ec009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2584a0cc369f88c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:16:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.659+0000 7ff9f3400640 1 -- 172.21.15.31:0/1265833062 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec0122d0 con 0x55bff6801840 2024-04-25T01:16:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.659+0000 7ff9f3400640 1 -- 172.21.15.31:0/1265833062 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9ec012e50 con 0x55bff6801840 2024-04-25T01:16:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.659+0000 7ff9f3400640 1 -- 172.21.15.31:0/1265833062 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec020330 con 0x55bff6801840 2024-04-25T01:16:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.660+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/1265833062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bff6801840 msgr2=0x55bff6812480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.660+0000 7ffa0cd8d740 1 --2- 172.21.15.31:0/1265833062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bff6801840 0x55bff6812480 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55bff67fd390 tx=0x7ff9ec009260 comp rx=0 tx=0).stop 2024-04-25T01:16:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.660+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/1265833062 shutdown_connections 2024-04-25T01:16:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.660+0000 7ffa0cd8d740 1 --2- 172.21.15.31:0/1265833062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bff68069b0 0x55bff681d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.660+0000 7ffa0cd8d740 1 --2- 172.21.15.31:0/1265833062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bff6801840 0x55bff6812480 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.660+0000 7ffa0cd8d740 1 --2- 172.21.15.31:0/1265833062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bff6800e50 0x55bff67f7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.660+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/1265833062 >> 172.21.15.31:0/1265833062 conn(0x55bff67fc4c0 msgr2=0x55bff67ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:16:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.660+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/1265833062 shutdown_connections 2024-04-25T01:16:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.660+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/1265833062 wait complete. 2024-04-25T01:16:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.660+0000 7ffa0cd8d740 1 -- start start 2024-04-25T01:16:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.661+0000 7ffa0cd8d740 1 Processor -- start 2024-04-25T01:16:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.661+0000 7ffa0cd8d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bff6800e50 0x55bff6811f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.661+0000 7ffa0cd8d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bff6801840 0x55bff68124d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.661+0000 7ffa0cd8d740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bff68069b0 0x55bff68105a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.661+0000 7ffa0cd8d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041ad4c0 con 0x55bff6801840 2024-04-25T01:16:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.661+0000 7ffa0cd8d740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041ae930 con 0x55bff68069b0 2024-04-25T01:16:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.661+0000 7ffa0cd8d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffa041adb90 con 0x55bff6800e50 2024-04-25T01:16:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.661+0000 7ffa0cd8d740 10 client.0 _ll_register_callbacks cb 0x7ff9f40d2d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:16:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.661+0000 7ff9fbe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bff6801840 0x55bff68124d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.661+0000 7ffa03400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bff6800e50 0x55bff6811f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.661+0000 7ffa03400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bff6800e50 0x55bff6811f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:45252/0 (socket says 172.21.15.31:45252) 2024-04-25T01:16:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.661+0000 7ffa03400640 1 -- 172.21.15.31:0/169996882 learned_addr learned my addr 172.21.15.31:0/169996882 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:16:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.662+0000 7ffa03400640 1 --2- 172.21.15.31:0/169996882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bff68069b0 0x55bff68105a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.662+0000 7ffa03400640 1 -- 172.21.15.31:0/169996882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bff68069b0 msgr2=0x55bff68105a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.662+0000 7ffa03400640 1 --2- 172.21.15.31:0/169996882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bff68069b0 0x55bff68105a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.662+0000 7ffa03400640 1 -- 172.21.15.31:0/169996882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bff6801840 msgr2=0x55bff68124d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.662+0000 7ffa03400640 1 --2- 172.21.15.31:0/169996882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bff6801840 0x55bff68124d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.662+0000 7ffa03400640 1 -- 172.21.15.31:0/169996882 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bff6810d80 con 0x55bff6800e50 2024-04-25T01:16:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.662+0000 7ff9fbe00640 1 --2- 172.21.15.31:0/169996882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bff6801840 0x55bff68124d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:16:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.662+0000 7ffa03400640 1 --2- 172.21.15.31:0/169996882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bff6800e50 0x55bff6811f90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009be0 tx=0x7ff9fc03f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4a0dbf6c71264fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:16:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.662+0000 7ff9f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:16:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.662+0000 7ff9f0c00640 1 -- 172.21.15.31:0/169996882 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc04a040 con 0x55bff6800e50 2024-04-25T01:16:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.662+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/169996882 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9ec012e50 con 0x55bff6800e50 2024-04-25T01:16:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.663+0000 7ff9f0c00640 1 -- 172.21.15.31:0/169996882 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7ff9fc0303a0 con 0x55bff6800e50 2024-04-25T01:16:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.663+0000 7ff9f0c00640 1 -- 172.21.15.31:0/169996882 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff9fc036070 con 0x55bff6800e50 2024-04-25T01:16:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.663+0000 7ffa0cd8d740 10 client.25249 fetch_fsmap learned FSMap version 24 2024-04-25T01:16:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.663+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/169996882 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bff687d7d0 con 0x55bff6800e50 2024-04-25T01:16:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.663+0000 7ff9f0c00640 1 -- 172.21.15.31:0/169996882 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9ec012e50 con 0x55bff6800e50 2024-04-25T01:16:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.663+0000 7ff9f0c00640 1 -- 172.21.15.31:0/169996882 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ff9fc03d4f0 con 0x55bff6800e50 2024-04-25T01:16:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.663+0000 7ffa0cd8d740 10 client.25249 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T01:16:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.663+0000 7ffa0cd8d740 10 client.25249 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:16:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.663+0000 7ffa0cd8d740 20 client.25249 populate_metadata read hostname 'smithi031' 2024-04-25T01:16:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.663+0000 7ffa0cd8d740 1 --2- 172.21.15.31:0/169996882 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55bff687dfc0 0x55bff68803e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.663+0000 7ffa0cd8d740 4 client.25249 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:16:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.663+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/169996882 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bff676b340 con 0x55bff687dfc0 2024-04-25T01:16:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.664+0000 7ff9fbe00640 1 --2- 172.21.15.31:0/169996882 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55bff687dfc0 0x55bff68803e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.664+0000 7ff9fbe00640 1 --2- 172.21.15.31:0/169996882 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55bff687dfc0 0x55bff68803e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f99137a573779878 server_cookie=3603333a4024791d in_seq=0 out_seq=0 2024-04-25T01:16:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.664+0000 7ff9f0c00640 10 client.25249 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:16:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.676+0000 7ff9f0c00640 1 -- 172.21.15.31:0/169996882 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bff676b340 con 0x55bff687dfc0 2024-04-25T01:16:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.676+0000 7ff9f0c00640 10 client.25249 handle_command_reply: tid=0 2024-04-25T01:16:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.676+0000 7ffa0cd8d740 1 client.25249 shutdown 2024-04-25T01:16:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.676+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/169996882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bff6800e50 msgr2=0x55bff6811f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.676+0000 7ffa0cd8d740 1 --2- 172.21.15.31:0/169996882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bff6800e50 0x55bff6811f90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009be0 tx=0x7ff9fc03f040 comp rx=0 tx=0).stop 2024-04-25T01:16:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.676+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/169996882 shutdown_connections 2024-04-25T01:16:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.676+0000 7ffa0cd8d740 1 --2- 172.21.15.31:0/169996882 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55bff687dfc0 0x55bff68803e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.676+0000 7ffa0cd8d740 1 --2- 172.21.15.31:0/169996882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55bff68069b0 0x55bff68105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.676+0000 7ffa0cd8d740 1 --2- 172.21.15.31:0/169996882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bff6801840 0x55bff68124d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.676+0000 7ffa0cd8d740 1 --2- 172.21.15.31:0/169996882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bff6800e50 0x55bff6811f90 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.676+0000 7ff9f0c00640 0 client.25249 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:16:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.676+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/169996882 >> 172.21.15.31:0/169996882 conn(0x55bff67fc4c0 msgr2=0x55bff67fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:16:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.677+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/169996882 shutdown_connections 2024-04-25T01:16:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.677+0000 7ffa0cd8d740 1 -- 172.21.15.31:0/169996882 wait complete. 2024-04-25T01:16:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.677+0000 7ffa0cd8d740 20 client.25249 trim_cache size 0 max 16384 2024-04-25T01:16:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:16:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:16:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:16:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:16:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.677+0000 7ffa03e00640 1 -- 172.21.15.31:0/713438148 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff9e80a2c00 msgr2=0x7ff9e80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.677+0000 7ffa03e00640 1 --2- 172.21.15.31:0/713438148 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff9e80a2c00 0x7ff9e80a50c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55bff6801c40 tx=0x7ff9f40d2e40 comp rx=0 tx=0).stop 2024-04-25T01:16:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.677+0000 7ffa03e00640 1 -- 172.21.15.31:0/713438148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa04101610 msgr2=0x7ffa040754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.677+0000 7ffa03e00640 1 --2- 172.21.15.31:0/713438148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa04101610 0x7ffa040754f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400e9c0 tx=0x7ff9f400ee80 comp rx=0 tx=0).stop 2024-04-25T01:16:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.678+0000 7ffa03e00640 1 -- 172.21.15.31:0/713438148 shutdown_connections 2024-04-25T01:16:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.678+0000 7ffa03e00640 1 --2- 172.21.15.31:0/713438148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa041031c0 0x7ffa04074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.678+0000 7ffa03e00640 1 --2- 172.21.15.31:0/713438148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa04102800 0x7ffa04073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.678+0000 7ffa03e00640 1 --2- 172.21.15.31:0/713438148 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff9e80a2c00 0x7ff9e80a50c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.678+0000 7ffa03e00640 1 --2- 172.21.15.31:0/713438148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa04101610 0x7ffa040754f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.678+0000 7ffa03e00640 1 -- 172.21.15.31:0/713438148 >> 172.21.15.31:0/713438148 conn(0x7ffa040fcba0 msgr2=0x7ffa04105aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:16:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.678+0000 7ffa03e00640 1 -- 172.21.15.31:0/713438148 shutdown_connections 2024-04-25T01:16:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:12.678+0000 7ffa03e00640 1 -- 172.21.15.31:0/713438148 wait complete. 2024-04-25T01:16:14.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:13 smithi169 ceph-mon[32759]: pgmap v399: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-25T01:16:14.196 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:13 smithi040 ceph-mon[32518]: pgmap v399: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-25T01:16:14.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:13 smithi031 ceph-mon[28053]: pgmap v399: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-25T01:16:16.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:15 smithi169 ceph-mon[32759]: pgmap v400: 97 pgs: 97 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-25T01:16:16.196 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:15 smithi040 ceph-mon[32518]: pgmap v400: 97 pgs: 97 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-25T01:16:16.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:15 smithi031 ceph-mon[28053]: pgmap v400: 97 pgs: 97 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-25T01:16:18.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:17 smithi169 ceph-mon[32759]: pgmap v401: 97 pgs: 97 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-25T01:16:18.196 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:17 smithi040 ceph-mon[32518]: pgmap v401: 97 pgs: 97 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-25T01:16:18.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:17 smithi031 ceph-mon[28053]: pgmap v401: 97 pgs: 97 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-25T01:16:20.193 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:19 smithi169 ceph-mon[32759]: pgmap v402: 97 pgs: 97 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-25T01:16:20.196 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:19 smithi040 ceph-mon[32518]: pgmap v402: 97 pgs: 97 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-25T01:16:20.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:19 smithi031 ceph-mon[28053]: pgmap v402: 97 pgs: 97 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-25T01:16:22.194 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:21 smithi169 ceph-mon[32759]: pgmap v403: 97 pgs: 97 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-04-25T01:16:22.196 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:21 smithi040 ceph-mon[32518]: pgmap v403: 97 pgs: 97 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-04-25T01:16:22.253 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:21 smithi031 ceph-mon[28053]: pgmap v403: 97 pgs: 97 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-04-25T01:16:24.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:23 smithi031 ceph-mon[28053]: pgmap v404: 97 pgs: 97 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-25T01:16:24.443 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:23 smithi169 ceph-mon[32759]: pgmap v404: 97 pgs: 97 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-25T01:16:24.446 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:23 smithi040 ceph-mon[32518]: pgmap v404: 97 pgs: 97 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-25T01:16:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.825+0000 7f1df7e00640 1 -- 172.21.15.31:0/579700332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df81011b0 msgr2=0x7f1df8101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.825+0000 7f1df7e00640 1 --2- 172.21.15.31:0/579700332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df81011b0 0x7f1df8101630 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009ca0 tx=0x7f1df002a940 comp rx=0 tx=0).stop 2024-04-25T01:16:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.825+0000 7f1df7e00640 1 -- 172.21.15.31:0/579700332 shutdown_connections 2024-04-25T01:16:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.825+0000 7f1df7e00640 1 --2- 172.21.15.31:0/579700332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df81011b0 0x7f1df8101630 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.825+0000 7f1df7e00640 1 --2- 172.21.15.31:0/579700332 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1df8100810 0x7f1df8100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.825+0000 7f1df7e00640 1 --2- 172.21.15.31:0/579700332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df80ffeb0 0x7f1df81002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.825+0000 7f1df7e00640 1 -- 172.21.15.31:0/579700332 >> 172.21.15.31:0/579700332 conn(0x7f1df80fba30 msgr2=0x7f1df80fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:16:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.826+0000 7f1df7e00640 1 -- 172.21.15.31:0/579700332 shutdown_connections 2024-04-25T01:16:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.826+0000 7f1df7e00640 1 -- 172.21.15.31:0/579700332 wait complete. 2024-04-25T01:16:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.827+0000 7f1df7e00640 1 Processor -- start 2024-04-25T01:16:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.827+0000 7f1df7e00640 1 -- start start 2024-04-25T01:16:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.827+0000 7f1df7e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1df80ffeb0 0x7f1df8199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.827+0000 7f1df7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df8100810 0x7f1df8199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.827+0000 7f1df7e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df81011b0 0x7f1df81a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.827+0000 7f1df7e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1df810f440 con 0x7f1df81011b0 2024-04-25T01:16:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.827+0000 7f1df7e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1df810f2c0 con 0x7f1df80ffeb0 2024-04-25T01:16:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.827+0000 7f1df7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1df810f5c0 con 0x7f1df8100810 2024-04-25T01:16:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.828+0000 7f1df6000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df8100810 0x7f1df8199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.828+0000 7f1df6a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1df80ffeb0 0x7f1df8199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.828+0000 7f1df6000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df8100810 0x7f1df8199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:38120/0 (socket says 172.21.15.31:38120) 2024-04-25T01:16:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.828+0000 7f1df6a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1df80ffeb0 0x7f1df8199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:47046/0 (socket says 172.21.15.31:47046) 2024-04-25T01:16:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.828+0000 7f1df6000640 1 -- 172.21.15.31:0/2822992986 learned_addr learned my addr 172.21.15.31:0/2822992986 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:16:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.828+0000 7f1df7400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df81011b0 0x7f1df81a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.828+0000 7f1df6000640 1 -- 172.21.15.31:0/2822992986 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1df80ffeb0 msgr2=0x7f1df8199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.828+0000 7f1df6000640 1 --2- 172.21.15.31:0/2822992986 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1df80ffeb0 0x7f1df8199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.828+0000 7f1df6000640 1 -- 172.21.15.31:0/2822992986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df81011b0 msgr2=0x7f1df81a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.828+0000 7f1df6000640 1 --2- 172.21.15.31:0/2822992986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df81011b0 0x7f1df81a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.828+0000 7f1df6000640 1 -- 172.21.15.31:0/2822992986 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df81a16d0 con 0x7f1df8100810 2024-04-25T01:16:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.828+0000 7f1df6000640 1 --2- 172.21.15.31:0/2822992986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df8100810 0x7f1df8199ff0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1de400e9f0 tx=0x7f1de400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:16:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.829+0000 7f1def400640 1 -- 172.21.15.31:0/2822992986 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de400cd80 con 0x7f1df8100810 2024-04-25T01:16:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.829+0000 7f1def400640 1 -- 172.21.15.31:0/2822992986 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1de4010910 con 0x7f1df8100810 2024-04-25T01:16:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.829+0000 7f1df7e00640 1 -- 172.21.15.31:0/2822992986 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df81a19c0 con 0x7f1df8100810 2024-04-25T01:16:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.829+0000 7f1def400640 1 -- 172.21.15.31:0/2822992986 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de40188f0 con 0x7f1df8100810 2024-04-25T01:16:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.829+0000 7f1df7e00640 1 -- 172.21.15.31:0/2822992986 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df81a1f60 con 0x7f1df8100810 2024-04-25T01:16:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.833+0000 7f1e007f1740 1 Processor -- start 2024-04-25T01:16:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.833+0000 7f1e007f1740 1 -- start start 2024-04-25T01:16:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.833+0000 7f1e007f1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef3ae50 0x55a56ef3b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.833+0000 7f1e007f1740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a56ef3b840 0x55a56ef40340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.833+0000 7f1e007f1740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a56ef409b0 0x55a56ef42e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.833+0000 7f1e007f1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a56ee9dab0 con 0x55a56ef3ae50 2024-04-25T01:16:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.833+0000 7f1e007f1740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a56ee46640 con 0x55a56ef409b0 2024-04-25T01:16:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.833+0000 7f1e007f1740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a56ee99a70 con 0x55a56ef3b840 2024-04-25T01:16:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.833+0000 7f1def400640 1 -- 172.21.15.31:0/2822992986 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1de4010ac0 con 0x7f1df8100810 2024-04-25T01:16:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.833+0000 7f1df7400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef3ae50 0x55a56ef3b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.833+0000 7f1df6a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a56ef3b840 0x55a56ef40340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.833+0000 7f1df6a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a56ef3b840 0x55a56ef40340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:38126/0 (socket says 172.21.15.31:38126) 2024-04-25T01:16:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.833+0000 7f1df6a00640 1 -- 172.21.15.31:0/3934458761 learned_addr learned my addr 172.21.15.31:0/3934458761 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:16:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.834+0000 7f1df6000640 1 --2- 172.21.15.31:0/3934458761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a56ef409b0 0x55a56ef42e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.834+0000 7f1df6a00640 1 -- 172.21.15.31:0/3934458761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a56ef409b0 msgr2=0x55a56ef42e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.834+0000 7f1df6a00640 1 --2- 172.21.15.31:0/3934458761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a56ef409b0 0x55a56ef42e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.834+0000 7f1df6a00640 1 -- 172.21.15.31:0/3934458761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef3ae50 msgr2=0x55a56ef3b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.834+0000 7f1df6a00640 1 --2- 172.21.15.31:0/3934458761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef3ae50 0x55a56ef3b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.834+0000 7f1df6a00640 1 -- 172.21.15.31:0/3934458761 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a56ebb2210 con 0x55a56ef3b840 2024-04-25T01:16:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.834+0000 7f1df7400640 1 --2- 172.21.15.31:0/3934458761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef3ae50 0x55a56ef3b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:16:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.834+0000 7f1df6a00640 1 --2- 172.21.15.31:0/3934458761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a56ef3b840 0x55a56ef40340 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1de8009d20 tx=0x7f1de8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5ee74079c512f11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:16:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.834+0000 7f1de3e00640 1 -- 172.21.15.31:0/3934458761 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de801e070 con 0x55a56ef3b840 2024-04-25T01:16:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.835+0000 7f1de3e00640 1 -- 172.21.15.31:0/3934458761 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1de8019b10 con 0x55a56ef3b840 2024-04-25T01:16:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.835+0000 7f1de3e00640 1 -- 172.21.15.31:0/3934458761 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de800ebb0 con 0x55a56ef3b840 2024-04-25T01:16:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.835+0000 7f1def400640 1 --2- 172.21.15.31:0/2822992986 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1ddc0a2c00 0x7f1ddc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.835+0000 7f1e007f1740 1 -- 172.21.15.31:0/3934458761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a56ef3b840 msgr2=0x55a56ef40340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.835+0000 7f1e007f1740 1 --2- 172.21.15.31:0/3934458761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a56ef3b840 0x55a56ef40340 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1de8009d20 tx=0x7f1de8011810 comp rx=0 tx=0).stop 2024-04-25T01:16:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.835+0000 7f1e007f1740 1 -- 172.21.15.31:0/3934458761 shutdown_connections 2024-04-25T01:16:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.835+0000 7f1e007f1740 1 --2- 172.21.15.31:0/3934458761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a56ef409b0 0x55a56ef42e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.835+0000 7f1e007f1740 1 --2- 172.21.15.31:0/3934458761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a56ef3b840 0x55a56ef40340 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.835+0000 7f1e007f1740 1 --2- 172.21.15.31:0/3934458761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef3ae50 0x55a56ef3b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.835+0000 7f1def400640 1 -- 172.21.15.31:0/2822992986 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f1de40cf2e0 con 0x7f1df8100810 2024-04-25T01:16:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.835+0000 7f1e007f1740 1 -- 172.21.15.31:0/3934458761 >> 172.21.15.31:0/3934458761 conn(0x55a56ef364c0 msgr2=0x55a56ef368c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:16:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.835+0000 7f1df7400640 1 --2- 172.21.15.31:0/2822992986 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1ddc0a2c00 0x7f1ddc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.836+0000 7f1e007f1740 1 -- 172.21.15.31:0/3934458761 shutdown_connections 2024-04-25T01:16:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.836+0000 7f1e007f1740 1 -- 172.21.15.31:0/3934458761 wait complete. 2024-04-25T01:16:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.836+0000 7f1e007f1740 1 -- start start 2024-04-25T01:16:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.836+0000 7f1df7400640 1 --2- 172.21.15.31:0/2822992986 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1ddc0a2c00 0x7f1ddc0a50c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55a56ef3c0a0 tx=0x7f1df003c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:16:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.836+0000 7f1e007f1740 1 Processor -- start 2024-04-25T01:16:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.837+0000 7f1e007f1740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a56ef3ae50 0x55a56ef450a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.837+0000 7f1e007f1740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a56ef3b840 0x55a56ef455e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.837+0000 7f1e007f1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef409b0 0x55a56ef4c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.837+0000 7f1e007f1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a56ef43960 con 0x55a56ef409b0 2024-04-25T01:16:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.837+0000 7f1e007f1740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55a56ee59990 con 0x55a56ef3ae50 2024-04-25T01:16:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.837+0000 7f1e007f1740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a56ef43ae0 con 0x55a56ef3b840 2024-04-25T01:16:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.837+0000 7f1e007f1740 10 client.0 _ll_register_callbacks cb 0x55a56ee5b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:16:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.837+0000 7f1df7400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef409b0 0x55a56ef4c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.837+0000 7f1df7400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef409b0 0x55a56ef4c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33924/0 (socket says 172.21.15.31:33924) 2024-04-25T01:16:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.837+0000 7f1df7400640 1 -- 172.21.15.31:0/1457692738 learned_addr learned my addr 172.21.15.31:0/1457692738 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:16:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.837+0000 7f1df6000640 1 --2- 172.21.15.31:0/1457692738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a56ef3b840 0x55a56ef455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.837+0000 7f1df6a00640 1 --2- 172.21.15.31:0/1457692738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a56ef3ae50 0x55a56ef450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.838+0000 7f1df7400640 1 -- 172.21.15.31:0/1457692738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a56ef3b840 msgr2=0x55a56ef455e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.838+0000 7f1df7400640 1 --2- 172.21.15.31:0/1457692738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a56ef3b840 0x55a56ef455e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.838+0000 7f1df7400640 1 -- 172.21.15.31:0/1457692738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a56ef3ae50 msgr2=0x55a56ef450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.838+0000 7f1df7400640 1 --2- 172.21.15.31:0/1457692738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a56ef3ae50 0x55a56ef450a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.838+0000 7f1df7400640 1 -- 172.21.15.31:0/1457692738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a56ec7e620 con 0x55a56ef409b0 2024-04-25T01:16:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.838+0000 7f1df7400640 1 --2- 172.21.15.31:0/1457692738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef409b0 0x55a56ef4c620 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1df002fd80 tx=0x7f1df002a940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57775c92dce15bcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:16:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.838+0000 7f1de0c00640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T01:16:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.838+0000 7f1de0c00640 1 -- 172.21.15.31:0/1457692738 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0034040 con 0x55a56ef409b0 2024-04-25T01:16:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.838+0000 7f1de0c00640 1 -- 172.21.15.31:0/1457692738 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f1df0030dc0 con 0x55a56ef409b0 2024-04-25T01:16:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.838+0000 7f1e007f1740 1 -- 172.21.15.31:0/1457692738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1de8019b10 con 0x55a56ef409b0 2024-04-25T01:16:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.839+0000 7f1de0c00640 1 -- 172.21.15.31:0/1457692738 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1df0003f50 con 0x55a56ef409b0 2024-04-25T01:16:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.839+0000 7f1e007f1740 10 client.15132 fetch_fsmap learned FSMap version 24 2024-04-25T01:16:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.839+0000 7f1e007f1740 1 -- 172.21.15.31:0/1457692738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a56ef4f6c0 con 0x55a56ef409b0 2024-04-25T01:16:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.839+0000 7f1de0c00640 1 -- 172.21.15.31:0/1457692738 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1de8019b10 con 0x55a56ef409b0 2024-04-25T01:16:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.839+0000 7f1de0c00640 1 -- 172.21.15.31:0/1457692738 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1df005a050 con 0x55a56ef409b0 2024-04-25T01:16:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.839+0000 7f1e007f1740 10 client.15132 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T01:16:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.839+0000 7f1e007f1740 10 client.15132 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:16:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.839+0000 7f1e007f1740 20 client.15132 populate_metadata read hostname 'smithi031' 2024-04-25T01:16:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.839+0000 7f1e007f1740 1 --2- 172.21.15.31:0/1457692738 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55a56ef50e00 0x55a56ef532c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.839+0000 7f1e007f1740 4 client.15132 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:16:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.839+0000 7f1e007f1740 1 -- 172.21.15.31:0/1457692738 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a56eeaaff0 con 0x55a56ef50e00 2024-04-25T01:16:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.840+0000 7f1df6a00640 1 --2- 172.21.15.31:0/1457692738 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55a56ef50e00 0x55a56ef532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.840+0000 7f1df6a00640 1 --2- 172.21.15.31:0/1457692738 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55a56ef50e00 0x55a56ef532c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bbbed936ee6ee4d server_cookie=877acf5b019dd7d1 in_seq=0 out_seq=0 2024-04-25T01:16:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.840+0000 7f1de0c00640 10 client.15132 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:16:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.841+0000 7f1de0c00640 1 -- 172.21.15.31:0/1457692738 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a56eeaaff0 con 0x55a56ef50e00 2024-04-25T01:16:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.841+0000 7f1de0c00640 10 client.15132 handle_command_reply: tid=0 2024-04-25T01:16:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.841+0000 7f1e007f1740 1 client.15132 shutdown 2024-04-25T01:16:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.841+0000 7f1e007f1740 1 -- 172.21.15.31:0/1457692738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef409b0 msgr2=0x55a56ef4c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.841+0000 7f1e007f1740 1 --2- 172.21.15.31:0/1457692738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef409b0 0x55a56ef4c620 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1df002fd80 tx=0x7f1df002a940 comp rx=0 tx=0).stop 2024-04-25T01:16:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.841+0000 7f1e007f1740 1 -- 172.21.15.31:0/1457692738 shutdown_connections 2024-04-25T01:16:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.841+0000 7f1e007f1740 1 --2- 172.21.15.31:0/1457692738 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55a56ef50e00 0x55a56ef532c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.841+0000 7f1e007f1740 1 --2- 172.21.15.31:0/1457692738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef409b0 0x55a56ef4c620 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.841+0000 7f1e007f1740 1 --2- 172.21.15.31:0/1457692738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a56ef3b840 0x55a56ef455e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.841+0000 7f1e007f1740 1 --2- 172.21.15.31:0/1457692738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a56ef3ae50 0x55a56ef450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.841+0000 7f1e007f1740 1 -- 172.21.15.31:0/1457692738 >> 172.21.15.31:0/1457692738 conn(0x55a56ef364c0 msgr2=0x55a56ef35620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:16:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.841+0000 7f1de0c00640 0 client.15132 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:16:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.842+0000 7f1e007f1740 1 -- 172.21.15.31:0/1457692738 shutdown_connections 2024-04-25T01:16:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.842+0000 7f1e007f1740 1 -- 172.21.15.31:0/1457692738 wait complete. 2024-04-25T01:16:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.842+0000 7f1e007f1740 20 client.15132 trim_cache size 0 max 16384 2024-04-25T01:16:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.858+0000 7f1e007f1740 1 Processor -- start 2024-04-25T01:16:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.858+0000 7f1e007f1740 1 -- start start 2024-04-25T01:16:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.858+0000 7f1e007f1740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a56ef3ae50 0x55a56ef40070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.858+0000 7f1e007f1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef3b840 0x55a56ef4c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.858+0000 7f1e007f1740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a56ef409b0 0x55a56ef394b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.858+0000 7f1e007f1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81ad2e0 con 0x55a56ef3b840 2024-04-25T01:16:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.858+0000 7f1e007f1740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81aabb0 con 0x55a56ef3ae50 2024-04-25T01:16:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.858+0000 7f1e007f1740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81a8a00 con 0x55a56ef409b0 2024-04-25T01:16:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.859+0000 7f1df6000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef3b840 0x55a56ef4c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.859+0000 7f1df6000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef3b840 0x55a56ef4c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33932/0 (socket says 172.21.15.31:33932) 2024-04-25T01:16:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.859+0000 7f1df6000640 1 -- 172.21.15.31:0/3106634707 learned_addr learned my addr 172.21.15.31:0/3106634707 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:16:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.859+0000 7f1df7400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a56ef409b0 0x55a56ef394b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.859+0000 7f1df6a00640 1 --2- 172.21.15.31:0/3106634707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a56ef3ae50 0x55a56ef40070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.859+0000 7f1df6000640 1 -- 172.21.15.31:0/3106634707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a56ef409b0 msgr2=0x55a56ef394b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.859+0000 7f1df6000640 1 --2- 172.21.15.31:0/3106634707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a56ef409b0 0x55a56ef394b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.859+0000 7f1df6000640 1 -- 172.21.15.31:0/3106634707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a56ef3ae50 msgr2=0x55a56ef40070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.859+0000 7f1df6000640 1 --2- 172.21.15.31:0/3106634707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a56ef3ae50 0x55a56ef40070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.859+0000 7f1df6000640 1 -- 172.21.15.31:0/3106634707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a56ef34a10 con 0x55a56ef3b840 2024-04-25T01:16:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.859+0000 7f1df6a00640 1 --2- 172.21.15.31:0/3106634707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a56ef3ae50 0x55a56ef40070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:16:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.859+0000 7f1df7400640 1 --2- 172.21.15.31:0/3106634707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a56ef409b0 0x55a56ef394b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:16:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.860+0000 7f1df6000640 1 --2- 172.21.15.31:0/3106634707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef3b840 0x55a56ef4c5e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1de40980c0 tx=0x7f1de409b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259c4dcbdafe4933 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:16:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.860+0000 7f1de3400640 1 -- 172.21.15.31:0/3106634707 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de40a4820 con 0x55a56ef3b840 2024-04-25T01:16:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.860+0000 7f1de3400640 1 -- 172.21.15.31:0/3106634707 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1de40ccf60 con 0x55a56ef3b840 2024-04-25T01:16:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.860+0000 7f1de3400640 1 -- 172.21.15.31:0/3106634707 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de409b310 con 0x55a56ef3b840 2024-04-25T01:16:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.860+0000 7f1e007f1740 1 -- 172.21.15.31:0/3106634707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef3b840 msgr2=0x55a56ef4c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.860+0000 7f1e007f1740 1 --2- 172.21.15.31:0/3106634707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef3b840 0x55a56ef4c5e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1de40980c0 tx=0x7f1de409b890 comp rx=0 tx=0).stop 2024-04-25T01:16:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.860+0000 7f1e007f1740 1 -- 172.21.15.31:0/3106634707 shutdown_connections 2024-04-25T01:16:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.860+0000 7f1e007f1740 1 --2- 172.21.15.31:0/3106634707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a56ef409b0 0x55a56ef394b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.860+0000 7f1e007f1740 1 --2- 172.21.15.31:0/3106634707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef3b840 0x55a56ef4c5e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.860+0000 7f1e007f1740 1 --2- 172.21.15.31:0/3106634707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a56ef3ae50 0x55a56ef40070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.860+0000 7f1e007f1740 1 -- 172.21.15.31:0/3106634707 >> 172.21.15.31:0/3106634707 conn(0x55a56ef364c0 msgr2=0x55a56ef368a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:16:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.861+0000 7f1e007f1740 1 -- 172.21.15.31:0/3106634707 shutdown_connections 2024-04-25T01:16:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.861+0000 7f1e007f1740 1 -- 172.21.15.31:0/3106634707 wait complete. 2024-04-25T01:16:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.861+0000 7f1e007f1740 1 -- start start 2024-04-25T01:16:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.861+0000 7f1e007f1740 1 Processor -- start 2024-04-25T01:16:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.862+0000 7f1e007f1740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a56ef3ae50 0x55a56ef4a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.862+0000 7f1e007f1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef3b840 0x55a56ef4a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.862+0000 7f1e007f1740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a56ef409b0 0x55a56ef4b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.862+0000 7f1e007f1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81a9070 con 0x55a56ef3b840 2024-04-25T01:16:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.862+0000 7f1e007f1740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81aa4e0 con 0x55a56ef3ae50 2024-04-25T01:16:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.862+0000 7f1e007f1740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81a9740 con 0x55a56ef409b0 2024-04-25T01:16:25.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.862+0000 7f1e007f1740 10 client.0 _ll_register_callbacks cb 0x7f1df0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:16:25.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.862+0000 7f1df6000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef3b840 0x55a56ef4a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:25.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.862+0000 7f1df6000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef3b840 0x55a56ef4a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33934/0 (socket says 172.21.15.31:33934) 2024-04-25T01:16:25.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.862+0000 7f1df6000640 1 -- 172.21.15.31:0/839511093 learned_addr learned my addr 172.21.15.31:0/839511093 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:16:25.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.862+0000 7f1df7400640 1 --2- 172.21.15.31:0/839511093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a56ef409b0 0x55a56ef4b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:25.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.862+0000 7f1df6a00640 1 --2- 172.21.15.31:0/839511093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a56ef3ae50 0x55a56ef4a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:25.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.862+0000 7f1df6000640 1 -- 172.21.15.31:0/839511093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a56ef409b0 msgr2=0x55a56ef4b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:25.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.862+0000 7f1df6000640 1 --2- 172.21.15.31:0/839511093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a56ef409b0 0x55a56ef4b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.862+0000 7f1df6000640 1 -- 172.21.15.31:0/839511093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a56ef3ae50 msgr2=0x55a56ef4a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:25.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.862+0000 7f1df6000640 1 --2- 172.21.15.31:0/839511093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a56ef3ae50 0x55a56ef4a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.862+0000 7f1df6000640 1 -- 172.21.15.31:0/839511093 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a56ef4bd00 con 0x55a56ef3b840 2024-04-25T01:16:25.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.862+0000 7f1df6a00640 1 --2- 172.21.15.31:0/839511093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a56ef3ae50 0x55a56ef4a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:16:25.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.863+0000 7f1df6000640 1 --2- 172.21.15.31:0/839511093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef3b840 0x55a56ef4a700 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1de4009380 tx=0x7f1de40a45c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b564215fac088d01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:16:25.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.863+0000 7f1de0c00640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T01:16:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.863+0000 7f1de0c00640 1 -- 172.21.15.31:0/839511093 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de40b2020 con 0x55a56ef3b840 2024-04-25T01:16:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.863+0000 7f1de0c00640 1 -- 172.21.15.31:0/839511093 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f1de4010640 con 0x55a56ef3b840 2024-04-25T01:16:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.863+0000 7f1e007f1740 1 -- 172.21.15.31:0/839511093 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1de40ccf60 con 0x55a56ef3b840 2024-04-25T01:16:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.863+0000 7f1de0c00640 1 -- 172.21.15.31:0/839511093 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1de40979b0 con 0x55a56ef3b840 2024-04-25T01:16:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.863+0000 7f1e007f1740 10 client.15138 fetch_fsmap learned FSMap version 24 2024-04-25T01:16:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.863+0000 7f1e007f1740 1 -- 172.21.15.31:0/839511093 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a56efb78f0 con 0x55a56ef3b840 2024-04-25T01:16:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.863+0000 7f1de0c00640 1 -- 172.21.15.31:0/839511093 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1de40ccf60 con 0x55a56ef3b840 2024-04-25T01:16:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.863+0000 7f1de0c00640 1 -- 172.21.15.31:0/839511093 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1de40ba200 con 0x55a56ef3b840 2024-04-25T01:16:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.864+0000 7f1e007f1740 10 client.15138 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T01:16:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.864+0000 7f1e007f1740 10 client.15138 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:16:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.864+0000 7f1e007f1740 20 client.15138 populate_metadata read hostname 'smithi031' 2024-04-25T01:16:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.864+0000 7f1e007f1740 1 --2- 172.21.15.31:0/839511093 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55a56efb8290 0x55a56efba6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.864+0000 7f1e007f1740 4 client.15138 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:16:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.864+0000 7f1e007f1740 1 -- 172.21.15.31:0/839511093 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a56eea5340 con 0x55a56efb8290 2024-04-25T01:16:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.864+0000 7f1df6a00640 1 --2- 172.21.15.31:0/839511093 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55a56efb8290 0x55a56efba6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.865+0000 7f1df6a00640 1 --2- 172.21.15.31:0/839511093 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55a56efb8290 0x55a56efba6b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99a5ac1630f5ab27 server_cookie=ed594921cc3f4f6d in_seq=0 out_seq=0 2024-04-25T01:16:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.865+0000 7f1de0c00640 10 client.15138 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:16:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.869+0000 7f1de0c00640 1 -- 172.21.15.31:0/839511093 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a56eea5340 con 0x55a56efb8290 2024-04-25T01:16:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.869+0000 7f1de0c00640 10 client.15138 handle_command_reply: tid=0 2024-04-25T01:16:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.869+0000 7f1e007f1740 1 client.15138 shutdown 2024-04-25T01:16:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.869+0000 7f1e007f1740 1 -- 172.21.15.31:0/839511093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef3b840 msgr2=0x55a56ef4a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.869+0000 7f1e007f1740 1 --2- 172.21.15.31:0/839511093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef3b840 0x55a56ef4a700 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1de4009380 tx=0x7f1de40a45c0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.869+0000 7f1e007f1740 1 -- 172.21.15.31:0/839511093 shutdown_connections 2024-04-25T01:16:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.869+0000 7f1e007f1740 1 --2- 172.21.15.31:0/839511093 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55a56efb8290 0x55a56efba6b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.869+0000 7f1de0c00640 0 client.15138 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:16:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.869+0000 7f1e007f1740 1 --2- 172.21.15.31:0/839511093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a56ef409b0 0x55a56ef4b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.869+0000 7f1e007f1740 1 --2- 172.21.15.31:0/839511093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a56ef3b840 0x55a56ef4a700 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.870+0000 7f1e007f1740 1 --2- 172.21.15.31:0/839511093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a56ef3ae50 0x55a56ef4a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.870+0000 7f1e007f1740 1 -- 172.21.15.31:0/839511093 >> 172.21.15.31:0/839511093 conn(0x55a56ef364c0 msgr2=0x55a56ef3fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:16:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.870+0000 7f1e007f1740 1 -- 172.21.15.31:0/839511093 shutdown_connections 2024-04-25T01:16:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.870+0000 7f1e007f1740 1 -- 172.21.15.31:0/839511093 wait complete. 2024-04-25T01:16:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.870+0000 7f1e007f1740 20 client.15138 trim_cache size 0 max 16384 2024-04-25T01:16:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:16:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:16:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:16:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:16:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.871+0000 7f1df7e00640 1 -- 172.21.15.31:0/2822992986 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1ddc0a2c00 msgr2=0x7f1ddc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.871+0000 7f1df7e00640 1 --2- 172.21.15.31:0/2822992986 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1ddc0a2c00 0x7f1ddc0a50c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55a56ef3c0a0 tx=0x7f1df003c040 comp rx=0 tx=0).stop 2024-04-25T01:16:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.871+0000 7f1df7e00640 1 -- 172.21.15.31:0/2822992986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df8100810 msgr2=0x7f1df8199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.871+0000 7f1df7e00640 1 --2- 172.21.15.31:0/2822992986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df8100810 0x7f1df8199ff0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1de400e9f0 tx=0x7f1de400eeb0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.871+0000 7f1df7e00640 1 -- 172.21.15.31:0/2822992986 shutdown_connections 2024-04-25T01:16:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.871+0000 7f1df7e00640 1 --2- 172.21.15.31:0/2822992986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1df81011b0 0x7f1df81a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.871+0000 7f1df7e00640 1 --2- 172.21.15.31:0/2822992986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df8100810 0x7f1df8199ff0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.871+0000 7f1df7e00640 1 --2- 172.21.15.31:0/2822992986 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1ddc0a2c00 0x7f1ddc0a50c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.871+0000 7f1df7e00640 1 --2- 172.21.15.31:0/2822992986 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1df80ffeb0 0x7f1df8199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.871+0000 7f1df7e00640 1 -- 172.21.15.31:0/2822992986 >> 172.21.15.31:0/2822992986 conn(0x7f1df80fba30 msgr2=0x7f1df80fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:16:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.871+0000 7f1df7e00640 1 -- 172.21.15.31:0/2822992986 shutdown_connections 2024-04-25T01:16:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:25.871+0000 7f1df7e00640 1 -- 172.21.15.31:0/2822992986 wait complete. 2024-04-25T01:16:26.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:25 smithi031 ceph-mon[28053]: pgmap v405: 97 pgs: 97 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-25T01:16:26.443 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:25 smithi169 ceph-mon[32759]: pgmap v405: 97 pgs: 97 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-25T01:16:26.446 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:25 smithi040 ceph-mon[32518]: pgmap v405: 97 pgs: 97 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-25T01:16:28.248 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:27 smithi040 ceph-mon[32518]: pgmap v406: 97 pgs: 97 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-25T01:16:28.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:27 smithi031 ceph-mon[28053]: pgmap v406: 97 pgs: 97 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-25T01:16:28.443 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:27 smithi169 ceph-mon[32759]: pgmap v406: 97 pgs: 97 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-25T01:16:30.253 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:29 smithi031 ceph-mon[28053]: pgmap v407: 97 pgs: 97 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-25T01:16:30.443 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:29 smithi169 ceph-mon[32759]: pgmap v407: 97 pgs: 97 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-25T01:16:30.446 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:29 smithi040 ceph-mon[32518]: pgmap v407: 97 pgs: 97 active+clean; 650 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-25T01:16:31.253 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:30 smithi031 ceph-mon[28053]: pgmap v408: 97 pgs: 97 active+clean; 651 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T01:16:31.443 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:30 smithi169 ceph-mon[32759]: pgmap v408: 97 pgs: 97 active+clean; 651 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T01:16:31.445 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:30 smithi040 ceph-mon[32518]: pgmap v408: 97 pgs: 97 active+clean; 651 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T01:16:33.503 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:33 smithi031 ceph-mon[28053]: pgmap v409: 97 pgs: 97 active+clean; 651 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:16:33.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:33 smithi169 ceph-mon[32759]: pgmap v409: 97 pgs: 97 active+clean; 651 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:16:33.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:33 smithi040 ceph-mon[32518]: pgmap v409: 97 pgs: 97 active+clean; 651 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:16:35.503 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:35 smithi031 ceph-mon[28053]: pgmap v410: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T01:16:35.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:35 smithi169 ceph-mon[32759]: pgmap v410: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T01:16:35.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:35 smithi040 ceph-mon[32518]: pgmap v410: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T01:16:37.503 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:37 smithi031 ceph-mon[28053]: pgmap v411: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T01:16:37.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:37 smithi169 ceph-mon[32759]: pgmap v411: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T01:16:37.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:37 smithi040 ceph-mon[32518]: pgmap v411: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T01:16:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.015+0000 7f4f04e00640 1 -- 172.21.15.31:0/2570272258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f00075780 msgr2=0x7f4f00073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.015+0000 7f4f04e00640 1 --2- 172.21.15.31:0/2570272258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f00075780 0x7f4f00073bd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009900 tx=0x7f4ef002a410 comp rx=0 tx=0).stop 2024-04-25T01:16:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.015+0000 7f4f04e00640 1 -- 172.21.15.31:0/2570272258 shutdown_connections 2024-04-25T01:16:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.015+0000 7f4f04e00640 1 --2- 172.21.15.31:0/2570272258 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f00109490 0x7f4f0010b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.015+0000 7f4f04e00640 1 --2- 172.21.15.31:0/2570272258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f000741a0 0x7f4f00074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.015+0000 7f4f04e00640 1 --2- 172.21.15.31:0/2570272258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f00075780 0x7f4f00073bd0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.015+0000 7f4f04e00640 1 -- 172.21.15.31:0/2570272258 >> 172.21.15.31:0/2570272258 conn(0x7f4f000fcb40 msgr2=0x7f4f000fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:16:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.015+0000 7f4f04e00640 1 -- 172.21.15.31:0/2570272258 shutdown_connections 2024-04-25T01:16:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.015+0000 7f4f04e00640 1 -- 172.21.15.31:0/2570272258 wait complete. 2024-04-25T01:16:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.017+0000 7f4f04e00640 1 Processor -- start 2024-04-25T01:16:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.017+0000 7f4f04e00640 1 -- start start 2024-04-25T01:16:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.017+0000 7f4f04e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f000741a0 0x7f4f00199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.017+0000 7f4f04e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f00075780 0x7f4f0019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.017+0000 7f4f04e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f00109490 0x7f4f001a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.017+0000 7f4f04e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0010f460 con 0x7f4f00075780 2024-04-25T01:16:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.017+0000 7f4f04e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0010f2e0 con 0x7f4f000741a0 2024-04-25T01:16:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.017+0000 7f4f04e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0010f5e0 con 0x7f4f00109490 2024-04-25T01:16:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.017+0000 7f4ef7e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f00075780 0x7f4f0019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.017+0000 7f4eff400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f000741a0 0x7f4f00199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.017+0000 7f4ef7e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f00075780 0x7f4f0019a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46428/0 (socket says 172.21.15.31:46428) 2024-04-25T01:16:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.017+0000 7f4ef7e00640 1 -- 172.21.15.31:0/1648241944 learned_addr learned my addr 172.21.15.31:0/1648241944 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:16:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.018+0000 7f4effe00640 1 --2- 172.21.15.31:0/1648241944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f00109490 0x7f4f001a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.018+0000 7f4ef7e00640 1 -- 172.21.15.31:0/1648241944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f00109490 msgr2=0x7f4f001a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.018+0000 7f4ef7e00640 1 --2- 172.21.15.31:0/1648241944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f00109490 0x7f4f001a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.018+0000 7f4ef7e00640 1 -- 172.21.15.31:0/1648241944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f000741a0 msgr2=0x7f4f00199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.018+0000 7f4ef7e00640 1 --2- 172.21.15.31:0/1648241944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f000741a0 0x7f4f00199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.018+0000 7f4ef7e00640 1 -- 172.21.15.31:0/1648241944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f001a1750 con 0x7f4f00075780 2024-04-25T01:16:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.018+0000 7f4eff400640 1 --2- 172.21.15.31:0/1648241944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f000741a0 0x7f4f00199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:16:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.018+0000 7f4ef7e00640 1 --2- 172.21.15.31:0/1648241944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f00075780 0x7f4f0019a0c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4ee800e9f0 tx=0x7f4ee800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:16:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.018+0000 7f4efcc00640 1 -- 172.21.15.31:0/1648241944 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee800cd80 con 0x7f4f00075780 2024-04-25T01:16:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.019+0000 7f4efcc00640 1 -- 172.21.15.31:0/1648241944 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ee8010910 con 0x7f4f00075780 2024-04-25T01:16:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.019+0000 7f4efcc00640 1 -- 172.21.15.31:0/1648241944 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8018990 con 0x7f4f00075780 2024-04-25T01:16:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.019+0000 7f4f04e00640 1 -- 172.21.15.31:0/1648241944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f001a1a40 con 0x7f4f00075780 2024-04-25T01:16:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.019+0000 7f4f04e00640 1 -- 172.21.15.31:0/1648241944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f001a1f40 con 0x7f4f00075780 2024-04-25T01:16:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.022+0000 7f4efcc00640 1 -- 172.21.15.31:0/1648241944 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4ee8010ac0 con 0x7f4f00075780 2024-04-25T01:16:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.023+0000 7f4efcc00640 1 --2- 172.21.15.31:0/1648241944 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f4ee40a2c00 0x7f4ee40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.023+0000 7f4f08f22740 1 Processor -- start 2024-04-25T01:16:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.023+0000 7f4efcc00640 1 -- 172.21.15.31:0/1648241944 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f4ee80cf2c0 con 0x7f4f00075780 2024-04-25T01:16:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.024+0000 7f4f08f22740 1 -- start start 2024-04-25T01:16:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.024+0000 7f4eff400640 1 --2- 172.21.15.31:0/1648241944 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f4ee40a2c00 0x7f4ee40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.024+0000 7f4f08f22740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c12303e50 0x558c12304270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.024+0000 7f4f08f22740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558c12304840 0x558c12309340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.024+0000 7f4f08f22740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c123099b0 0x558c1230be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.024+0000 7f4f08f22740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558c12266ab0 con 0x558c12303e50 2024-04-25T01:16:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.024+0000 7f4f08f22740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x558c1220f640 con 0x558c12304840 2024-04-25T01:16:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.024+0000 7f4f08f22740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x558c12262a70 con 0x558c123099b0 2024-04-25T01:16:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.024+0000 7f4ef7e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558c12304840 0x558c12309340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.024+0000 7f4eff400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c12303e50 0x558c12304270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.024+0000 7f4effe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c123099b0 0x558c1230be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.024+0000 7f4effe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c123099b0 0x558c1230be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:57428/0 (socket says 172.21.15.31:57428) 2024-04-25T01:16:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.025+0000 7f4eff400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c12303e50 0x558c12304270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46440/0 (socket says 172.21.15.31:46440) 2024-04-25T01:16:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.025+0000 7f4effe00640 1 -- 172.21.15.31:0/2196836660 learned_addr learned my addr 172.21.15.31:0/2196836660 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:16:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.024+0000 7f4ef7e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558c12304840 0x558c12309340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:34002/0 (socket says 172.21.15.31:34002) 2024-04-25T01:16:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.025+0000 7f4eff400640 1 --2- 172.21.15.31:0/1648241944 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f4ee40a2c00 0x7f4ee40a50c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0031440 tx=0x7f4ef0004490 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:16:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.025+0000 7f4ef7e00640 1 -- 172.21.15.31:0/2196836660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c123099b0 msgr2=0x558c1230be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.025+0000 7f4ef7e00640 1 --2- 172.21.15.31:0/2196836660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c123099b0 0x558c1230be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.025+0000 7f4ef7e00640 1 -- 172.21.15.31:0/2196836660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c12303e50 msgr2=0x558c12304270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.025+0000 7f4ef7e00640 1 --2- 172.21.15.31:0/2196836660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c12303e50 0x558c12304270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.025+0000 7f4ef7e00640 1 -- 172.21.15.31:0/2196836660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c11f7b210 con 0x558c12304840 2024-04-25T01:16:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.025+0000 7f4ef7e00640 1 --2- 172.21.15.31:0/2196836660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558c12304840 0x558c12309340 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4ee800e9c0 tx=0x7f4ee809bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aae71de5ab1dd155 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:16:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.025+0000 7f4ef4c00640 1 -- 172.21.15.31:0/2196836660 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8099ec0 con 0x558c12304840 2024-04-25T01:16:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.025+0000 7f4ef4c00640 1 -- 172.21.15.31:0/2196836660 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ee8098230 con 0x558c12304840 2024-04-25T01:16:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.025+0000 7f4ef4c00640 1 -- 172.21.15.31:0/2196836660 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8097910 con 0x558c12304840 2024-04-25T01:16:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.026+0000 7f4f08f22740 1 -- 172.21.15.31:0/2196836660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558c12304840 msgr2=0x558c12309340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.026+0000 7f4f08f22740 1 --2- 172.21.15.31:0/2196836660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558c12304840 0x558c12309340 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4ee800e9c0 tx=0x7f4ee809bc80 comp rx=0 tx=0).stop 2024-04-25T01:16:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.026+0000 7f4f08f22740 1 -- 172.21.15.31:0/2196836660 shutdown_connections 2024-04-25T01:16:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.026+0000 7f4f08f22740 1 --2- 172.21.15.31:0/2196836660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c123099b0 0x558c1230be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.026+0000 7f4f08f22740 1 --2- 172.21.15.31:0/2196836660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558c12304840 0x558c12309340 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.026+0000 7f4f08f22740 1 --2- 172.21.15.31:0/2196836660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c12303e50 0x558c12304270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.026+0000 7f4f08f22740 1 -- 172.21.15.31:0/2196836660 >> 172.21.15.31:0/2196836660 conn(0x558c122ff4c0 msgr2=0x558c122ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:16:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.027+0000 7f4f08f22740 1 -- 172.21.15.31:0/2196836660 shutdown_connections 2024-04-25T01:16:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.027+0000 7f4f08f22740 1 -- 172.21.15.31:0/2196836660 wait complete. 2024-04-25T01:16:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.027+0000 7f4f08f22740 1 -- start start 2024-04-25T01:16:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.027+0000 7f4f08f22740 1 Processor -- start 2024-04-25T01:16:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.028+0000 7f4f08f22740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c12303e50 0x558c1230e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.028+0000 7f4f08f22740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558c12304840 0x558c1230e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.028+0000 7f4f08f22740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c123099b0 0x558c123155c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.028+0000 7f4f08f22740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558c1230c8a0 con 0x558c123099b0 2024-04-25T01:16:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.028+0000 7f4f08f22740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x558c12222990 con 0x558c12304840 2024-04-25T01:16:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.028+0000 7f4f08f22740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x558c1230ca20 con 0x558c12303e50 2024-04-25T01:16:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.028+0000 7f4f08f22740 10 client.0 _ll_register_callbacks cb 0x558c12224950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:16:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.028+0000 7f4effe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c123099b0 0x558c123155c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.028+0000 7f4eff400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c12303e50 0x558c1230e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.028+0000 7f4effe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c123099b0 0x558c123155c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46450/0 (socket says 172.21.15.31:46450) 2024-04-25T01:16:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.028+0000 7f4effe00640 1 -- 172.21.15.31:0/47984884 learned_addr learned my addr 172.21.15.31:0/47984884 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:16:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.028+0000 7f4ef7e00640 1 --2- 172.21.15.31:0/47984884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558c12304840 0x558c1230e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.029+0000 7f4effe00640 1 -- 172.21.15.31:0/47984884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c12303e50 msgr2=0x558c1230e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.029+0000 7f4effe00640 1 --2- 172.21.15.31:0/47984884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c12303e50 0x558c1230e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.029+0000 7f4effe00640 1 -- 172.21.15.31:0/47984884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558c12304840 msgr2=0x558c1230e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.029+0000 7f4effe00640 1 --2- 172.21.15.31:0/47984884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558c12304840 0x558c1230e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.029+0000 7f4effe00640 1 -- 172.21.15.31:0/47984884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c12047620 con 0x558c123099b0 2024-04-25T01:16:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.029+0000 7f4eff400640 1 --2- 172.21.15.31:0/47984884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c12303e50 0x558c1230e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:16:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.029+0000 7f4ef7e00640 1 --2- 172.21.15.31:0/47984884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558c12304840 0x558c1230e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:16:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.029+0000 7f4effe00640 1 --2- 172.21.15.31:0/47984884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c123099b0 0x558c123155c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8003cb0 tx=0x7f4ef8004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae130cfc11297497 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:16:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.029+0000 7f4eed600640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T01:16:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.029+0000 7f4eed600640 1 -- 172.21.15.31:0/47984884 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8011650 con 0x558c123099b0 2024-04-25T01:16:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.029+0000 7f4eed600640 1 -- 172.21.15.31:0/47984884 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f4ef80117f0 con 0x558c123099b0 2024-04-25T01:16:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.029+0000 7f4f08f22740 1 -- 172.21.15.31:0/47984884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ee8098230 con 0x558c123099b0 2024-04-25T01:16:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.030+0000 7f4eed600640 1 -- 172.21.15.31:0/47984884 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4ef80119f0 con 0x558c123099b0 2024-04-25T01:16:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.030+0000 7f4f08f22740 10 client.15150 fetch_fsmap learned FSMap version 24 2024-04-25T01:16:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.030+0000 7f4f08f22740 1 -- 172.21.15.31:0/47984884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558c12318670 con 0x558c123099b0 2024-04-25T01:16:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.030+0000 7f4eed600640 1 -- 172.21.15.31:0/47984884 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ee8098230 con 0x558c123099b0 2024-04-25T01:16:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.030+0000 7f4eed600640 1 -- 172.21.15.31:0/47984884 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4ef8024050 con 0x558c123099b0 2024-04-25T01:16:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.030+0000 7f4f08f22740 10 client.15150 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T01:16:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.030+0000 7f4f08f22740 10 client.15150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:16:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.031+0000 7f4f08f22740 20 client.15150 populate_metadata read hostname 'smithi031' 2024-04-25T01:16:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.031+0000 7f4f08f22740 1 --2- 172.21.15.31:0/47984884 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x558c12319d60 0x558c1231c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.031+0000 7f4f08f22740 4 client.15150 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:16:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.031+0000 7f4f08f22740 1 -- 172.21.15.31:0/47984884 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c12273ff0 con 0x558c12319d60 2024-04-25T01:16:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.031+0000 7f4eff400640 1 --2- 172.21.15.31:0/47984884 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x558c12319d60 0x558c1231c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.032+0000 7f4eed600640 10 client.15150 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:16:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.032+0000 7f4eff400640 1 --2- 172.21.15.31:0/47984884 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x558c12319d60 0x558c1231c190 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf4c26d4c3e509fc server_cookie=fc69997c0014a6e4 in_seq=0 out_seq=0 2024-04-25T01:16:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.033+0000 7f4eed600640 1 -- 172.21.15.31:0/47984884 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558c12273ff0 con 0x558c12319d60 2024-04-25T01:16:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.033+0000 7f4eed600640 10 client.15150 handle_command_reply: tid=0 2024-04-25T01:16:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.033+0000 7f4f08f22740 1 client.15150 shutdown 2024-04-25T01:16:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.033+0000 7f4f08f22740 1 -- 172.21.15.31:0/47984884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c123099b0 msgr2=0x558c123155c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.033+0000 7f4f08f22740 1 --2- 172.21.15.31:0/47984884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c123099b0 0x558c123155c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8003cb0 tx=0x7f4ef8004410 comp rx=0 tx=0).stop 2024-04-25T01:16:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.033+0000 7f4f08f22740 1 -- 172.21.15.31:0/47984884 shutdown_connections 2024-04-25T01:16:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.033+0000 7f4f08f22740 1 --2- 172.21.15.31:0/47984884 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x558c12319d60 0x558c1231c190 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.033+0000 7f4f08f22740 1 --2- 172.21.15.31:0/47984884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c123099b0 0x558c123155c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.033+0000 7f4f08f22740 1 --2- 172.21.15.31:0/47984884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558c12304840 0x558c1230e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.033+0000 7f4f08f22740 1 --2- 172.21.15.31:0/47984884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c12303e50 0x558c1230e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.033+0000 7f4f08f22740 1 -- 172.21.15.31:0/47984884 >> 172.21.15.31:0/47984884 conn(0x558c122ff4c0 msgr2=0x558c122fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:16:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.033+0000 7f4eed600640 0 client.15150 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:16:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.033+0000 7f4f08f22740 1 -- 172.21.15.31:0/47984884 shutdown_connections 2024-04-25T01:16:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.033+0000 7f4f08f22740 1 -- 172.21.15.31:0/47984884 wait complete. 2024-04-25T01:16:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.033+0000 7f4f08f22740 20 client.15150 trim_cache size 0 max 16384 2024-04-25T01:16:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.051+0000 7f4f08f22740 1 Processor -- start 2024-04-25T01:16:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.051+0000 7f4f08f22740 1 -- start start 2024-04-25T01:16:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.052+0000 7f4f08f22740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c12303e50 0x558c123021a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.052+0000 7f4f08f22740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558c12304840 0x558c12315580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.052+0000 7f4f08f22740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c123099b0 0x558c123026e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.052+0000 7f4f08f22740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4f001ad570 con 0x558c12303e50 2024-04-25T01:16:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.052+0000 7f4f08f22740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4f001aae40 con 0x558c12304840 2024-04-25T01:16:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.052+0000 7f4f08f22740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4f001a8cb0 con 0x558c123099b0 2024-04-25T01:16:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.052+0000 7f4eff400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c12303e50 0x558c123021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.052+0000 7f4ef7e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558c12304840 0x558c12315580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.052+0000 7f4ef7e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558c12304840 0x558c12315580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:34012/0 (socket says 172.21.15.31:34012) 2024-04-25T01:16:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.052+0000 7f4ef7e00640 1 -- 172.21.15.31:0/4106938153 learned_addr learned my addr 172.21.15.31:0/4106938153 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:16:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.052+0000 7f4effe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c123099b0 0x558c123026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.052+0000 7f4ef7e00640 1 -- 172.21.15.31:0/4106938153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c123099b0 msgr2=0x558c123026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.053+0000 7f4ef7e00640 1 --2- 172.21.15.31:0/4106938153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c123099b0 0x558c123026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.053+0000 7f4ef7e00640 1 -- 172.21.15.31:0/4106938153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c12303e50 msgr2=0x558c123021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.053+0000 7f4ef7e00640 1 --2- 172.21.15.31:0/4106938153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c12303e50 0x558c123021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.053+0000 7f4ef7e00640 1 -- 172.21.15.31:0/4106938153 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c122ff2f0 con 0x558c12304840 2024-04-25T01:16:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.053+0000 7f4ef7e00640 1 --2- 172.21.15.31:0/4106938153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558c12304840 0x558c12315580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8009380 tx=0x7f4ee8018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72b72c8fe4c24f8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:16:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.053+0000 7f4eefe00640 1 -- 172.21.15.31:0/4106938153 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8098230 con 0x558c12304840 2024-04-25T01:16:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.053+0000 7f4eefe00640 1 -- 172.21.15.31:0/4106938153 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ee8018b30 con 0x558c12304840 2024-04-25T01:16:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.053+0000 7f4eefe00640 1 -- 172.21.15.31:0/4106938153 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee809ace0 con 0x558c12304840 2024-04-25T01:16:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.054+0000 7f4f08f22740 1 -- 172.21.15.31:0/4106938153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558c12304840 msgr2=0x558c12315580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.054+0000 7f4f08f22740 1 --2- 172.21.15.31:0/4106938153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558c12304840 0x558c12315580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8009380 tx=0x7f4ee8018420 comp rx=0 tx=0).stop 2024-04-25T01:16:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.054+0000 7f4f08f22740 1 -- 172.21.15.31:0/4106938153 shutdown_connections 2024-04-25T01:16:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.054+0000 7f4f08f22740 1 --2- 172.21.15.31:0/4106938153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c123099b0 0x558c123026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.054+0000 7f4f08f22740 1 --2- 172.21.15.31:0/4106938153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558c12304840 0x558c12315580 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.054+0000 7f4f08f22740 1 --2- 172.21.15.31:0/4106938153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c12303e50 0x558c123021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.054+0000 7f4f08f22740 1 -- 172.21.15.31:0/4106938153 >> 172.21.15.31:0/4106938153 conn(0x558c122ff4c0 msgr2=0x558c1230e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:16:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.054+0000 7f4f08f22740 1 -- 172.21.15.31:0/4106938153 shutdown_connections 2024-04-25T01:16:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.054+0000 7f4f08f22740 1 -- 172.21.15.31:0/4106938153 wait complete. 2024-04-25T01:16:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.054+0000 7f4f08f22740 1 -- start start 2024-04-25T01:16:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.055+0000 7f4f08f22740 1 Processor -- start 2024-04-25T01:16:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.055+0000 7f4f08f22740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558c12303e50 0x558c12312fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.055+0000 7f4f08f22740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c12304840 0x558c123134f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.055+0000 7f4f08f22740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c123099b0 0x558c12314440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.055+0000 7f4f08f22740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4f001a9300 con 0x558c12304840 2024-04-25T01:16:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.055+0000 7f4f08f22740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4f001aa770 con 0x558c12303e50 2024-04-25T01:16:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.055+0000 7f4f08f22740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4f001a99d0 con 0x558c123099b0 2024-04-25T01:16:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.055+0000 7f4f08f22740 10 client.0 _ll_register_callbacks cb 0x7f4ef8009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:16:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.055+0000 7f4ef7e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c12304840 0x558c123134f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.055+0000 7f4ef7e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c12304840 0x558c123134f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46472/0 (socket says 172.21.15.31:46472) 2024-04-25T01:16:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.055+0000 7f4ef7e00640 1 -- 172.21.15.31:0/4005868352 learned_addr learned my addr 172.21.15.31:0/4005868352 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:16:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.055+0000 7f4effe00640 1 --2- 172.21.15.31:0/4005868352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c123099b0 0x558c12314440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.055+0000 7f4eff400640 1 --2- 172.21.15.31:0/4005868352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558c12303e50 0x558c12312fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.056+0000 7f4ef7e00640 1 -- 172.21.15.31:0/4005868352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c123099b0 msgr2=0x558c12314440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.056+0000 7f4ef7e00640 1 --2- 172.21.15.31:0/4005868352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c123099b0 0x558c12314440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.056+0000 7f4ef7e00640 1 -- 172.21.15.31:0/4005868352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558c12303e50 msgr2=0x558c12312fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.056+0000 7f4ef7e00640 1 --2- 172.21.15.31:0/4005868352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558c12303e50 0x558c12312fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.056+0000 7f4ef7e00640 1 -- 172.21.15.31:0/4005868352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c1230d120 con 0x558c12304840 2024-04-25T01:16:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.056+0000 7f4effe00640 1 --2- 172.21.15.31:0/4005868352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c123099b0 0x558c12314440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:16:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.056+0000 7f4ef7e00640 1 --2- 172.21.15.31:0/4005868352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c12304840 0x558c123134f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4ee809b8c0 tx=0x7f4ee8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dccfed1eba74daf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:16:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.056+0000 7f4eed600640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T01:16:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.056+0000 7f4eed600640 1 -- 172.21.15.31:0/4005868352 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee80a9070 con 0x558c12304840 2024-04-25T01:16:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.056+0000 7f4eed600640 1 -- 172.21.15.31:0/4005868352 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f4ee8010430 con 0x558c12304840 2024-04-25T01:16:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.056+0000 7f4f08f22740 1 -- 172.21.15.31:0/4005868352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ee8018b30 con 0x558c12304840 2024-04-25T01:16:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.057+0000 7f4eed600640 1 -- 172.21.15.31:0/4005868352 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4ee80ba040 con 0x558c12304840 2024-04-25T01:16:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.057+0000 7f4f08f22740 10 client.15156 fetch_fsmap learned FSMap version 24 2024-04-25T01:16:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.057+0000 7f4f08f22740 1 -- 172.21.15.31:0/4005868352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558c12380370 con 0x558c12304840 2024-04-25T01:16:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.057+0000 7f4eed600640 1 -- 172.21.15.31:0/4005868352 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ee8018b30 con 0x558c12304840 2024-04-25T01:16:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.057+0000 7f4eed600640 1 -- 172.21.15.31:0/4005868352 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4ee80ac030 con 0x558c12304840 2024-04-25T01:16:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.057+0000 7f4f08f22740 10 client.15156 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T01:16:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.057+0000 7f4f08f22740 10 client.15156 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:16:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.057+0000 7f4f08f22740 20 client.15156 populate_metadata read hostname 'smithi031' 2024-04-25T01:16:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.057+0000 7f4f08f22740 1 --2- 172.21.15.31:0/4005868352 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x558c123808a0 0x558c12382c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.057+0000 7f4f08f22740 4 client.15156 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:16:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.057+0000 7f4f08f22740 1 -- 172.21.15.31:0/4005868352 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558c1226e340 con 0x558c123808a0 2024-04-25T01:16:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.058+0000 7f4eff400640 1 --2- 172.21.15.31:0/4005868352 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x558c123808a0 0x558c12382c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.058+0000 7f4eff400640 1 --2- 172.21.15.31:0/4005868352 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x558c123808a0 0x558c12382c90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f54ad4d4328e10ac server_cookie=72f207b813eb2156 in_seq=0 out_seq=0 2024-04-25T01:16:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.058+0000 7f4eed600640 10 client.15156 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:16:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.066+0000 7f4eed600640 1 -- 172.21.15.31:0/4005868352 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558c1226e340 con 0x558c123808a0 2024-04-25T01:16:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.066+0000 7f4eed600640 10 client.15156 handle_command_reply: tid=0 2024-04-25T01:16:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.066+0000 7f4f08f22740 1 client.15156 shutdown 2024-04-25T01:16:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.066+0000 7f4f08f22740 1 -- 172.21.15.31:0/4005868352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c12304840 msgr2=0x558c123134f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.066+0000 7f4f08f22740 1 --2- 172.21.15.31:0/4005868352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c12304840 0x558c123134f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4ee809b8c0 tx=0x7f4ee8009310 comp rx=0 tx=0).stop 2024-04-25T01:16:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.066+0000 7f4f08f22740 1 -- 172.21.15.31:0/4005868352 shutdown_connections 2024-04-25T01:16:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.066+0000 7f4f08f22740 1 --2- 172.21.15.31:0/4005868352 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x558c123808a0 0x558c12382c90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.066+0000 7f4f08f22740 1 --2- 172.21.15.31:0/4005868352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558c123099b0 0x558c12314440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.066+0000 7f4f08f22740 1 --2- 172.21.15.31:0/4005868352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c12304840 0x558c123134f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.066+0000 7f4f08f22740 1 --2- 172.21.15.31:0/4005868352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558c12303e50 0x558c12312fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.066+0000 7f4eed600640 0 client.15156 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:16:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.066+0000 7f4f08f22740 1 -- 172.21.15.31:0/4005868352 >> 172.21.15.31:0/4005868352 conn(0x558c122ff4c0 msgr2=0x558c1230e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:16:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.066+0000 7f4f08f22740 1 -- 172.21.15.31:0/4005868352 shutdown_connections 2024-04-25T01:16:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.066+0000 7f4f08f22740 1 -- 172.21.15.31:0/4005868352 wait complete. 2024-04-25T01:16:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.066+0000 7f4f08f22740 20 client.15156 trim_cache size 0 max 16384 2024-04-25T01:16:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:16:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:16:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:16:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:16:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.067+0000 7f4f04e00640 1 -- 172.21.15.31:0/1648241944 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f4ee40a2c00 msgr2=0x7f4ee40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.067+0000 7f4f04e00640 1 --2- 172.21.15.31:0/1648241944 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f4ee40a2c00 0x7f4ee40a50c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0031440 tx=0x7f4ef0004490 comp rx=0 tx=0).stop 2024-04-25T01:16:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.067+0000 7f4f04e00640 1 -- 172.21.15.31:0/1648241944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f00075780 msgr2=0x7f4f0019a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.067+0000 7f4f04e00640 1 --2- 172.21.15.31:0/1648241944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f00075780 0x7f4f0019a0c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4ee800e9f0 tx=0x7f4ee800eeb0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.067+0000 7f4f04e00640 1 -- 172.21.15.31:0/1648241944 shutdown_connections 2024-04-25T01:16:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.067+0000 7f4f04e00640 1 --2- 172.21.15.31:0/1648241944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f00109490 0x7f4f001a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.067+0000 7f4f04e00640 1 --2- 172.21.15.31:0/1648241944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f00075780 0x7f4f0019a0c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.067+0000 7f4f04e00640 1 --2- 172.21.15.31:0/1648241944 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f4ee40a2c00 0x7f4ee40a50c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.067+0000 7f4f04e00640 1 --2- 172.21.15.31:0/1648241944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f000741a0 0x7f4f00199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.067+0000 7f4f04e00640 1 -- 172.21.15.31:0/1648241944 >> 172.21.15.31:0/1648241944 conn(0x7f4f000fcb40 msgr2=0x7f4f00105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:16:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.067+0000 7f4f04e00640 1 -- 172.21.15.31:0/1648241944 shutdown_connections 2024-04-25T01:16:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:39.067+0000 7f4f04e00640 1 -- 172.21.15.31:0/1648241944 wait complete. 2024-04-25T01:16:39.503 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:39 smithi031 ceph-mon[28053]: pgmap v412: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T01:16:39.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:39 smithi169 ceph-mon[32759]: pgmap v412: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T01:16:39.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:39 smithi040 ceph-mon[32518]: pgmap v412: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T01:16:41.503 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:41 smithi031 ceph-mon[28053]: pgmap v413: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T01:16:41.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:41 smithi169 ceph-mon[32759]: pgmap v413: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T01:16:41.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:41 smithi040 ceph-mon[32518]: pgmap v413: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T01:16:43.503 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:43 smithi031 ceph-mon[28053]: pgmap v414: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-25T01:16:43.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:43 smithi169 ceph-mon[32759]: pgmap v414: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-25T01:16:43.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:43 smithi040 ceph-mon[32518]: pgmap v414: 97 pgs: 97 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-25T01:16:45.503 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:45 smithi031 ceph-mon[28053]: pgmap v415: 97 pgs: 97 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T01:16:45.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:45 smithi169 ceph-mon[32759]: pgmap v415: 97 pgs: 97 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T01:16:45.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:45 smithi040 ceph-mon[32518]: pgmap v415: 97 pgs: 97 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T01:16:47.503 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:47 smithi031 ceph-mon[28053]: pgmap v416: 97 pgs: 97 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-25T01:16:47.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:47 smithi169 ceph-mon[32759]: pgmap v416: 97 pgs: 97 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-25T01:16:47.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:47 smithi040 ceph-mon[32518]: pgmap v416: 97 pgs: 97 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-25T01:16:49.503 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:49 smithi031 ceph-mon[28053]: pgmap v417: 97 pgs: 97 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-25T01:16:49.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:49 smithi169 ceph-mon[32759]: pgmap v417: 97 pgs: 97 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-25T01:16:49.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:49 smithi040 ceph-mon[32518]: pgmap v417: 97 pgs: 97 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-25T01:16:51.495 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:51 smithi169 ceph-mon[32759]: pgmap v418: 97 pgs: 97 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T01:16:51.503 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:51 smithi031 ceph-mon[28053]: pgmap v418: 97 pgs: 97 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T01:16:51.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:51 smithi040 ceph-mon[32518]: pgmap v418: 97 pgs: 97 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T01:16:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.244+0000 7ff1fb200640 1 -- 172.21.15.31:0/47939740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc0ffe30 msgr2=0x7ff1fc100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.244+0000 7ff1fb200640 1 --2- 172.21.15.31:0/47939740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc0ffe30 0x7ff1fc100250 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec009900 tx=0x7ff1ec02a3f0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.245+0000 7ff1fb200640 1 -- 172.21.15.31:0/47939740 shutdown_connections 2024-04-25T01:16:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.245+0000 7ff1fb200640 1 --2- 172.21.15.31:0/47939740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1fc101130 0x7ff1fc1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.245+0000 7ff1fb200640 1 --2- 172.21.15.31:0/47939740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1fc100790 0x7ff1fc100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.245+0000 7ff1fb200640 1 --2- 172.21.15.31:0/47939740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc0ffe30 0x7ff1fc100250 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.245+0000 7ff1fb200640 1 -- 172.21.15.31:0/47939740 >> 172.21.15.31:0/47939740 conn(0x7ff1fc0fba30 msgr2=0x7ff1fc0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:16:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.245+0000 7ff1fb200640 1 -- 172.21.15.31:0/47939740 shutdown_connections 2024-04-25T01:16:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.245+0000 7ff1fb200640 1 -- 172.21.15.31:0/47939740 wait complete. 2024-04-25T01:16:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.246+0000 7ff1fb200640 1 Processor -- start 2024-04-25T01:16:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.246+0000 7ff1fb200640 1 -- start start 2024-04-25T01:16:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.247+0000 7ff1fb200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc0ffe30 0x7ff1fc199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.247+0000 7ff1fb200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1fc100790 0x7ff1fc199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.247+0000 7ff1fb200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1fc101130 0x7ff1fc1a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.247+0000 7ff1fb200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc10f470 con 0x7ff1fc100790 2024-04-25T01:16:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.247+0000 7ff1fb200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc10f2f0 con 0x7ff1fc101130 2024-04-25T01:16:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.247+0000 7ff1fb200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc10f5f0 con 0x7ff1fc0ffe30 2024-04-25T01:16:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.247+0000 7ff1f9e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc0ffe30 0x7ff1fc199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.247+0000 7ff1f9e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc0ffe30 0x7ff1fc199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:43590/0 (socket says 172.21.15.31:43590) 2024-04-25T01:16:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.247+0000 7ff1f9e00640 1 -- 172.21.15.31:0/2480374097 learned_addr learned my addr 172.21.15.31:0/2480374097 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:16:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.247+0000 7ff1f9400640 1 --2- 172.21.15.31:0/2480374097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1fc100790 0x7ff1fc199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.248+0000 7ff1fa800640 1 --2- 172.21.15.31:0/2480374097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1fc101130 0x7ff1fc1a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.248+0000 7ff1f9e00640 1 -- 172.21.15.31:0/2480374097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1fc101130 msgr2=0x7ff1fc1a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.248+0000 7ff1f9e00640 1 --2- 172.21.15.31:0/2480374097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1fc101130 0x7ff1fc1a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.248+0000 7ff1f9e00640 1 -- 172.21.15.31:0/2480374097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1fc100790 msgr2=0x7ff1fc199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.248+0000 7ff1f9e00640 1 --2- 172.21.15.31:0/2480374097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1fc100790 0x7ff1fc199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.248+0000 7ff1f9e00640 1 -- 172.21.15.31:0/2480374097 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1fc1a16d0 con 0x7ff1fc0ffe30 2024-04-25T01:16:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.248+0000 7ff1f9400640 1 --2- 172.21.15.31:0/2480374097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1fc100790 0x7ff1fc199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:16:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.248+0000 7ff1f9e00640 1 --2- 172.21.15.31:0/2480374097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc0ffe30 0x7ff1fc199ab0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec031440 tx=0x7ff1ec032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:16:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.249+0000 7ff1f2a00640 1 -- 172.21.15.31:0/2480374097 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec03f030 con 0x7ff1fc0ffe30 2024-04-25T01:16:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.249+0000 7ff1f2a00640 1 -- 172.21.15.31:0/2480374097 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1ec003ce0 con 0x7ff1fc0ffe30 2024-04-25T01:16:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.249+0000 7ff1f2a00640 1 -- 172.21.15.31:0/2480374097 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec035070 con 0x7ff1fc0ffe30 2024-04-25T01:16:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.249+0000 7ff1fb200640 1 -- 172.21.15.31:0/2480374097 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1fc1a1960 con 0x7ff1fc0ffe30 2024-04-25T01:16:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.249+0000 7ff1fb200640 1 -- 172.21.15.31:0/2480374097 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1fc1a1e00 con 0x7ff1fc0ffe30 2024-04-25T01:16:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.253+0000 7ff2041c9740 1 Processor -- start 2024-04-25T01:16:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.254+0000 7ff2041c9740 1 -- start start 2024-04-25T01:16:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.254+0000 7ff2041c9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ec546ce50 0x563ec546d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.254+0000 7ff2041c9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563ec546d840 0x563ec5472340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.254+0000 7ff2041c9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec54729b0 0x563ec5474e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.254+0000 7ff2041c9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563ec53cfab0 con 0x563ec546d840 2024-04-25T01:16:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.254+0000 7ff2041c9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563ec5378640 con 0x563ec54729b0 2024-04-25T01:16:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.254+0000 7ff2041c9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563ec53cba70 con 0x563ec546ce50 2024-04-25T01:16:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.255+0000 7ff1f9400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec54729b0 0x563ec5474e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.255+0000 7ff1f2a00640 1 -- 172.21.15.31:0/2480374097 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff1ec042030 con 0x7ff1fc0ffe30 2024-04-25T01:16:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.255+0000 7ff1f9400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec54729b0 0x563ec5474e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:34908/0 (socket says 172.21.15.31:34908) 2024-04-25T01:16:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.255+0000 7ff1f9400640 1 -- 172.21.15.31:0/529517190 learned_addr learned my addr 172.21.15.31:0/529517190 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:16:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.255+0000 7ff1f9e00640 1 --2- 172.21.15.31:0/529517190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563ec546d840 0x563ec5472340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.255+0000 7ff1fa800640 1 --2- 172.21.15.31:0/529517190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ec546ce50 0x563ec546d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.255+0000 7ff1f9400640 1 -- 172.21.15.31:0/529517190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ec546ce50 msgr2=0x563ec546d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.255+0000 7ff1f9400640 1 --2- 172.21.15.31:0/529517190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ec546ce50 0x563ec546d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.255+0000 7ff1f9400640 1 -- 172.21.15.31:0/529517190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563ec546d840 msgr2=0x563ec5472340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.255+0000 7ff1f9400640 1 --2- 172.21.15.31:0/529517190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563ec546d840 0x563ec5472340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.255+0000 7ff1f9400640 1 -- 172.21.15.31:0/529517190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ec50e4210 con 0x563ec54729b0 2024-04-25T01:16:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.255+0000 7ff1fa800640 1 --2- 172.21.15.31:0/529517190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ec546ce50 0x563ec546d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:16:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.256+0000 7ff1f9400640 1 --2- 172.21.15.31:0/529517190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec54729b0 0x563ec5474e40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009cf0 tx=0x7ff1e80117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=764179f73058a997 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:16:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.257+0000 7ff1e7400640 1 -- 172.21.15.31:0/529517190 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8012450 con 0x563ec54729b0 2024-04-25T01:16:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.257+0000 7ff1e7400640 1 -- 172.21.15.31:0/529517190 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1e8012ad0 con 0x563ec54729b0 2024-04-25T01:16:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.257+0000 7ff1e7400640 1 -- 172.21.15.31:0/529517190 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e800fc30 con 0x563ec54729b0 2024-04-25T01:16:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.257+0000 7ff1f2a00640 1 --2- 172.21.15.31:0/2480374097 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff1e00a2c80 0x7ff1e00a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.257+0000 7ff1f2a00640 1 -- 172.21.15.31:0/2480374097 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7ff1ec0ef550 con 0x7ff1fc0ffe30 2024-04-25T01:16:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.258+0000 7ff2041c9740 1 -- 172.21.15.31:0/529517190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec54729b0 msgr2=0x563ec5474e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.258+0000 7ff2041c9740 1 --2- 172.21.15.31:0/529517190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec54729b0 0x563ec5474e40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009cf0 tx=0x7ff1e80117b0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.258+0000 7ff1fa800640 1 --2- 172.21.15.31:0/2480374097 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff1e00a2c80 0x7ff1e00a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.258+0000 7ff2041c9740 1 -- 172.21.15.31:0/529517190 shutdown_connections 2024-04-25T01:16:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.258+0000 7ff2041c9740 1 --2- 172.21.15.31:0/529517190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec54729b0 0x563ec5474e40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.258+0000 7ff2041c9740 1 --2- 172.21.15.31:0/529517190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563ec546d840 0x563ec5472340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.258+0000 7ff2041c9740 1 --2- 172.21.15.31:0/529517190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ec546ce50 0x563ec546d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.258+0000 7ff2041c9740 1 -- 172.21.15.31:0/529517190 >> 172.21.15.31:0/529517190 conn(0x563ec54684c0 msgr2=0x563ec54688c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:16:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.259+0000 7ff1fa800640 1 --2- 172.21.15.31:0/2480374097 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff1e00a2c80 0x7ff1e00a5140 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x563ec546e0a0 tx=0x7ff1f4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:16:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.260+0000 7ff2041c9740 1 -- 172.21.15.31:0/529517190 shutdown_connections 2024-04-25T01:16:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.260+0000 7ff2041c9740 1 -- 172.21.15.31:0/529517190 wait complete. 2024-04-25T01:16:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.260+0000 7ff2041c9740 1 -- start start 2024-04-25T01:16:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.261+0000 7ff2041c9740 1 Processor -- start 2024-04-25T01:16:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.261+0000 7ff2041c9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563ec546ce50 0x563ec5477090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.261+0000 7ff2041c9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ec546d840 0x563ec54775d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.261+0000 7ff2041c9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec54729b0 0x563ec547e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.261+0000 7ff2041c9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563ec5475960 con 0x563ec546ce50 2024-04-25T01:16:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.261+0000 7ff2041c9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563ec538b990 con 0x563ec54729b0 2024-04-25T01:16:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.261+0000 7ff2041c9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563ec5475ae0 con 0x563ec546d840 2024-04-25T01:16:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.261+0000 7ff2041c9740 10 client.0 _ll_register_callbacks cb 0x563ec538d950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:16:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.262+0000 7ff1f9e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563ec546ce50 0x563ec5477090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.262+0000 7ff1f9e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563ec546ce50 0x563ec5477090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54928/0 (socket says 172.21.15.31:54928) 2024-04-25T01:16:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.262+0000 7ff1f9e00640 1 -- 172.21.15.31:0/1339340017 learned_addr learned my addr 172.21.15.31:0/1339340017 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:16:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.262+0000 7ff1f9400640 1 --2- 172.21.15.31:0/1339340017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ec546d840 0x563ec54775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.262+0000 7ff1fa800640 1 --2- 172.21.15.31:0/1339340017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec54729b0 0x563ec547e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.262+0000 7ff1f9e00640 1 -- 172.21.15.31:0/1339340017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ec546d840 msgr2=0x563ec54775d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.262+0000 7ff1f9e00640 1 --2- 172.21.15.31:0/1339340017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ec546d840 0x563ec54775d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.262+0000 7ff1f9e00640 1 -- 172.21.15.31:0/1339340017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec54729b0 msgr2=0x563ec547e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.262+0000 7ff1f9e00640 1 --2- 172.21.15.31:0/1339340017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec54729b0 0x563ec547e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.262+0000 7ff1f9e00640 1 -- 172.21.15.31:0/1339340017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ec51b0620 con 0x563ec546ce50 2024-04-25T01:16:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.262+0000 7ff1fa800640 1 --2- 172.21.15.31:0/1339340017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec54729b0 0x563ec547e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:16:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.263+0000 7ff1f9e00640 1 --2- 172.21.15.31:0/1339340017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563ec546ce50 0x563ec5477090 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec000c00 tx=0x7ff1ec0b8100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b438e13537405b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:16:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.263+0000 7ff1dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T01:16:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.263+0000 7ff1dbe00640 1 -- 172.21.15.31:0/1339340017 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec0bc4a0 con 0x563ec546ce50 2024-04-25T01:16:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.263+0000 7ff1dbe00640 1 -- 172.21.15.31:0/1339340017 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7ff1ec0b8410 con 0x563ec546ce50 2024-04-25T01:16:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.264+0000 7ff2041c9740 1 -- 172.21.15.31:0/1339340017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1e8012ad0 con 0x563ec546ce50 2024-04-25T01:16:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.264+0000 7ff1dbe00640 1 -- 172.21.15.31:0/1339340017 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff1ec0bbda0 con 0x563ec546ce50 2024-04-25T01:16:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.264+0000 7ff2041c9740 10 client.15165 fetch_fsmap learned FSMap version 24 2024-04-25T01:16:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.264+0000 7ff2041c9740 1 -- 172.21.15.31:0/1339340017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563ec5481560 con 0x563ec546ce50 2024-04-25T01:16:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.264+0000 7ff1dbe00640 1 -- 172.21.15.31:0/1339340017 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1e8012ad0 con 0x563ec546ce50 2024-04-25T01:16:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.265+0000 7ff1dbe00640 1 -- 172.21.15.31:0/1339340017 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ff1ec0ef0d0 con 0x563ec546ce50 2024-04-25T01:16:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.265+0000 7ff2041c9740 10 client.15165 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T01:16:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.265+0000 7ff2041c9740 10 client.15165 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:16:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.265+0000 7ff2041c9740 20 client.15165 populate_metadata read hostname 'smithi031' 2024-04-25T01:16:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.265+0000 7ff2041c9740 1 --2- 172.21.15.31:0/1339340017 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x563ec5482d60 0x563ec54851f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.265+0000 7ff2041c9740 4 client.15165 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:16:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.265+0000 7ff2041c9740 1 -- 172.21.15.31:0/1339340017 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ec53dcff0 con 0x563ec5482d60 2024-04-25T01:16:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.265+0000 7ff1f9400640 1 --2- 172.21.15.31:0/1339340017 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x563ec5482d60 0x563ec54851f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.266+0000 7ff1f9400640 1 --2- 172.21.15.31:0/1339340017 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x563ec5482d60 0x563ec54851f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=468529b086d4ad74 server_cookie=a45df7bd18f2096c in_seq=0 out_seq=0 2024-04-25T01:16:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.266+0000 7ff1dbe00640 10 client.15165 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:16:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.267+0000 7ff1dbe00640 1 -- 172.21.15.31:0/1339340017 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563ec53dcff0 con 0x563ec5482d60 2024-04-25T01:16:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.267+0000 7ff1dbe00640 10 client.15165 handle_command_reply: tid=0 2024-04-25T01:16:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.267+0000 7ff2041c9740 1 client.15165 shutdown 2024-04-25T01:16:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.267+0000 7ff2041c9740 1 -- 172.21.15.31:0/1339340017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563ec546ce50 msgr2=0x563ec5477090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.267+0000 7ff2041c9740 1 --2- 172.21.15.31:0/1339340017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563ec546ce50 0x563ec5477090 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec000c00 tx=0x7ff1ec0b8100 comp rx=0 tx=0).stop 2024-04-25T01:16:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.267+0000 7ff2041c9740 1 -- 172.21.15.31:0/1339340017 shutdown_connections 2024-04-25T01:16:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.267+0000 7ff2041c9740 1 --2- 172.21.15.31:0/1339340017 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x563ec5482d60 0x563ec54851f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.267+0000 7ff2041c9740 1 --2- 172.21.15.31:0/1339340017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec54729b0 0x563ec547e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.267+0000 7ff2041c9740 1 --2- 172.21.15.31:0/1339340017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ec546d840 0x563ec54775d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.267+0000 7ff1dbe00640 0 client.15165 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:16:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.267+0000 7ff2041c9740 1 --2- 172.21.15.31:0/1339340017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563ec546ce50 0x563ec5477090 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.267+0000 7ff2041c9740 1 -- 172.21.15.31:0/1339340017 >> 172.21.15.31:0/1339340017 conn(0x563ec54684c0 msgr2=0x563ec5467620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:16:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.268+0000 7ff2041c9740 1 -- 172.21.15.31:0/1339340017 shutdown_connections 2024-04-25T01:16:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.268+0000 7ff2041c9740 1 -- 172.21.15.31:0/1339340017 wait complete. 2024-04-25T01:16:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.268+0000 7ff2041c9740 20 client.15165 trim_cache size 0 max 16384 2024-04-25T01:16:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.299+0000 7ff2041c9740 1 Processor -- start 2024-04-25T01:16:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.299+0000 7ff2041c9740 1 -- start start 2024-04-25T01:16:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.299+0000 7ff2041c9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec546ce50 0x563ec547f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.299+0000 7ff2041c9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ec546d840 0x563ec547e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.299+0000 7ff2041c9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563ec54729b0 0x563ec546b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.299+0000 7ff2041c9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc1ad3f0 con 0x563ec54729b0 2024-04-25T01:16:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.299+0000 7ff2041c9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc1aacc0 con 0x563ec546ce50 2024-04-25T01:16:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.299+0000 7ff2041c9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc1a8b10 con 0x563ec546d840 2024-04-25T01:16:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.300+0000 7ff1f9e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec546ce50 0x563ec547f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.300+0000 7ff1f9e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec546ce50 0x563ec547f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:34926/0 (socket says 172.21.15.31:34926) 2024-04-25T01:16:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.300+0000 7ff1f9e00640 1 -- 172.21.15.31:0/3832719355 learned_addr learned my addr 172.21.15.31:0/3832719355 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:16:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.300+0000 7ff1fa800640 1 --2- 172.21.15.31:0/3832719355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563ec54729b0 0x563ec546b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.300+0000 7ff1f9400640 1 --2- 172.21.15.31:0/3832719355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ec546d840 0x563ec547e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.300+0000 7ff1f9e00640 1 -- 172.21.15.31:0/3832719355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ec546d840 msgr2=0x563ec547e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.300+0000 7ff1f9e00640 1 --2- 172.21.15.31:0/3832719355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ec546d840 0x563ec547e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.300+0000 7ff1f9e00640 1 -- 172.21.15.31:0/3832719355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563ec54729b0 msgr2=0x563ec546b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.300+0000 7ff1f9e00640 1 --2- 172.21.15.31:0/3832719355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563ec54729b0 0x563ec546b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.300+0000 7ff1f9e00640 1 -- 172.21.15.31:0/3832719355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ec54682f0 con 0x563ec546ce50 2024-04-25T01:16:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.301+0000 7ff1f9e00640 1 --2- 172.21.15.31:0/3832719355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec546ce50 0x563ec547f3b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0bbaa0 tx=0x7ff1ec0ba2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df3a8d81fd59e30b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:16:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.301+0000 7ff1e6a00640 1 -- 172.21.15.31:0/3832719355 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec0c0150 con 0x563ec546ce50 2024-04-25T01:16:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.301+0000 7ff1e6a00640 1 -- 172.21.15.31:0/3832719355 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1ec0c4a10 con 0x563ec546ce50 2024-04-25T01:16:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.301+0000 7ff1e6a00640 1 -- 172.21.15.31:0/3832719355 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec0bb240 con 0x563ec546ce50 2024-04-25T01:16:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.302+0000 7ff2041c9740 1 -- 172.21.15.31:0/3832719355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec546ce50 msgr2=0x563ec547f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.302+0000 7ff2041c9740 1 --2- 172.21.15.31:0/3832719355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec546ce50 0x563ec547f3b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0bbaa0 tx=0x7ff1ec0ba2b0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.302+0000 7ff2041c9740 1 -- 172.21.15.31:0/3832719355 shutdown_connections 2024-04-25T01:16:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.302+0000 7ff2041c9740 1 --2- 172.21.15.31:0/3832719355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563ec54729b0 0x563ec546b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.302+0000 7ff2041c9740 1 --2- 172.21.15.31:0/3832719355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ec546d840 0x563ec547e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.302+0000 7ff2041c9740 1 --2- 172.21.15.31:0/3832719355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec546ce50 0x563ec547f3b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.302+0000 7ff2041c9740 1 -- 172.21.15.31:0/3832719355 >> 172.21.15.31:0/3832719355 conn(0x563ec54684c0 msgr2=0x563ec547eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:16:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.302+0000 7ff2041c9740 1 -- 172.21.15.31:0/3832719355 shutdown_connections 2024-04-25T01:16:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.302+0000 7ff2041c9740 1 -- 172.21.15.31:0/3832719355 wait complete. 2024-04-25T01:16:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.303+0000 7ff2041c9740 1 -- start start 2024-04-25T01:16:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.303+0000 7ff2041c9740 1 Processor -- start 2024-04-25T01:16:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.303+0000 7ff2041c9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ec546ce50 0x563ec547ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.303+0000 7ff2041c9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563ec546d840 0x563ec547bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.304+0000 7ff2041c9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec54729b0 0x563ec547cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.304+0000 7ff2041c9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc1a9180 con 0x563ec546d840 2024-04-25T01:16:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.304+0000 7ff2041c9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc1aa5f0 con 0x563ec54729b0 2024-04-25T01:16:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.304+0000 7ff2041c9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc1a9850 con 0x563ec546ce50 2024-04-25T01:16:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.304+0000 7ff2041c9740 10 client.0 _ll_register_callbacks cb 0x7ff1ec003eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:16:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.304+0000 7ff1f9e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ec546ce50 0x563ec547ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.304+0000 7ff1f9e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ec546ce50 0x563ec547ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:43622/0 (socket says 172.21.15.31:43622) 2024-04-25T01:16:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.304+0000 7ff1f9e00640 1 -- 172.21.15.31:0/1458273422 learned_addr learned my addr 172.21.15.31:0/1458273422 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:16:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.304+0000 7ff1fa800640 1 --2- 172.21.15.31:0/1458273422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec54729b0 0x563ec547cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.304+0000 7ff1f9400640 1 --2- 172.21.15.31:0/1458273422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563ec546d840 0x563ec547bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.305+0000 7ff1fa800640 1 -- 172.21.15.31:0/1458273422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ec546ce50 msgr2=0x563ec547ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.305+0000 7ff1fa800640 1 --2- 172.21.15.31:0/1458273422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ec546ce50 0x563ec547ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.305+0000 7ff1fa800640 1 -- 172.21.15.31:0/1458273422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563ec546d840 msgr2=0x563ec547bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:16:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.305+0000 7ff1fa800640 1 --2- 172.21.15.31:0/1458273422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563ec546d840 0x563ec547bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.305+0000 7ff1fa800640 1 -- 172.21.15.31:0/1458273422 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ec547d520 con 0x563ec54729b0 2024-04-25T01:16:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.305+0000 7ff1fa800640 1 --2- 172.21.15.31:0/1458273422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec54729b0 0x563ec547cf30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x563ec547ebd0 tx=0x7ff1f4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d66dd54c2fe93a0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:16:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.305+0000 7ff1dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T01:16:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.305+0000 7ff1dbe00640 1 -- 172.21.15.31:0/1458273422 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f402c060 con 0x563ec54729b0 2024-04-25T01:16:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.306+0000 7ff1dbe00640 1 -- 172.21.15.31:0/1458273422 <== mon.2 v2:172.21.15.40:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7ff1f40114a0 con 0x563ec54729b0 2024-04-25T01:16:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.306+0000 7ff2041c9740 1 -- 172.21.15.31:0/1458273422 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1ec0c4a10 con 0x563ec54729b0 2024-04-25T01:16:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.306+0000 7ff1dbe00640 1 -- 172.21.15.31:0/1458273422 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff1f40116a0 con 0x563ec54729b0 2024-04-25T01:16:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.306+0000 7ff2041c9740 10 client.25292 fetch_fsmap learned FSMap version 24 2024-04-25T01:16:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.306+0000 7ff2041c9740 1 -- 172.21.15.31:0/1458273422 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563ec547e8a0 con 0x563ec54729b0 2024-04-25T01:16:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.306+0000 7ff1dbe00640 1 -- 172.21.15.31:0/1458273422 <== mon.2 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1f4011d20 con 0x563ec54729b0 2024-04-25T01:16:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.307+0000 7ff1dbe00640 1 -- 172.21.15.31:0/1458273422 <== mon.2 v2:172.21.15.40:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ff1f4011840 con 0x563ec54729b0 2024-04-25T01:16:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.307+0000 7ff2041c9740 10 client.25292 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T01:16:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.307+0000 7ff2041c9740 10 client.25292 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:16:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.307+0000 7ff2041c9740 20 client.25292 populate_metadata read hostname 'smithi031' 2024-04-25T01:16:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.307+0000 7ff2041c9740 1 --2- 172.21.15.31:0/1458273422 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x563ec54f21e0 0x563ec54f4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:16:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.307+0000 7ff2041c9740 4 client.25292 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:16:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.308+0000 7ff2041c9740 1 -- 172.21.15.31:0/1458273422 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7ff1ec0c4a10 con 0x563ec54f21e0 2024-04-25T01:16:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.308+0000 7ff1f9e00640 1 --2- 172.21.15.31:0/1458273422 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x563ec54f21e0 0x563ec54f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:16:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.308+0000 7ff1f9e00640 1 --2- 172.21.15.31:0/1458273422 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x563ec54f21e0 0x563ec54f4600 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=884e4ab476de4d09 server_cookie=2d4604b66aa771fe in_seq=0 out_seq=0 2024-04-25T01:16:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.308+0000 7ff1dbe00640 10 client.25292 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:16:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.313+0000 7ff1dbe00640 1 -- 172.21.15.31:0/1458273422 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7ff1ec0c4a10 con 0x563ec54f21e0 2024-04-25T01:16:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.313+0000 7ff1dbe00640 10 client.25292 handle_command_reply: tid=0 2024-04-25T01:16:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.313+0000 7ff2041c9740 1 client.25292 shutdown 2024-04-25T01:16:52.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.314+0000 7ff2041c9740 1 -- 172.21.15.31:0/1458273422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec54729b0 msgr2=0x563ec547cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:52.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.314+0000 7ff2041c9740 1 --2- 172.21.15.31:0/1458273422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec54729b0 0x563ec547cf30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x563ec547ebd0 tx=0x7ff1f4004a40 comp rx=0 tx=0).stop 2024-04-25T01:16:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.314+0000 7ff2041c9740 1 -- 172.21.15.31:0/1458273422 shutdown_connections 2024-04-25T01:16:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.314+0000 7ff2041c9740 1 --2- 172.21.15.31:0/1458273422 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x563ec54f21e0 0x563ec54f4600 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.314+0000 7ff2041c9740 1 --2- 172.21.15.31:0/1458273422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563ec54729b0 0x563ec547cf30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.314+0000 7ff2041c9740 1 --2- 172.21.15.31:0/1458273422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563ec546d840 0x563ec547bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.314+0000 7ff2041c9740 1 --2- 172.21.15.31:0/1458273422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ec546ce50 0x563ec547ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.314+0000 7ff2041c9740 1 -- 172.21.15.31:0/1458273422 >> 172.21.15.31:0/1458273422 conn(0x563ec54684c0 msgr2=0x563ec546b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:16:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.314+0000 7ff1dbe00640 0 client.25292 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:16:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.314+0000 7ff2041c9740 1 -- 172.21.15.31:0/1458273422 shutdown_connections 2024-04-25T01:16:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.314+0000 7ff2041c9740 1 -- 172.21.15.31:0/1458273422 wait complete. 2024-04-25T01:16:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.314+0000 7ff2041c9740 20 client.25292 trim_cache size 0 max 16384 2024-04-25T01:16:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:16:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:16:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:16:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:16:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.315+0000 7ff1fb200640 1 -- 172.21.15.31:0/2480374097 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff1e00a2c80 msgr2=0x7ff1e00a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.315+0000 7ff1fb200640 1 --2- 172.21.15.31:0/2480374097 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff1e00a2c80 0x7ff1e00a5140 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x563ec546e0a0 tx=0x7ff1f4009500 comp rx=0 tx=0).stop 2024-04-25T01:16:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.315+0000 7ff1fb200640 1 -- 172.21.15.31:0/2480374097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc0ffe30 msgr2=0x7ff1fc199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:16:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.315+0000 7ff1fb200640 1 --2- 172.21.15.31:0/2480374097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc0ffe30 0x7ff1fc199ab0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec031440 tx=0x7ff1ec032ce0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.315+0000 7ff1fb200640 1 -- 172.21.15.31:0/2480374097 shutdown_connections 2024-04-25T01:16:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.315+0000 7ff1fb200640 1 --2- 172.21.15.31:0/2480374097 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff1e00a2c80 0x7ff1e00a5140 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.315+0000 7ff1fb200640 1 --2- 172.21.15.31:0/2480374097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1fc101130 0x7ff1fc1a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.315+0000 7ff1fb200640 1 --2- 172.21.15.31:0/2480374097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1fc100790 0x7ff1fc199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.315+0000 7ff1fb200640 1 --2- 172.21.15.31:0/2480374097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc0ffe30 0x7ff1fc199ab0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:16:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.315+0000 7ff1fb200640 1 -- 172.21.15.31:0/2480374097 >> 172.21.15.31:0/2480374097 conn(0x7ff1fc0fba30 msgr2=0x7ff1fc0fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:16:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.315+0000 7ff1fb200640 1 -- 172.21.15.31:0/2480374097 shutdown_connections 2024-04-25T01:16:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:16:52.316+0000 7ff1fb200640 1 -- 172.21.15.31:0/2480374097 wait complete. 2024-04-25T01:16:53.503 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:53 smithi031 ceph-mon[28053]: pgmap v419: 97 pgs: 97 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-25T01:16:53.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:53 smithi169 ceph-mon[32759]: pgmap v419: 97 pgs: 97 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-25T01:16:53.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:53 smithi040 ceph-mon[32518]: pgmap v419: 97 pgs: 97 active+clean; 661 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-25T01:16:55.503 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:55 smithi031 ceph-mon[28053]: pgmap v420: 97 pgs: 97 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-25T01:16:55.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:55 smithi169 ceph-mon[32759]: pgmap v420: 97 pgs: 97 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-25T01:16:55.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:55 smithi040 ceph-mon[32518]: pgmap v420: 97 pgs: 97 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-25T01:16:57.503 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:57 smithi031 ceph-mon[28053]: pgmap v421: 97 pgs: 97 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T01:16:57.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:57 smithi169 ceph-mon[32759]: pgmap v421: 97 pgs: 97 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T01:16:57.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:57 smithi040 ceph-mon[32518]: pgmap v421: 97 pgs: 97 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T01:16:59.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:16:59 smithi169 ceph-mon[32759]: pgmap v422: 97 pgs: 97 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:16:59.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:16:59 smithi040 ceph-mon[32518]: pgmap v422: 97 pgs: 97 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:16:59.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:16:59 smithi031 ceph-mon[28053]: pgmap v422: 97 pgs: 97 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:17:01.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:01 smithi169 ceph-mon[32759]: pgmap v423: 97 pgs: 97 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T01:17:01.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:01 smithi040 ceph-mon[32518]: pgmap v423: 97 pgs: 97 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T01:17:01.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:01 smithi031 ceph-mon[28053]: pgmap v423: 97 pgs: 97 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T01:17:03.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:03 smithi169 ceph-mon[32759]: pgmap v424: 97 pgs: 97 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:17:03.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:03 smithi040 ceph-mon[32518]: pgmap v424: 97 pgs: 97 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:17:03.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:03 smithi031 ceph-mon[28053]: pgmap v424: 97 pgs: 97 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:17:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.476+0000 7f6b49a00640 1 -- 172.21.15.31:0/4040525076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b44101600 msgr2=0x7f6b44101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.476+0000 7f6b49a00640 1 --2- 172.21.15.31:0/4040525076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b44101600 0x7f6b44101a20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6b34009900 tx=0x7f6b3402a400 comp rx=0 tx=0).stop 2024-04-25T01:17:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.476+0000 7f6b49a00640 1 -- 172.21.15.31:0/4040525076 shutdown_connections 2024-04-25T01:17:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.476+0000 7f6b49a00640 1 --2- 172.21.15.31:0/4040525076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b441031b0 0x7f6b44109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.476+0000 7f6b49a00640 1 --2- 172.21.15.31:0/4040525076 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b441027f0 0x7f6b44102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.476+0000 7f6b49a00640 1 --2- 172.21.15.31:0/4040525076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b44101600 0x7f6b44101a20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.476+0000 7f6b49a00640 1 -- 172.21.15.31:0/4040525076 >> 172.21.15.31:0/4040525076 conn(0x7f6b440fcb70 msgr2=0x7f6b440fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.476+0000 7f6b49a00640 1 -- 172.21.15.31:0/4040525076 shutdown_connections 2024-04-25T01:17:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.476+0000 7f6b49a00640 1 -- 172.21.15.31:0/4040525076 wait complete. 2024-04-25T01:17:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.477+0000 7f6b49a00640 1 Processor -- start 2024-04-25T01:17:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.478+0000 7f6b49a00640 1 -- start start 2024-04-25T01:17:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.478+0000 7f6b49a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b44101600 0x7f6b44199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.478+0000 7f6b49a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b441027f0 0x7f6b4419a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.478+0000 7f6b49a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b441031b0 0x7f6b441a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.478+0000 7f6b49a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4410d2c0 con 0x7f6b441031b0 2024-04-25T01:17:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.478+0000 7f6b49a00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4410d140 con 0x7f6b44101600 2024-04-25T01:17:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.478+0000 7f6b49a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4410d440 con 0x7f6b441027f0 2024-04-25T01:17:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.478+0000 7f6b49000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b441031b0 0x7f6b441a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.479+0000 7f6b49000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b441031b0 0x7f6b441a1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33038/0 (socket says 172.21.15.31:33038) 2024-04-25T01:17:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.479+0000 7f6b49000640 1 -- 172.21.15.31:0/1714563754 learned_addr learned my addr 172.21.15.31:0/1714563754 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:17:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.479+0000 7f6b43e00640 1 --2- 172.21.15.31:0/1714563754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b44101600 0x7f6b44199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.479+0000 7f6b43400640 1 --2- 172.21.15.31:0/1714563754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b441027f0 0x7f6b4419a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.479+0000 7f6b49000640 1 -- 172.21.15.31:0/1714563754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b441027f0 msgr2=0x7f6b4419a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.479+0000 7f6b49000640 1 --2- 172.21.15.31:0/1714563754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b441027f0 0x7f6b4419a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.479+0000 7f6b49000640 1 -- 172.21.15.31:0/1714563754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b44101600 msgr2=0x7f6b44199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.479+0000 7f6b49000640 1 --2- 172.21.15.31:0/1714563754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b44101600 0x7f6b44199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.479+0000 7f6b49000640 1 -- 172.21.15.31:0/1714563754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b441a17f0 con 0x7f6b441031b0 2024-04-25T01:17:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.480+0000 7f6b43400640 1 --2- 172.21.15.31:0/1714563754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b441027f0 0x7f6b4419a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:17:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.480+0000 7f6b43e00640 1 --2- 172.21.15.31:0/1714563754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b44101600 0x7f6b44199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:17:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.480+0000 7f6b49000640 1 --2- 172.21.15.31:0/1714563754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b441031b0 0x7f6b441a1180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00efd0 tx=0x7f6b3c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.480+0000 7f6b40c00640 1 -- 172.21.15.31:0/1714563754 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c003b70 con 0x7f6b441031b0 2024-04-25T01:17:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.480+0000 7f6b40c00640 1 -- 172.21.15.31:0/1714563754 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b3c003d10 con 0x7f6b441031b0 2024-04-25T01:17:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.480+0000 7f6b40c00640 1 -- 172.21.15.31:0/1714563754 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c0109f0 con 0x7f6b441031b0 2024-04-25T01:17:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.481+0000 7f6b49a00640 1 -- 172.21.15.31:0/1714563754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b441a1ae0 con 0x7f6b441031b0 2024-04-25T01:17:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.481+0000 7f6b49a00640 1 -- 172.21.15.31:0/1714563754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b441a1fe0 con 0x7f6b441031b0 2024-04-25T01:17:05.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.486+0000 7f6b40c00640 1 -- 172.21.15.31:0/1714563754 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6b3c004020 con 0x7f6b441031b0 2024-04-25T01:17:05.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.486+0000 7f6b4dd70740 1 Processor -- start 2024-04-25T01:17:05.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.486+0000 7f6b4dd70740 1 -- start start 2024-04-25T01:17:05.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.487+0000 7f6b4dd70740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560d7ccfde50 0x560d7ccfe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.487+0000 7f6b4dd70740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d7ccfe840 0x560d7cd03340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.487+0000 7f6b4dd70740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560d7cd039b0 0x560d7cd05e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.487+0000 7f6b4dd70740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560d7cc60ab0 con 0x560d7cd039b0 2024-04-25T01:17:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.487+0000 7f6b4dd70740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560d7cc09640 con 0x560d7ccfde50 2024-04-25T01:17:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.487+0000 7f6b4dd70740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560d7cc5ca70 con 0x560d7ccfe840 2024-04-25T01:17:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.487+0000 7f6b43e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d7ccfe840 0x560d7cd03340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.487+0000 7f6b43e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d7ccfe840 0x560d7cd03340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:38416/0 (socket says 172.21.15.31:38416) 2024-04-25T01:17:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.487+0000 7f6b43e00640 1 -- 172.21.15.31:0/1999948351 learned_addr learned my addr 172.21.15.31:0/1999948351 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:17:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.488+0000 7f6b43400640 1 --2- 172.21.15.31:0/1999948351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560d7cd039b0 0x560d7cd05e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.488+0000 7f6b43e00640 1 -- 172.21.15.31:0/1999948351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560d7ccfde50 msgr2=0x560d7ccfe270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.488+0000 7f6b43400640 1 --2- 172.21.15.31:0/1999948351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560d7ccfde50 0x560d7ccfe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.488+0000 7f6b43e00640 1 --2- 172.21.15.31:0/1999948351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560d7ccfde50 0x560d7ccfe270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.488+0000 7f6b43e00640 1 -- 172.21.15.31:0/1999948351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560d7cd039b0 msgr2=0x560d7cd05e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.488+0000 7f6b43e00640 1 --2- 172.21.15.31:0/1999948351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560d7cd039b0 0x560d7cd05e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.488+0000 7f6b43e00640 1 -- 172.21.15.31:0/1999948351 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d7c975210 con 0x560d7ccfe840 2024-04-25T01:17:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.488+0000 7f6b43400640 1 --2- 172.21.15.31:0/1999948351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560d7cd039b0 0x560d7cd05e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:17:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.488+0000 7f6b43e00640 1 --2- 172.21.15.31:0/1999948351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d7ccfe840 0x560d7cd03340 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6b34009900 tx=0x7f6b34005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=283c51fbff64536a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.488+0000 7f6b40c00640 1 --2- 172.21.15.31:0/1714563754 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6b280a2c00 0x7f6b280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.489+0000 7f6b40c00640 1 -- 172.21.15.31:0/1714563754 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f6b3c0cf340 con 0x7f6b441031b0 2024-04-25T01:17:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.489+0000 7f6b39600640 1 -- 172.21.15.31:0/1999948351 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b34035040 con 0x560d7ccfe840 2024-04-25T01:17:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.489+0000 7f6b39600640 1 -- 172.21.15.31:0/1999948351 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b34004740 con 0x560d7ccfe840 2024-04-25T01:17:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.489+0000 7f6b39600640 1 -- 172.21.15.31:0/1999948351 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b34031a30 con 0x560d7ccfe840 2024-04-25T01:17:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.490+0000 7f6b4dd70740 1 -- 172.21.15.31:0/1999948351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d7ccfe840 msgr2=0x560d7cd03340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.490+0000 7f6b4dd70740 1 --2- 172.21.15.31:0/1999948351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d7ccfe840 0x560d7cd03340 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6b34009900 tx=0x7f6b34005dd0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.490+0000 7f6b4dd70740 1 -- 172.21.15.31:0/1999948351 shutdown_connections 2024-04-25T01:17:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.490+0000 7f6b4dd70740 1 --2- 172.21.15.31:0/1999948351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560d7cd039b0 0x560d7cd05e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.490+0000 7f6b4dd70740 1 --2- 172.21.15.31:0/1999948351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d7ccfe840 0x560d7cd03340 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.490+0000 7f6b4dd70740 1 --2- 172.21.15.31:0/1999948351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560d7ccfde50 0x560d7ccfe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.490+0000 7f6b4dd70740 1 -- 172.21.15.31:0/1999948351 >> 172.21.15.31:0/1999948351 conn(0x560d7ccf94c0 msgr2=0x560d7ccf98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.491+0000 7f6b43400640 1 --2- 172.21.15.31:0/1714563754 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6b280a2c00 0x7f6b280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.491+0000 7f6b4dd70740 1 -- 172.21.15.31:0/1999948351 shutdown_connections 2024-04-25T01:17:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.491+0000 7f6b4dd70740 1 -- 172.21.15.31:0/1999948351 wait complete. 2024-04-25T01:17:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.491+0000 7f6b4dd70740 1 -- start start 2024-04-25T01:17:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.492+0000 7f6b43400640 1 --2- 172.21.15.31:0/1714563754 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6b280a2c00 0x7f6b280a50c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6b30005d00 tx=0x7f6b300093b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.492+0000 7f6b4dd70740 1 Processor -- start 2024-04-25T01:17:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.492+0000 7f6b4dd70740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560d7ccfde50 0x560d7cd08060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.492+0000 7f6b4dd70740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d7ccfe840 0x560d7cd085a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.492+0000 7f6b4dd70740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560d7cd039b0 0x560d7cd0f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.492+0000 7f6b4dd70740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560d7cd06900 con 0x560d7ccfde50 2024-04-25T01:17:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.492+0000 7f6b4dd70740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560d7cc1c990 con 0x560d7cd039b0 2024-04-25T01:17:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.492+0000 7f6b4dd70740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560d7cd06a80 con 0x560d7ccfe840 2024-04-25T01:17:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.493+0000 7f6b4dd70740 10 client.0 _ll_register_callbacks cb 0x560d7cc1e950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:17:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.493+0000 7f6b49000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560d7cd039b0 0x560d7cd0f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.493+0000 7f6b43e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560d7ccfde50 0x560d7cd08060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.493+0000 7f6b43e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560d7ccfde50 0x560d7cd08060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33058/0 (socket says 172.21.15.31:33058) 2024-04-25T01:17:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.493+0000 7f6b43e00640 1 -- 172.21.15.31:0/3858385097 learned_addr learned my addr 172.21.15.31:0/3858385097 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:17:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.493+0000 7f6b43400640 1 --2- 172.21.15.31:0/3858385097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d7ccfe840 0x560d7cd085a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.494+0000 7f6b49000640 1 -- 172.21.15.31:0/3858385097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d7ccfe840 msgr2=0x560d7cd085a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.494+0000 7f6b49000640 1 --2- 172.21.15.31:0/3858385097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d7ccfe840 0x560d7cd085a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.494+0000 7f6b49000640 1 -- 172.21.15.31:0/3858385097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560d7ccfde50 msgr2=0x560d7cd08060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.494+0000 7f6b49000640 1 --2- 172.21.15.31:0/3858385097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560d7ccfde50 0x560d7cd08060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.494+0000 7f6b49000640 1 -- 172.21.15.31:0/3858385097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d7ca41620 con 0x560d7cd039b0 2024-04-25T01:17:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.494+0000 7f6b43e00640 1 --2- 172.21.15.31:0/3858385097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560d7ccfde50 0x560d7cd08060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:17:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.494+0000 7f6b49000640 1 --2- 172.21.15.31:0/3858385097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560d7cd039b0 0x560d7cd0f5e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c0ccf20 tx=0x7f6b3c09b650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c36c69c6f783fcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.494+0000 7f6b2e000640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T01:17:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.494+0000 7f6b2e000640 1 -- 172.21.15.31:0/3858385097 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c010d90 con 0x560d7cd039b0 2024-04-25T01:17:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.495+0000 7f6b2e000640 1 -- 172.21.15.31:0/3858385097 <== mon.2 v2:172.21.15.40:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f6b3c004b40 con 0x560d7cd039b0 2024-04-25T01:17:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.495+0000 7f6b4dd70740 1 -- 172.21.15.31:0/3858385097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b34004740 con 0x560d7cd039b0 2024-04-25T01:17:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.495+0000 7f6b2e000640 1 -- 172.21.15.31:0/3858385097 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6b3c0a48a0 con 0x560d7cd039b0 2024-04-25T01:17:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.495+0000 7f6b4dd70740 10 client.25301 fetch_fsmap learned FSMap version 24 2024-04-25T01:17:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.495+0000 7f6b4dd70740 1 -- 172.21.15.31:0/3858385097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560d7cd125a0 con 0x560d7cd039b0 2024-04-25T01:17:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.495+0000 7f6b2e000640 1 -- 172.21.15.31:0/3858385097 <== mon.2 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b34004740 con 0x560d7cd039b0 2024-04-25T01:17:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.495+0000 7f6b2e000640 1 -- 172.21.15.31:0/3858385097 <== mon.2 v2:172.21.15.40:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f6b3c0b4020 con 0x560d7cd039b0 2024-04-25T01:17:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.496+0000 7f6b4dd70740 10 client.25301 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T01:17:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.496+0000 7f6b4dd70740 10 client.25301 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:17:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.496+0000 7f6b4dd70740 20 client.25301 populate_metadata read hostname 'smithi031' 2024-04-25T01:17:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.496+0000 7f6b4dd70740 1 --2- 172.21.15.31:0/3858385097 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560d7cd13c80 0x560d7cd16110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.496+0000 7f6b4dd70740 4 client.25301 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:17:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.496+0000 7f6b4dd70740 1 -- 172.21.15.31:0/3858385097 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d7cc6dff0 con 0x560d7cd13c80 2024-04-25T01:17:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.496+0000 7f6b43e00640 1 --2- 172.21.15.31:0/3858385097 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560d7cd13c80 0x560d7cd16110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.497+0000 7f6b43e00640 1 --2- 172.21.15.31:0/3858385097 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560d7cd13c80 0x560d7cd16110 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2daed92fadaec863 server_cookie=fce7444b72d035e0 in_seq=0 out_seq=0 2024-04-25T01:17:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.497+0000 7f6b2e000640 10 client.25301 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:17:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.498+0000 7f6b2e000640 1 -- 172.21.15.31:0/3858385097 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d7cc6dff0 con 0x560d7cd13c80 2024-04-25T01:17:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.498+0000 7f6b2e000640 10 client.25301 handle_command_reply: tid=0 2024-04-25T01:17:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.498+0000 7f6b4dd70740 1 client.25301 shutdown 2024-04-25T01:17:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.498+0000 7f6b4dd70740 1 -- 172.21.15.31:0/3858385097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560d7cd039b0 msgr2=0x560d7cd0f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.498+0000 7f6b4dd70740 1 --2- 172.21.15.31:0/3858385097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560d7cd039b0 0x560d7cd0f5e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c0ccf20 tx=0x7f6b3c09b650 comp rx=0 tx=0).stop 2024-04-25T01:17:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.498+0000 7f6b4dd70740 1 -- 172.21.15.31:0/3858385097 shutdown_connections 2024-04-25T01:17:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.498+0000 7f6b4dd70740 1 --2- 172.21.15.31:0/3858385097 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560d7cd13c80 0x560d7cd16110 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.498+0000 7f6b4dd70740 1 --2- 172.21.15.31:0/3858385097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560d7cd039b0 0x560d7cd0f5e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.498+0000 7f6b4dd70740 1 --2- 172.21.15.31:0/3858385097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d7ccfe840 0x560d7cd085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.498+0000 7f6b4dd70740 1 --2- 172.21.15.31:0/3858385097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560d7ccfde50 0x560d7cd08060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.498+0000 7f6b4dd70740 1 -- 172.21.15.31:0/3858385097 >> 172.21.15.31:0/3858385097 conn(0x560d7ccf94c0 msgr2=0x560d7ccf8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.498+0000 7f6b2e000640 0 client.25301 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:17:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.498+0000 7f6b4dd70740 1 -- 172.21.15.31:0/3858385097 shutdown_connections 2024-04-25T01:17:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.498+0000 7f6b4dd70740 1 -- 172.21.15.31:0/3858385097 wait complete. 2024-04-25T01:17:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.499+0000 7f6b4dd70740 20 client.25301 trim_cache size 0 max 16384 2024-04-25T01:17:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.521+0000 7f6b4dd70740 1 Processor -- start 2024-04-25T01:17:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.521+0000 7f6b4dd70740 1 -- start start 2024-04-25T01:17:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.521+0000 7f6b4dd70740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d7ccfde50 0x560d7cd03060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.521+0000 7f6b4dd70740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560d7ccfe840 0x560d7cd0f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.521+0000 7f6b4dd70740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560d7cd039b0 0x560d7ccffa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.521+0000 7f6b4dd70740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6b24010b70 con 0x560d7cd039b0 2024-04-25T01:17:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.521+0000 7f6b4dd70740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2400e440 con 0x560d7ccfe840 2024-04-25T01:17:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.521+0000 7f6b4dd70740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2400c290 con 0x560d7ccfde50 2024-04-25T01:17:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.522+0000 7f6b43e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d7ccfde50 0x560d7cd03060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.522+0000 7f6b43e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d7ccfde50 0x560d7cd03060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:38444/0 (socket says 172.21.15.31:38444) 2024-04-25T01:17:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.522+0000 7f6b43e00640 1 -- 172.21.15.31:0/748995585 learned_addr learned my addr 172.21.15.31:0/748995585 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:17:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.522+0000 7f6b49000640 1 --2- 172.21.15.31:0/748995585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560d7cd039b0 0x560d7ccffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.522+0000 7f6b43400640 1 --2- 172.21.15.31:0/748995585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560d7ccfe840 0x560d7cd0f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.522+0000 7f6b43e00640 1 -- 172.21.15.31:0/748995585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560d7ccfe840 msgr2=0x560d7cd0f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.522+0000 7f6b43e00640 1 --2- 172.21.15.31:0/748995585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560d7ccfe840 0x560d7cd0f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.522+0000 7f6b43e00640 1 -- 172.21.15.31:0/748995585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560d7cd039b0 msgr2=0x560d7ccffa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.522+0000 7f6b43e00640 1 --2- 172.21.15.31:0/748995585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560d7cd039b0 0x560d7ccffa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.522+0000 7f6b43e00640 1 -- 172.21.15.31:0/748995585 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d7cd02ce0 con 0x560d7ccfde50 2024-04-25T01:17:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.522+0000 7f6b49000640 1 --2- 172.21.15.31:0/748995585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560d7cd039b0 0x560d7ccffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:17:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.523+0000 7f6b43e00640 1 --2- 172.21.15.31:0/748995585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d7ccfde50 0x560d7cd03060 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6b34009900 tx=0x7f6b34031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb83eae615595c33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.523+0000 7f6b38c00640 1 -- 172.21.15.31:0/748995585 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b34035040 con 0x560d7ccfde50 2024-04-25T01:17:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.523+0000 7f6b38c00640 1 -- 172.21.15.31:0/748995585 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b34031d90 con 0x560d7ccfde50 2024-04-25T01:17:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.523+0000 7f6b38c00640 1 -- 172.21.15.31:0/748995585 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b34003850 con 0x560d7ccfde50 2024-04-25T01:17:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.524+0000 7f6b4dd70740 1 -- 172.21.15.31:0/748995585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d7ccfde50 msgr2=0x560d7cd03060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.524+0000 7f6b4dd70740 1 --2- 172.21.15.31:0/748995585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d7ccfde50 0x560d7cd03060 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6b34009900 tx=0x7f6b34031400 comp rx=0 tx=0).stop 2024-04-25T01:17:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.524+0000 7f6b4dd70740 1 -- 172.21.15.31:0/748995585 shutdown_connections 2024-04-25T01:17:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.524+0000 7f6b4dd70740 1 --2- 172.21.15.31:0/748995585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560d7cd039b0 0x560d7ccffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.524+0000 7f6b4dd70740 1 --2- 172.21.15.31:0/748995585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560d7ccfe840 0x560d7cd0f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.524+0000 7f6b4dd70740 1 --2- 172.21.15.31:0/748995585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d7ccfde50 0x560d7cd03060 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.524+0000 7f6b4dd70740 1 -- 172.21.15.31:0/748995585 >> 172.21.15.31:0/748995585 conn(0x560d7ccf94c0 msgr2=0x560d7ccf98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.524+0000 7f6b4dd70740 1 -- 172.21.15.31:0/748995585 shutdown_connections 2024-04-25T01:17:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.524+0000 7f6b4dd70740 1 -- 172.21.15.31:0/748995585 wait complete. 2024-04-25T01:17:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.524+0000 7f6b4dd70740 1 -- start start 2024-04-25T01:17:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.524+0000 7f6b4dd70740 1 Processor -- start 2024-04-25T01:17:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.525+0000 7f6b4dd70740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560d7ccfde50 0x560d7cd0c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.525+0000 7f6b4dd70740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d7ccfe840 0x560d7cd0c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.525+0000 7f6b4dd70740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560d7cd039b0 0x560d7cd80a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.525+0000 7f6b4dd70740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2400c900 con 0x560d7cd039b0 2024-04-25T01:17:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.525+0000 7f6b4dd70740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2400dd70 con 0x560d7ccfde50 2024-04-25T01:17:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.525+0000 7f6b4dd70740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2400cfd0 con 0x560d7ccfe840 2024-04-25T01:17:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.525+0000 7f6b4dd70740 10 client.0 _ll_register_callbacks cb 0x7f6b3c010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:17:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.525+0000 7f6b43e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560d7ccfde50 0x560d7cd0c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.525+0000 7f6b43e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560d7ccfde50 0x560d7cd0c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:49792/0 (socket says 172.21.15.31:49792) 2024-04-25T01:17:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.525+0000 7f6b43e00640 1 -- 172.21.15.31:0/3229199530 learned_addr learned my addr 172.21.15.31:0/3229199530 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:17:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.525+0000 7f6b43400640 1 --2- 172.21.15.31:0/3229199530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d7ccfe840 0x560d7cd0c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.525+0000 7f6b49000640 1 --2- 172.21.15.31:0/3229199530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560d7cd039b0 0x560d7cd80a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.526+0000 7f6b43e00640 1 -- 172.21.15.31:0/3229199530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d7ccfe840 msgr2=0x560d7cd0c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.526+0000 7f6b43e00640 1 --2- 172.21.15.31:0/3229199530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d7ccfe840 0x560d7cd0c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.526+0000 7f6b43e00640 1 -- 172.21.15.31:0/3229199530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560d7cd039b0 msgr2=0x560d7cd80a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.526+0000 7f6b43e00640 1 --2- 172.21.15.31:0/3229199530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560d7cd039b0 0x560d7cd80a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.526+0000 7f6b43e00640 1 -- 172.21.15.31:0/3229199530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d7cd0f8a0 con 0x560d7ccfde50 2024-04-25T01:17:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.526+0000 7f6b49000640 1 --2- 172.21.15.31:0/3229199530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560d7cd039b0 0x560d7cd80a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:17:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.526+0000 7f6b43e00640 1 --2- 172.21.15.31:0/3229199530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560d7ccfde50 0x560d7cd0c300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6b3402a920 tx=0x7f6b34005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc328b14e3cb5504 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.526+0000 7f6b2e000640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T01:17:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.526+0000 7f6b2e000640 1 -- 172.21.15.31:0/3229199530 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b340316b0 con 0x560d7ccfde50 2024-04-25T01:17:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.526+0000 7f6b4dd70740 1 -- 172.21.15.31:0/3229199530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b34031d90 con 0x560d7ccfde50 2024-04-25T01:17:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.526+0000 7f6b2e000640 1 -- 172.21.15.31:0/3229199530 <== mon.2 v2:172.21.15.40:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f6b34031850 con 0x560d7ccfde50 2024-04-25T01:17:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.527+0000 7f6b2e000640 1 -- 172.21.15.31:0/3229199530 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6b34031a50 con 0x560d7ccfde50 2024-04-25T01:17:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.527+0000 7f6b4dd70740 10 client.25307 fetch_fsmap learned FSMap version 24 2024-04-25T01:17:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.527+0000 7f6b4dd70740 1 -- 172.21.15.31:0/3229199530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560d7cd81fb0 con 0x560d7ccfde50 2024-04-25T01:17:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.527+0000 7f6b2e000640 1 -- 172.21.15.31:0/3229199530 <== mon.2 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b34031d90 con 0x560d7ccfde50 2024-04-25T01:17:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.527+0000 7f6b2e000640 1 -- 172.21.15.31:0/3229199530 <== mon.2 v2:172.21.15.40:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f6b340035f0 con 0x560d7ccfde50 2024-04-25T01:17:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.528+0000 7f6b4dd70740 10 client.25307 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T01:17:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.528+0000 7f6b4dd70740 10 client.25307 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:17:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.528+0000 7f6b4dd70740 20 client.25307 populate_metadata read hostname 'smithi031' 2024-04-25T01:17:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.528+0000 7f6b4dd70740 1 --2- 172.21.15.31:0/3229199530 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560d7cd824f0 0x560d7cd848e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.528+0000 7f6b4dd70740 4 client.25307 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:17:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.528+0000 7f6b4dd70740 1 -- 172.21.15.31:0/3229199530 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560d7cc68340 con 0x560d7cd824f0 2024-04-25T01:17:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.528+0000 7f6b43400640 1 --2- 172.21.15.31:0/3229199530 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560d7cd824f0 0x560d7cd848e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.528+0000 7f6b43400640 1 --2- 172.21.15.31:0/3229199530 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560d7cd824f0 0x560d7cd848e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45228044a39d6ed0 server_cookie=36eeb00d05164667 in_seq=0 out_seq=0 2024-04-25T01:17:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.528+0000 7f6b2e000640 10 client.25307 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:17:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.532+0000 7f6b2e000640 1 -- 172.21.15.31:0/3229199530 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560d7cc68340 con 0x560d7cd824f0 2024-04-25T01:17:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.532+0000 7f6b2e000640 10 client.25307 handle_command_reply: tid=0 2024-04-25T01:17:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.532+0000 7f6b4dd70740 1 client.25307 shutdown 2024-04-25T01:17:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.533+0000 7f6b4dd70740 1 -- 172.21.15.31:0/3229199530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560d7ccfde50 msgr2=0x560d7cd0c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.533+0000 7f6b4dd70740 1 --2- 172.21.15.31:0/3229199530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560d7ccfde50 0x560d7cd0c300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6b3402a920 tx=0x7f6b34005dd0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.533+0000 7f6b4dd70740 1 -- 172.21.15.31:0/3229199530 shutdown_connections 2024-04-25T01:17:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.533+0000 7f6b4dd70740 1 --2- 172.21.15.31:0/3229199530 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560d7cd824f0 0x560d7cd848e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.533+0000 7f6b4dd70740 1 --2- 172.21.15.31:0/3229199530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560d7cd039b0 0x560d7cd80a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.533+0000 7f6b4dd70740 1 --2- 172.21.15.31:0/3229199530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560d7ccfe840 0x560d7cd0c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.533+0000 7f6b4dd70740 1 --2- 172.21.15.31:0/3229199530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560d7ccfde50 0x560d7cd0c300 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.533+0000 7f6b4dd70740 1 -- 172.21.15.31:0/3229199530 >> 172.21.15.31:0/3229199530 conn(0x560d7ccf94c0 msgr2=0x560d7ccffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.533+0000 7f6b2e000640 0 client.25307 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:17:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.533+0000 7f6b4dd70740 1 -- 172.21.15.31:0/3229199530 shutdown_connections 2024-04-25T01:17:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.533+0000 7f6b4dd70740 1 -- 172.21.15.31:0/3229199530 wait complete. 2024-04-25T01:17:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.533+0000 7f6b4dd70740 20 client.25307 trim_cache size 0 max 16384 2024-04-25T01:17:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:17:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:17:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:17:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:17:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.534+0000 7f6b49a00640 1 -- 172.21.15.31:0/1714563754 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6b280a2c00 msgr2=0x7f6b280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.534+0000 7f6b49a00640 1 --2- 172.21.15.31:0/1714563754 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6b280a2c00 0x7f6b280a50c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6b30005d00 tx=0x7f6b300093b0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.534+0000 7f6b49a00640 1 -- 172.21.15.31:0/1714563754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b441031b0 msgr2=0x7f6b441a1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.534+0000 7f6b49a00640 1 --2- 172.21.15.31:0/1714563754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b441031b0 0x7f6b441a1180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00efd0 tx=0x7f6b3c00c6a0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.534+0000 7f6b49a00640 1 -- 172.21.15.31:0/1714563754 shutdown_connections 2024-04-25T01:17:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.534+0000 7f6b49a00640 1 --2- 172.21.15.31:0/1714563754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b441031b0 0x7f6b441a1180 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.534+0000 7f6b49a00640 1 --2- 172.21.15.31:0/1714563754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b441027f0 0x7f6b4419a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.534+0000 7f6b49a00640 1 --2- 172.21.15.31:0/1714563754 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f6b280a2c00 0x7f6b280a50c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.534+0000 7f6b49a00640 1 --2- 172.21.15.31:0/1714563754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b44101600 0x7f6b44199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.534+0000 7f6b49a00640 1 -- 172.21.15.31:0/1714563754 >> 172.21.15.31:0/1714563754 conn(0x7f6b440fcb70 msgr2=0x7f6b44073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.535+0000 7f6b49a00640 1 -- 172.21.15.31:0/1714563754 shutdown_connections 2024-04-25T01:17:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:05.535+0000 7f6b49a00640 1 -- 172.21.15.31:0/1714563754 wait complete. 2024-04-25T01:17:05.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:05 smithi169 ceph-mon[32759]: pgmap v425: 97 pgs: 97 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:17:05.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:05 smithi040 ceph-mon[32518]: pgmap v425: 97 pgs: 97 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:17:05.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:05 smithi031 ceph-mon[28053]: pgmap v425: 97 pgs: 97 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:17:07.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:07 smithi169 ceph-mon[32759]: pgmap v426: 97 pgs: 97 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T01:17:07.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:07 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:17:07.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:07 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:17:07.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:07 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:17:07.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:07 smithi040 ceph-mon[32518]: pgmap v426: 97 pgs: 97 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T01:17:07.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:07 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:17:07.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:07 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:17:07.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:07 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:17:07.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:07 smithi031 ceph-mon[28053]: pgmap v426: 97 pgs: 97 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T01:17:07.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:07 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:17:07.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:07 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:17:07.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:07 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:17:08.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:08 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:17:08.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:08 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:17:08.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:08 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:17:08.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:08 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:17:08.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:08 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:17:08.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:08 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:17:08.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:08 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:17:08.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:08 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:17:08.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:08 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:17:09.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:09 smithi169 ceph-mon[32759]: pgmap v427: 97 pgs: 97 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:17:09.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:09 smithi040 ceph-mon[32518]: pgmap v427: 97 pgs: 97 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:17:09.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:09 smithi031 ceph-mon[28053]: pgmap v427: 97 pgs: 97 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T01:17:11.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:11 smithi169 ceph-mon[32759]: pgmap v428: 97 pgs: 97 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T01:17:11.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:11 smithi040 ceph-mon[32518]: pgmap v428: 97 pgs: 97 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T01:17:11.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:11 smithi031 ceph-mon[28053]: pgmap v428: 97 pgs: 97 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T01:17:13.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:13 smithi169 ceph-mon[32759]: pgmap v429: 97 pgs: 97 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:17:13.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:13 smithi040 ceph-mon[32518]: pgmap v429: 97 pgs: 97 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:17:13.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:13 smithi031 ceph-mon[28053]: pgmap v429: 97 pgs: 97 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:17:15.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:15 smithi169 ceph-mon[32759]: pgmap v430: 97 pgs: 97 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:17:15.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:15 smithi040 ceph-mon[32518]: pgmap v430: 97 pgs: 97 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:17:15.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:15 smithi031 ceph-mon[28053]: pgmap v430: 97 pgs: 97 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:17:16.914 INFO:tasks.workunit.client.0.smithi031.stderr:Updating files: 61% (7721/12492) Updating files: 62% (7746/12492) Updating files: 63% (7870/12492) Updating files: 64% (7995/12492) Updating files: 65% (8120/12492) Updating files: 66% (8245/12492) Updating files: 67% (8370/12492) Updating files: 68% (8495/12492) Updating files: 69% (8620/12492) Updating files: 70% (8745/12492) Updating files: 71% (8870/12492) Updating files: 72% (8995/12492) Updating files: 73% (9120/12492) Updating files: 74% (9245/12492) Updating files: 75% (9369/12492) Updating files: 76% (9494/12492) Updating files: 77% (9619/12492) Updating files: 78% (9744/12492) Updating files: 79% (9869/12492) Updating files: 80% (9994/12492) Updating files: 81% (10119/12492) Updating files: 82% (10244/12492) Updating files: 83% (10369/12492) Updating files: 84% (10494/12492) Updating files: 85% (10619/12492) Updating files: 86% (10744/12492) Updating files: 87% (10869/12492) Updating files: 88% (10993/12492) Updating files: 89% (11118/12492) Updating files: 90% (11243/12492) Updating files: 91% (11368/12492) Updating files: 92% (11493/12492) Updating files: 93% (11618/12492) Updating files: 94% (11743/12492) Updating files: 95% (11868/12492) Updating files: 96% (11993/12492) Updating files: 97% (12118/12492) Updating files: 97% (12236/12492) Updating files: 98% (12243/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-25T01:17:17.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:17 smithi169 ceph-mon[32759]: pgmap v431: 97 pgs: 97 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T01:17:17.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:17 smithi040 ceph-mon[32518]: pgmap v431: 97 pgs: 97 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T01:17:17.741 INFO:tasks.workunit.client.0.smithi031.stderr:Note: switching to 'd2c45d977142e024a59b68a8f1dce8f0b9840551'. 2024-04-25T01:17:17.741 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-25T01:17:17.741 INFO:tasks.workunit.client.0.smithi031.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T01:17:17.741 INFO:tasks.workunit.client.0.smithi031.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T01:17:17.741 INFO:tasks.workunit.client.0.smithi031.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T01:17:17.741 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-25T01:17:17.742 INFO:tasks.workunit.client.0.smithi031.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T01:17:17.742 INFO:tasks.workunit.client.0.smithi031.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T01:17:17.742 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-25T01:17:17.742 INFO:tasks.workunit.client.0.smithi031.stderr: git switch -c 2024-04-25T01:17:17.742 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-25T01:17:17.742 INFO:tasks.workunit.client.0.smithi031.stderr:Or undo this operation with: 2024-04-25T01:17:17.742 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-25T01:17:17.742 INFO:tasks.workunit.client.0.smithi031.stderr: git switch - 2024-04-25T01:17:17.742 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-25T01:17:17.742 INFO:tasks.workunit.client.0.smithi031.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T01:17:17.742 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-25T01:17:17.743 INFO:tasks.workunit.client.0.smithi031.stderr:HEAD is now at d2c45d97714 Merge pull request #56528 from adk3798/squid-mypy-bump 2024-04-25T01:17:17.751 DEBUG:teuthology.orchestra.run.smithi031:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T01:17:17.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:17 smithi031 ceph-mon[28053]: pgmap v431: 97 pgs: 97 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T01:17:17.810 INFO:tasks.workunit.client.0.smithi031.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T01:17:17.818 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T01:17:17.818 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T01:17:18.662 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T01:17:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.680+0000 7fc467200640 1 -- 172.21.15.31:0/3073394912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc468075780 msgr2=0x7fc468073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.680+0000 7fc467200640 1 --2- 172.21.15.31:0/3073394912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc468075780 0x7fc468073bd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc458009900 tx=0x7fc45802a3f0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.681+0000 7fc467200640 1 -- 172.21.15.31:0/3073394912 shutdown_connections 2024-04-25T01:17:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.681+0000 7fc467200640 1 --2- 172.21.15.31:0/3073394912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc468109490 0x7fc46810b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.681+0000 7fc467200640 1 --2- 172.21.15.31:0/3073394912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4680741a0 0x7fc468074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.681+0000 7fc467200640 1 --2- 172.21.15.31:0/3073394912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc468075780 0x7fc468073bd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.681+0000 7fc467200640 1 -- 172.21.15.31:0/3073394912 >> 172.21.15.31:0/3073394912 conn(0x7fc4680fcb40 msgr2=0x7fc4680fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.681+0000 7fc467200640 1 -- 172.21.15.31:0/3073394912 shutdown_connections 2024-04-25T01:17:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.681+0000 7fc467200640 1 -- 172.21.15.31:0/3073394912 wait complete. 2024-04-25T01:17:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.682+0000 7fc467200640 1 Processor -- start 2024-04-25T01:17:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.682+0000 7fc467200640 1 -- start start 2024-04-25T01:17:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.682+0000 7fc467200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4680741a0 0x7fc468199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.682+0000 7fc467200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc468075780 0x7fc46819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.682+0000 7fc467200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc468109490 0x7fc4681a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.682+0000 7fc467200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc46810f480 con 0x7fc468075780 2024-04-25T01:17:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.682+0000 7fc467200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc46810f300 con 0x7fc4680741a0 2024-04-25T01:17:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.682+0000 7fc467200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc46810f600 con 0x7fc468109490 2024-04-25T01:17:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.683+0000 7fc465400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc468075780 0x7fc46819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.683+0000 7fc465e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4680741a0 0x7fc468199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.683+0000 7fc465400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc468075780 0x7fc46819a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59568/0 (socket says 172.21.15.31:59568) 2024-04-25T01:17:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.683+0000 7fc465400640 1 -- 172.21.15.31:0/2964077116 learned_addr learned my addr 172.21.15.31:0/2964077116 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:17:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.683+0000 7fc466800640 1 --2- 172.21.15.31:0/2964077116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc468109490 0x7fc4681a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.683+0000 7fc465e00640 1 -- 172.21.15.31:0/2964077116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc468109490 msgr2=0x7fc4681a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.683+0000 7fc465e00640 1 --2- 172.21.15.31:0/2964077116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc468109490 0x7fc4681a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.683+0000 7fc465e00640 1 -- 172.21.15.31:0/2964077116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc468075780 msgr2=0x7fc46819a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.683+0000 7fc465e00640 1 --2- 172.21.15.31:0/2964077116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc468075780 0x7fc46819a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.683+0000 7fc465e00640 1 -- 172.21.15.31:0/2964077116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4681a16c0 con 0x7fc4680741a0 2024-04-25T01:17:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.684+0000 7fc465e00640 1 --2- 172.21.15.31:0/2964077116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4680741a0 0x7fc468199af0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc45802a910 tx=0x7fc458032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.684+0000 7fc45ea00640 1 -- 172.21.15.31:0/2964077116 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45803f030 con 0x7fc4680741a0 2024-04-25T01:17:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.684+0000 7fc467200640 1 -- 172.21.15.31:0/2964077116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4681a1950 con 0x7fc4680741a0 2024-04-25T01:17:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.684+0000 7fc45ea00640 1 -- 172.21.15.31:0/2964077116 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc458003ce0 con 0x7fc4680741a0 2024-04-25T01:17:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.684+0000 7fc45ea00640 1 -- 172.21.15.31:0/2964077116 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc458035070 con 0x7fc4680741a0 2024-04-25T01:17:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.684+0000 7fc467200640 1 -- 172.21.15.31:0/2964077116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4681a1e60 con 0x7fc4680741a0 2024-04-25T01:17:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.687+0000 7fc46f806740 1 Processor -- start 2024-04-25T01:17:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.687+0000 7fc45ea00640 1 -- 172.21.15.31:0/2964077116 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc458042030 con 0x7fc4680741a0 2024-04-25T01:17:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.688+0000 7fc46f806740 1 -- start start 2024-04-25T01:17:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.688+0000 7fc46f806740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fffd017e50 0x55fffd018270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.688+0000 7fc46f806740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fffd018840 0x55fffd01d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.688+0000 7fc46f806740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fffd01d9b0 0x55fffd01fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.688+0000 7fc46f806740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55fffcf7aab0 con 0x55fffd017e50 2024-04-25T01:17:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.688+0000 7fc46f806740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55fffcf23640 con 0x55fffd018840 2024-04-25T01:17:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.688+0000 7fc46f806740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55fffcf76a70 con 0x55fffd01d9b0 2024-04-25T01:17:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.688+0000 7fc466800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fffd017e50 0x55fffd018270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.688+0000 7fc465e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fffd018840 0x55fffd01d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.688+0000 7fc465e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fffd018840 0x55fffd01d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:43502/0 (socket says 172.21.15.31:43502) 2024-04-25T01:17:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.688+0000 7fc465e00640 1 -- 172.21.15.31:0/2485375270 learned_addr learned my addr 172.21.15.31:0/2485375270 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:17:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.688+0000 7fc465400640 1 --2- 172.21.15.31:0/2485375270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fffd01d9b0 0x55fffd01fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.689+0000 7fc465e00640 1 -- 172.21.15.31:0/2485375270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fffd01d9b0 msgr2=0x55fffd01fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.689+0000 7fc465e00640 1 --2- 172.21.15.31:0/2485375270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fffd01d9b0 0x55fffd01fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.689+0000 7fc465e00640 1 -- 172.21.15.31:0/2485375270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fffd017e50 msgr2=0x55fffd018270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.689+0000 7fc465e00640 1 --2- 172.21.15.31:0/2485375270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fffd017e50 0x55fffd018270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.689+0000 7fc465e00640 1 -- 172.21.15.31:0/2485375270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fffcc8f210 con 0x55fffd018840 2024-04-25T01:17:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.689+0000 7fc465e00640 1 --2- 172.21.15.31:0/2485375270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fffd018840 0x55fffd01d340 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc4580f7560 tx=0x7fc4580f7650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6aca55c2e8614927 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.689+0000 7fc45ea00640 1 --2- 172.21.15.31:0/2964077116 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc44c0a2b80 0x7fc44c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.689+0000 7fc453000640 1 -- 172.21.15.31:0/2485375270 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4580b7fa0 con 0x55fffd018840 2024-04-25T01:17:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.690+0000 7fc453000640 1 -- 172.21.15.31:0/2485375270 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc45803b460 con 0x55fffd018840 2024-04-25T01:17:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.690+0000 7fc453000640 1 -- 172.21.15.31:0/2485375270 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4580bb150 con 0x55fffd018840 2024-04-25T01:17:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.690+0000 7fc45ea00640 1 -- 172.21.15.31:0/2964077116 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fc4580ef400 con 0x7fc4680741a0 2024-04-25T01:17:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.690+0000 7fc466800640 1 --2- 172.21.15.31:0/2964077116 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc44c0a2b80 0x7fc44c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.690+0000 7fc46f806740 1 -- 172.21.15.31:0/2485375270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fffd018840 msgr2=0x55fffd01d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.691+0000 7fc46f806740 1 --2- 172.21.15.31:0/2485375270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fffd018840 0x55fffd01d340 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc4580f7560 tx=0x7fc4580f7650 comp rx=0 tx=0).stop 2024-04-25T01:17:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.690+0000 7fc466800640 1 --2- 172.21.15.31:0/2964077116 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc44c0a2b80 0x7fc44c0a5040 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55fffd0190a0 tx=0x7fc460009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.691+0000 7fc46f806740 1 -- 172.21.15.31:0/2485375270 shutdown_connections 2024-04-25T01:17:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.691+0000 7fc46f806740 1 --2- 172.21.15.31:0/2485375270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fffd01d9b0 0x55fffd01fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.691+0000 7fc46f806740 1 --2- 172.21.15.31:0/2485375270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fffd018840 0x55fffd01d340 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.691+0000 7fc46f806740 1 --2- 172.21.15.31:0/2485375270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fffd017e50 0x55fffd018270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.691+0000 7fc46f806740 1 -- 172.21.15.31:0/2485375270 >> 172.21.15.31:0/2485375270 conn(0x55fffd0134c0 msgr2=0x55fffd0138c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.691+0000 7fc46f806740 1 -- 172.21.15.31:0/2485375270 shutdown_connections 2024-04-25T01:17:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.691+0000 7fc46f806740 1 -- 172.21.15.31:0/2485375270 wait complete. 2024-04-25T01:17:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.691+0000 7fc46f806740 1 -- start start 2024-04-25T01:17:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.692+0000 7fc46f806740 1 Processor -- start 2024-04-25T01:17:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.692+0000 7fc46f806740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fffd017e50 0x55fffd021d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.692+0000 7fc46f806740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fffd018840 0x55fffd022240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.692+0000 7fc46f806740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fffd01d9b0 0x55fffd029280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.692+0000 7fc46f806740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55fffd020720 con 0x55fffd01d9b0 2024-04-25T01:17:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.692+0000 7fc46f806740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55fffcf36990 con 0x55fffd018840 2024-04-25T01:17:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.692+0000 7fc46f806740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55fffd0208a0 con 0x55fffd017e50 2024-04-25T01:17:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.693+0000 7fc46f806740 10 client.0 _ll_register_callbacks cb 0x55fffcf38950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:17:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.693+0000 7fc465e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fffd017e50 0x55fffd021d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.693+0000 7fc466800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fffd01d9b0 0x55fffd029280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.693+0000 7fc465e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fffd017e50 0x55fffd021d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:42316/0 (socket says 172.21.15.31:42316) 2024-04-25T01:17:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.693+0000 7fc465e00640 1 -- 172.21.15.31:0/708321426 learned_addr learned my addr 172.21.15.31:0/708321426 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:17:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.693+0000 7fc465400640 1 --2- 172.21.15.31:0/708321426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fffd018840 0x55fffd022240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.693+0000 7fc465e00640 1 -- 172.21.15.31:0/708321426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fffd018840 msgr2=0x55fffd022240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.693+0000 7fc465e00640 1 --2- 172.21.15.31:0/708321426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fffd018840 0x55fffd022240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.693+0000 7fc465e00640 1 -- 172.21.15.31:0/708321426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fffd01d9b0 msgr2=0x55fffd029280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.693+0000 7fc465e00640 1 --2- 172.21.15.31:0/708321426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fffd01d9b0 0x55fffd029280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.693+0000 7fc465e00640 1 -- 172.21.15.31:0/708321426 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fffcd5b620 con 0x55fffd017e50 2024-04-25T01:17:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.693+0000 7fc465e00640 1 --2- 172.21.15.31:0/708321426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fffd017e50 0x55fffd021d00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc4580f7b50 tx=0x7fc4580f29b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecef936295afc725 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.693+0000 7fc447400640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:17:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.694+0000 7fc447400640 1 -- 172.21.15.31:0/708321426 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc458049070 con 0x55fffd017e50 2024-04-25T01:17:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.694+0000 7fc447400640 1 -- 172.21.15.31:0/708321426 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fc45802f6e0 con 0x55fffd017e50 2024-04-25T01:17:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.694+0000 7fc46f806740 1 -- 172.21.15.31:0/708321426 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc45803b460 con 0x55fffd017e50 2024-04-25T01:17:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.694+0000 7fc447400640 1 -- 172.21.15.31:0/708321426 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc4580d1070 con 0x55fffd017e50 2024-04-25T01:17:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.694+0000 7fc46f806740 10 client.25327 fetch_fsmap learned FSMap version 24 2024-04-25T01:17:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.694+0000 7fc46f806740 1 -- 172.21.15.31:0/708321426 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fffd020720 con 0x55fffd017e50 2024-04-25T01:17:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.694+0000 7fc447400640 1 -- 172.21.15.31:0/708321426 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc45803b460 con 0x55fffd017e50 2024-04-25T01:17:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.694+0000 7fc447400640 1 -- 172.21.15.31:0/708321426 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fc4580bb150 con 0x55fffd017e50 2024-04-25T01:17:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.695+0000 7fc46f806740 10 client.25327 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T01:17:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.695+0000 7fc46f806740 10 client.25327 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:17:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.695+0000 7fc46f806740 20 client.25327 populate_metadata read hostname 'smithi031' 2024-04-25T01:17:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.695+0000 7fc46f806740 1 --2- 172.21.15.31:0/708321426 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55fffd02d6d0 0x55fffd02fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.695+0000 7fc46f806740 4 client.25327 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:17:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.695+0000 7fc46f806740 1 -- 172.21.15.31:0/708321426 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fffcf87ff0 con 0x55fffd02d6d0 2024-04-25T01:17:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.695+0000 7fc465400640 1 --2- 172.21.15.31:0/708321426 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55fffd02d6d0 0x55fffd02fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:18.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.696+0000 7fc465400640 1 --2- 172.21.15.31:0/708321426 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55fffd02d6d0 0x55fffd02fb00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fe4c5eb296ad612 server_cookie=b2207b511c80dc51 in_seq=0 out_seq=0 2024-04-25T01:17:18.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.696+0000 7fc447400640 10 client.25327 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:17:18.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.696+0000 7fc447400640 1 -- 172.21.15.31:0/708321426 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fffcf87ff0 con 0x55fffd02d6d0 2024-04-25T01:17:18.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.696+0000 7fc447400640 10 client.25327 handle_command_reply: tid=0 2024-04-25T01:17:18.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.696+0000 7fc46f806740 1 client.25327 shutdown 2024-04-25T01:17:18.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.697+0000 7fc46f806740 1 -- 172.21.15.31:0/708321426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fffd017e50 msgr2=0x55fffd021d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:18.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.697+0000 7fc46f806740 1 --2- 172.21.15.31:0/708321426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fffd017e50 0x55fffd021d00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc4580f7b50 tx=0x7fc4580f29b0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.697+0000 7fc46f806740 1 -- 172.21.15.31:0/708321426 shutdown_connections 2024-04-25T01:17:18.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.697+0000 7fc46f806740 1 --2- 172.21.15.31:0/708321426 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55fffd02d6d0 0x55fffd02fb00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.697+0000 7fc46f806740 1 --2- 172.21.15.31:0/708321426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fffd01d9b0 0x55fffd029280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.697+0000 7fc46f806740 1 --2- 172.21.15.31:0/708321426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fffd018840 0x55fffd022240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.697+0000 7fc46f806740 1 --2- 172.21.15.31:0/708321426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fffd017e50 0x55fffd021d00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.697+0000 7fc46f806740 1 -- 172.21.15.31:0/708321426 >> 172.21.15.31:0/708321426 conn(0x55fffd0134c0 msgr2=0x55fffd012620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.697+0000 7fc447400640 0 client.25327 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:17:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.697+0000 7fc46f806740 1 -- 172.21.15.31:0/708321426 shutdown_connections 2024-04-25T01:17:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.697+0000 7fc46f806740 1 -- 172.21.15.31:0/708321426 wait complete. 2024-04-25T01:17:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.697+0000 7fc46f806740 20 client.25327 trim_cache size 0 max 16384 2024-04-25T01:17:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.713+0000 7fc46f806740 1 Processor -- start 2024-04-25T01:17:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.714+0000 7fc46f806740 1 -- start start 2024-04-25T01:17:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.714+0000 7fc46f806740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fffd017e50 0x55fffd0124a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.714+0000 7fc46f806740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fffd018840 0x55fffd0129e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.714+0000 7fc46f806740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fffd01d9b0 0x55fffd029210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.714+0000 7fc46f806740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc4681ad440 con 0x55fffd01d9b0 2024-04-25T01:17:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.714+0000 7fc46f806740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc4681aad10 con 0x55fffd017e50 2024-04-25T01:17:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.714+0000 7fc46f806740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc4681a8b60 con 0x55fffd018840 2024-04-25T01:17:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.714+0000 7fc466800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fffd01d9b0 0x55fffd029210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.714+0000 7fc465400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fffd018840 0x55fffd0129e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.714+0000 7fc466800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fffd01d9b0 0x55fffd029210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59586/0 (socket says 172.21.15.31:59586) 2024-04-25T01:17:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.714+0000 7fc465400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fffd018840 0x55fffd0129e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:42326/0 (socket says 172.21.15.31:42326) 2024-04-25T01:17:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.714+0000 7fc466800640 1 -- 172.21.15.31:0/1191359644 learned_addr learned my addr 172.21.15.31:0/1191359644 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:17:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.714+0000 7fc465e00640 1 --2- 172.21.15.31:0/1191359644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fffd017e50 0x55fffd0124a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.715+0000 7fc466800640 1 -- 172.21.15.31:0/1191359644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fffd018840 msgr2=0x55fffd0129e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.715+0000 7fc466800640 1 --2- 172.21.15.31:0/1191359644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fffd018840 0x55fffd0129e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.715+0000 7fc466800640 1 -- 172.21.15.31:0/1191359644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fffd017e50 msgr2=0x55fffd0124a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.715+0000 7fc466800640 1 --2- 172.21.15.31:0/1191359644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fffd017e50 0x55fffd0124a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.715+0000 7fc466800640 1 -- 172.21.15.31:0/1191359644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fffd030150 con 0x55fffd01d9b0 2024-04-25T01:17:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.715+0000 7fc466800640 1 --2- 172.21.15.31:0/1191359644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fffd01d9b0 0x55fffd029210 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc460003e90 tx=0x7fc46001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2391a4914f506cb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.715+0000 7fc452600640 1 -- 172.21.15.31:0/1191359644 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46000f5a0 con 0x55fffd01d9b0 2024-04-25T01:17:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.716+0000 7fc452600640 1 -- 172.21.15.31:0/1191359644 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc46000fc20 con 0x55fffd01d9b0 2024-04-25T01:17:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.716+0000 7fc452600640 1 -- 172.21.15.31:0/1191359644 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460029cf0 con 0x55fffd01d9b0 2024-04-25T01:17:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.716+0000 7fc46f806740 1 -- 172.21.15.31:0/1191359644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fffd01d9b0 msgr2=0x55fffd029210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.716+0000 7fc46f806740 1 --2- 172.21.15.31:0/1191359644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fffd01d9b0 0x55fffd029210 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc460003e90 tx=0x7fc46001a000 comp rx=0 tx=0).stop 2024-04-25T01:17:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.716+0000 7fc46f806740 1 -- 172.21.15.31:0/1191359644 shutdown_connections 2024-04-25T01:17:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.716+0000 7fc46f806740 1 --2- 172.21.15.31:0/1191359644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fffd01d9b0 0x55fffd029210 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.716+0000 7fc46f806740 1 --2- 172.21.15.31:0/1191359644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fffd018840 0x55fffd0129e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.716+0000 7fc46f806740 1 --2- 172.21.15.31:0/1191359644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fffd017e50 0x55fffd0124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.716+0000 7fc46f806740 1 -- 172.21.15.31:0/1191359644 >> 172.21.15.31:0/1191359644 conn(0x55fffd0134c0 msgr2=0x55fffd011d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.717+0000 7fc46f806740 1 -- 172.21.15.31:0/1191359644 shutdown_connections 2024-04-25T01:17:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.717+0000 7fc46f806740 1 -- 172.21.15.31:0/1191359644 wait complete. 2024-04-25T01:17:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.717+0000 7fc46f806740 1 -- start start 2024-04-25T01:17:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.717+0000 7fc46f806740 1 Processor -- start 2024-04-25T01:17:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.718+0000 7fc46f806740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fffd017e50 0x55fffd027070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.718+0000 7fc46f806740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fffd018840 0x55fffd0275b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.718+0000 7fc46f806740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fffd01d9b0 0x55fffd0285c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.718+0000 7fc46f806740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc4681a91d0 con 0x55fffd018840 2024-04-25T01:17:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.718+0000 7fc46f806740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc4681aa640 con 0x55fffd01d9b0 2024-04-25T01:17:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.718+0000 7fc46f806740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc4681a98a0 con 0x55fffd017e50 2024-04-25T01:17:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.718+0000 7fc46f806740 10 client.0 _ll_register_callbacks cb 0x7fc460009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:17:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.718+0000 7fc465400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fffd018840 0x55fffd0275b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.718+0000 7fc465400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fffd018840 0x55fffd0275b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59602/0 (socket says 172.21.15.31:59602) 2024-04-25T01:17:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.718+0000 7fc465400640 1 -- 172.21.15.31:0/1882813757 learned_addr learned my addr 172.21.15.31:0/1882813757 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:17:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.718+0000 7fc465e00640 1 --2- 172.21.15.31:0/1882813757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fffd017e50 0x55fffd027070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.718+0000 7fc466800640 1 --2- 172.21.15.31:0/1882813757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fffd01d9b0 0x55fffd0285c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.718+0000 7fc465400640 1 -- 172.21.15.31:0/1882813757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fffd017e50 msgr2=0x55fffd027070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.718+0000 7fc465400640 1 --2- 172.21.15.31:0/1882813757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fffd017e50 0x55fffd027070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.718+0000 7fc465400640 1 -- 172.21.15.31:0/1882813757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fffd01d9b0 msgr2=0x55fffd0285c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.718+0000 7fc465400640 1 --2- 172.21.15.31:0/1882813757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fffd01d9b0 0x55fffd0285c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.718+0000 7fc465400640 1 -- 172.21.15.31:0/1882813757 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fffd0132f0 con 0x55fffd018840 2024-04-25T01:17:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.719+0000 7fc465400640 1 --2- 172.21.15.31:0/1882813757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fffd018840 0x55fffd0275b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55fffd014390 tx=0x7fc454013910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25891c3d03160a6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.719+0000 7fc447400640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T01:17:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.719+0000 7fc447400640 1 -- 172.21.15.31:0/1882813757 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc454004460 con 0x55fffd018840 2024-04-25T01:17:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.719+0000 7fc46f806740 1 -- 172.21.15.31:0/1882813757 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc46000fc20 con 0x55fffd018840 2024-04-25T01:17:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.719+0000 7fc447400640 1 -- 172.21.15.31:0/1882813757 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fc454022110 con 0x55fffd018840 2024-04-25T01:17:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.719+0000 7fc447400640 1 -- 172.21.15.31:0/1882813757 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc454022310 con 0x55fffd018840 2024-04-25T01:17:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.719+0000 7fc46f806740 10 client.15198 fetch_fsmap learned FSMap version 24 2024-04-25T01:17:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.719+0000 7fc46f806740 1 -- 172.21.15.31:0/1882813757 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fffd09cb00 con 0x55fffd018840 2024-04-25T01:17:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.720+0000 7fc447400640 1 -- 172.21.15.31:0/1882813757 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc46000fc20 con 0x55fffd018840 2024-04-25T01:17:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.720+0000 7fc447400640 1 -- 172.21.15.31:0/1882813757 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fc45402b440 con 0x55fffd018840 2024-04-25T01:17:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.720+0000 7fc46f806740 10 client.15198 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T01:17:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.720+0000 7fc46f806740 10 client.15198 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:17:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.720+0000 7fc46f806740 20 client.15198 populate_metadata read hostname 'smithi031' 2024-04-25T01:17:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.720+0000 7fc46f806740 1 --2- 172.21.15.31:0/1882813757 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55fffd09d530 0x55fffd09f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.720+0000 7fc46f806740 4 client.15198 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:17:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.720+0000 7fc46f806740 1 -- 172.21.15.31:0/1882813757 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fffcf82340 con 0x55fffd09d530 2024-04-25T01:17:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.720+0000 7fc465e00640 1 --2- 172.21.15.31:0/1882813757 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55fffd09d530 0x55fffd09f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.721+0000 7fc465e00640 1 --2- 172.21.15.31:0/1882813757 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55fffd09d530 0x55fffd09f950 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb6ec9895f2b724a server_cookie=60c09e8488cd4b4a in_seq=0 out_seq=0 2024-04-25T01:17:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.721+0000 7fc447400640 10 client.15198 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:17:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.727+0000 7fc447400640 1 -- 172.21.15.31:0/1882813757 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fffcf82340 con 0x55fffd09d530 2024-04-25T01:17:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.727+0000 7fc447400640 10 client.15198 handle_command_reply: tid=0 2024-04-25T01:17:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.727+0000 7fc46f806740 1 client.15198 shutdown 2024-04-25T01:17:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.727+0000 7fc46f806740 1 -- 172.21.15.31:0/1882813757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fffd018840 msgr2=0x55fffd0275b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.727+0000 7fc46f806740 1 --2- 172.21.15.31:0/1882813757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fffd018840 0x55fffd0275b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55fffd014390 tx=0x7fc454013910 comp rx=0 tx=0).stop 2024-04-25T01:17:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.727+0000 7fc46f806740 1 -- 172.21.15.31:0/1882813757 shutdown_connections 2024-04-25T01:17:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.727+0000 7fc46f806740 1 --2- 172.21.15.31:0/1882813757 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55fffd09d530 0x55fffd09f950 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.727+0000 7fc46f806740 1 --2- 172.21.15.31:0/1882813757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fffd01d9b0 0x55fffd0285c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.727+0000 7fc46f806740 1 --2- 172.21.15.31:0/1882813757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fffd018840 0x55fffd0275b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.727+0000 7fc447400640 0 client.15198 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:17:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.727+0000 7fc46f806740 1 --2- 172.21.15.31:0/1882813757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fffd017e50 0x55fffd027070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.727+0000 7fc46f806740 1 -- 172.21.15.31:0/1882813757 >> 172.21.15.31:0/1882813757 conn(0x55fffd0134c0 msgr2=0x55fffd012620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.727+0000 7fc46f806740 1 -- 172.21.15.31:0/1882813757 shutdown_connections 2024-04-25T01:17:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.728+0000 7fc46f806740 1 -- 172.21.15.31:0/1882813757 wait complete. 2024-04-25T01:17:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.728+0000 7fc46f806740 20 client.15198 trim_cache size 0 max 16384 2024-04-25T01:17:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:17:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:17:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:17:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:17:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.728+0000 7fc467200640 1 -- 172.21.15.31:0/2964077116 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc44c0a2b80 msgr2=0x7fc44c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.728+0000 7fc467200640 1 --2- 172.21.15.31:0/2964077116 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc44c0a2b80 0x7fc44c0a5040 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55fffd0190a0 tx=0x7fc460009500 comp rx=0 tx=0).stop 2024-04-25T01:17:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.728+0000 7fc467200640 1 -- 172.21.15.31:0/2964077116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4680741a0 msgr2=0x7fc468199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.728+0000 7fc467200640 1 --2- 172.21.15.31:0/2964077116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4680741a0 0x7fc468199af0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc45802a910 tx=0x7fc458032ce0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.728+0000 7fc467200640 1 -- 172.21.15.31:0/2964077116 shutdown_connections 2024-04-25T01:17:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.728+0000 7fc467200640 1 --2- 172.21.15.31:0/2964077116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc468109490 0x7fc4681a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.728+0000 7fc467200640 1 --2- 172.21.15.31:0/2964077116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc468075780 0x7fc46819a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.728+0000 7fc467200640 1 --2- 172.21.15.31:0/2964077116 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc44c0a2b80 0x7fc44c0a5040 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.728+0000 7fc467200640 1 --2- 172.21.15.31:0/2964077116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4680741a0 0x7fc468199af0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.728+0000 7fc467200640 1 -- 172.21.15.31:0/2964077116 >> 172.21.15.31:0/2964077116 conn(0x7fc4680fcb40 msgr2=0x7fc468105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.729+0000 7fc467200640 1 -- 172.21.15.31:0/2964077116 shutdown_connections 2024-04-25T01:17:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:18.729+0000 7fc467200640 1 -- 172.21.15.31:0/2964077116 wait complete. 2024-04-25T01:17:18.837 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T01:17:18.919 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T01:17:18.925 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T01:17:18.925 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T01:17:19.006 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T01:17:19.017 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T01:17:19.017 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T01:17:19.073 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-04-25T01:17:19.074 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-04-25T01:17:19.075 DEBUG:teuthology.orchestra.run.smithi031:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-04-25T01:17:19.152 INFO:tasks.workunit.client.0.smithi031.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-04-25T01:17:19.154 INFO:tasks.workunit.client.0.smithi031.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites 2024-04-25T01:17:19.155 INFO:tasks.workunit.client.0.smithi031.stderr:+ trap -- retry EXIT 2024-04-25T01:17:19.155 INFO:tasks.workunit.client.0.smithi031.stderr:+ rm -rf ffsb 2024-04-25T01:17:19.158 INFO:tasks.workunit.client.0.smithi031.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2024-04-25T01:17:19.168 INFO:tasks.workunit.client.0.smithi031.stderr:Cloning into 'ffsb'... 2024-04-25T01:17:19.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:19 smithi169 ceph-mon[32759]: pgmap v432: 97 pgs: 97 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:17:19.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:19 smithi040 ceph-mon[32518]: pgmap v432: 97 pgs: 97 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:17:19.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:19 smithi031 ceph-mon[28053]: pgmap v432: 97 pgs: 97 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:17:20.546 INFO:tasks.workunit.client.0.smithi031.stderr:+ trap - EXIT 2024-04-25T01:17:20.546 INFO:tasks.workunit.client.0.smithi031.stderr:+ cd ffsb 2024-04-25T01:17:20.546 INFO:tasks.workunit.client.0.smithi031.stderr:+ ./configure 2024-04-25T01:17:20.931 INFO:tasks.workunit.client.0.smithi031.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-04-25T01:17:21.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:21 smithi169 ceph-mon[32759]: pgmap v433: 97 pgs: 97 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T01:17:21.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:21 smithi040 ceph-mon[32518]: pgmap v433: 97 pgs: 97 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T01:17:21.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:21 smithi031 ceph-mon[28053]: pgmap v433: 97 pgs: 97 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T01:17:21.950 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether build environment is sane... yes 2024-04-25T01:17:21.980 INFO:tasks.workunit.client.0.smithi031.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-04-25T01:17:21.981 INFO:tasks.workunit.client.0.smithi031.stdout:checking for gawk... gawk 2024-04-25T01:17:22.015 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether make sets $(MAKE)... yes 2024-04-25T01:17:22.123 INFO:tasks.workunit.client.0.smithi031.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-04-25T01:17:22.123 INFO:tasks.workunit.client.0.smithi031.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-04-25T01:17:22.144 INFO:tasks.workunit.client.0.smithi031.stdout:checking for style of include used by make... GNU 2024-04-25T01:17:22.151 INFO:tasks.workunit.client.0.smithi031.stdout:checking for gcc... gcc 2024-04-25T01:17:22.294 INFO:tasks.workunit.client.0.smithi031.stdout:checking for C compiler default output file name... a.out 2024-04-25T01:17:22.304 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether the C compiler works... yes 2024-04-25T01:17:22.309 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether we are cross compiling... no 2024-04-25T01:17:22.397 INFO:tasks.workunit.client.0.smithi031.stdout:checking for suffix of executables... 2024-04-25T01:17:22.481 INFO:tasks.workunit.client.0.smithi031.stdout:checking for suffix of object files... o 2024-04-25T01:17:22.570 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether we are using the GNU C compiler... yes 2024-04-25T01:17:22.659 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether gcc accepts -g... yes 2024-04-25T01:17:22.803 INFO:tasks.workunit.client.0.smithi031.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-25T01:17:23.007 INFO:tasks.workunit.client.0.smithi031.stdout:checking dependency style of gcc... gcc3 2024-04-25T01:17:23.168 INFO:tasks.workunit.client.0.smithi031.stdout:checking how to run the C preprocessor... gcc -E 2024-04-25T01:17:23.305 INFO:tasks.workunit.client.0.smithi031.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-25T01:17:23.313 INFO:tasks.workunit.client.0.smithi031.stdout:checking for egrep... /usr/bin/grep -E 2024-04-25T01:17:23.346 INFO:tasks.workunit.client.0.smithi031.stdout:checking for AIX... no 2024-04-25T01:17:23.372 INFO:tasks.workunit.client.0.smithi031.stdout:checking for gcc... (cached) gcc 2024-04-25T01:17:23.410 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-25T01:17:23.413 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-25T01:17:23.414 INFO:tasks.workunit.client.0.smithi031.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-25T01:17:23.415 INFO:tasks.workunit.client.0.smithi031.stdout:checking dependency style of gcc... (cached) gcc3 2024-04-25T01:17:23.546 INFO:tasks.workunit.client.0.smithi031.stdout:checking for main in -lm... yes 2024-04-25T01:17:23.665 INFO:tasks.workunit.client.0.smithi031.stdout:checking for main in -lpthread... yes 2024-04-25T01:17:23.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:23 smithi169 ceph-mon[32759]: pgmap v434: 97 pgs: 97 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-25T01:17:23.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:23 smithi040 ceph-mon[32518]: pgmap v434: 97 pgs: 97 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-25T01:17:23.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:23 smithi031 ceph-mon[28053]: pgmap v434: 97 pgs: 97 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-25T01:17:24.034 INFO:tasks.workunit.client.0.smithi031.stdout:checking for ANSI C header files... yes 2024-04-25T01:17:24.180 INFO:tasks.workunit.client.0.smithi031.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-04-25T01:17:24.295 INFO:tasks.workunit.client.0.smithi031.stdout:checking for sys/types.h... yes 2024-04-25T01:17:24.475 INFO:tasks.workunit.client.0.smithi031.stdout:checking for sys/stat.h... yes 2024-04-25T01:17:24.582 INFO:tasks.workunit.client.0.smithi031.stdout:checking for stdlib.h... yes 2024-04-25T01:17:24.700 INFO:tasks.workunit.client.0.smithi031.stdout:checking for string.h... yes 2024-04-25T01:17:24.811 INFO:tasks.workunit.client.0.smithi031.stdout:checking for memory.h... yes 2024-04-25T01:17:24.931 INFO:tasks.workunit.client.0.smithi031.stdout:checking for strings.h... yes 2024-04-25T01:17:25.054 INFO:tasks.workunit.client.0.smithi031.stdout:checking for inttypes.h... yes 2024-04-25T01:17:25.186 INFO:tasks.workunit.client.0.smithi031.stdout:checking for stdint.h... yes 2024-04-25T01:17:25.322 INFO:tasks.workunit.client.0.smithi031.stdout:checking for unistd.h... yes 2024-04-25T01:17:25.479 INFO:tasks.workunit.client.0.smithi031.stdout:checking pthread.h usability... yes 2024-04-25T01:17:25.543 INFO:tasks.workunit.client.0.smithi031.stdout:checking pthread.h presence... yes 2024-04-25T01:17:25.545 INFO:tasks.workunit.client.0.smithi031.stdout:checking for pthread.h... yes 2024-04-25T01:17:25.679 INFO:tasks.workunit.client.0.smithi031.stdout:checking fcntl.h usability... yes 2024-04-25T01:17:25.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:25 smithi169 ceph-mon[32759]: pgmap v435: 97 pgs: 97 active+clean; 2.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 162 KiB/s wr, 14 op/s 2024-04-25T01:17:25.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:25 smithi040 ceph-mon[32518]: pgmap v435: 97 pgs: 97 active+clean; 2.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 162 KiB/s wr, 14 op/s 2024-04-25T01:17:25.737 INFO:tasks.workunit.client.0.smithi031.stdout:checking fcntl.h presence... yes 2024-04-25T01:17:25.738 INFO:tasks.workunit.client.0.smithi031.stdout:checking for fcntl.h... yes 2024-04-25T01:17:25.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:25 smithi031 ceph-mon[28053]: pgmap v435: 97 pgs: 97 active+clean; 2.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 162 KiB/s wr, 14 op/s 2024-04-25T01:17:25.877 INFO:tasks.workunit.client.0.smithi031.stdout:checking limits.h usability... yes 2024-04-25T01:17:25.940 INFO:tasks.workunit.client.0.smithi031.stdout:checking limits.h presence... yes 2024-04-25T01:17:25.943 INFO:tasks.workunit.client.0.smithi031.stdout:checking for limits.h... yes 2024-04-25T01:17:25.982 INFO:tasks.workunit.client.0.smithi031.stdout:checking for stdint.h... (cached) yes 2024-04-25T01:17:26.099 INFO:tasks.workunit.client.0.smithi031.stdout:checking sys/time.h usability... yes 2024-04-25T01:17:26.157 INFO:tasks.workunit.client.0.smithi031.stdout:checking sys/time.h presence... yes 2024-04-25T01:17:26.160 INFO:tasks.workunit.client.0.smithi031.stdout:checking for sys/time.h... yes 2024-04-25T01:17:26.205 INFO:tasks.workunit.client.0.smithi031.stdout:checking for unistd.h... (cached) yes 2024-04-25T01:17:26.322 INFO:tasks.workunit.client.0.smithi031.stdout:checking sys/vfs.h usability... yes 2024-04-25T01:17:26.382 INFO:tasks.workunit.client.0.smithi031.stdout:checking sys/vfs.h presence... yes 2024-04-25T01:17:26.382 INFO:tasks.workunit.client.0.smithi031.stdout:checking for sys/vfs.h... yes 2024-04-25T01:17:26.497 INFO:tasks.workunit.client.0.smithi031.stdout:checking sys/limits.h usability... no 2024-04-25T01:17:26.560 INFO:tasks.workunit.client.0.smithi031.stdout:checking sys/limits.h presence... no 2024-04-25T01:17:26.562 INFO:tasks.workunit.client.0.smithi031.stdout:checking for sys/limits.h... no 2024-04-25T01:17:26.656 INFO:tasks.workunit.client.0.smithi031.stdout:checking for an ANSI C-conforming const... yes 2024-04-25T01:17:26.739 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-04-25T01:17:26.744 INFO:tasks.workunit.client.0.smithi031.stdout:checking for function prototypes... yes 2024-04-25T01:17:26.816 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether setvbuf arguments are reversed... no 2024-04-25T01:17:26.972 INFO:tasks.workunit.client.0.smithi031.stdout:checking for system... yes 2024-04-25T01:17:27.124 INFO:tasks.workunit.client.0.smithi031.stdout:checking for gettimeofday... yes 2024-04-25T01:17:27.276 INFO:tasks.workunit.client.0.smithi031.stdout:checking for mkdir... yes 2024-04-25T01:17:27.428 INFO:tasks.workunit.client.0.smithi031.stdout:checking for strerror... yes 2024-04-25T01:17:27.579 INFO:tasks.workunit.client.0.smithi031.stdout:checking for open64... yes 2024-04-25T01:17:27.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:27 smithi169 ceph-mon[32759]: pgmap v436: 97 pgs: 97 active+clean; 2.6 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 186 KiB/s wr, 21 op/s 2024-04-25T01:17:27.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:27 smithi040 ceph-mon[32518]: pgmap v436: 97 pgs: 97 active+clean; 2.6 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 186 KiB/s wr, 21 op/s 2024-04-25T01:17:27.733 INFO:tasks.workunit.client.0.smithi031.stdout:checking for stat64... yes 2024-04-25T01:17:27.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:27 smithi031 ceph-mon[28053]: pgmap v436: 97 pgs: 97 active+clean; 2.6 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 186 KiB/s wr, 21 op/s 2024-04-25T01:17:27.878 INFO:tasks.workunit.client.0.smithi031.stdout:checking for fseeko64... yes 2024-04-25T01:17:28.022 INFO:tasks.workunit.client.0.smithi031.stdout:checking for lrand48_r... yes 2024-04-25T01:17:28.174 INFO:tasks.workunit.client.0.smithi031.stdout:checking for srand48_r... yes 2024-04-25T01:17:28.230 INFO:tasks.workunit.client.0.smithi031.stdout:configure: creating ./config.status 2024-04-25T01:17:28.542 INFO:tasks.workunit.client.0.smithi031.stdout:config.status: creating Makefile 2024-04-25T01:17:28.591 INFO:tasks.workunit.client.0.smithi031.stdout:config.status: creating config.h 2024-04-25T01:17:28.647 INFO:tasks.workunit.client.0.smithi031.stdout:config.status: executing depfiles commands 2024-04-25T01:17:28.797 INFO:tasks.workunit.client.0.smithi031.stdout:config.status: executing default commands 2024-04-25T01:17:28.894 INFO:tasks.workunit.client.0.smithi031.stderr:+ make 2024-04-25T01:17:28.908 INFO:tasks.workunit.client.0.smithi031.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-04-25T01:17:28.913 INFO:tasks.workunit.client.0.smithi031.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-04-25T01:17:28.918 INFO:tasks.workunit.client.0.smithi031.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-04-25T01:17:28.918 INFO:tasks.workunit.client.0.smithi031.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-04-25T01:17:28.918 INFO:tasks.workunit.client.0.smithi031.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-04-25T01:17:28.918 INFO:tasks.workunit.client.0.smithi031.stderr: any GNU archive site. 2024-04-25T01:17:28.924 INFO:tasks.workunit.client.0.smithi031.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-04-25T01:17:28.929 INFO:tasks.workunit.client.0.smithi031.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-04-25T01:17:28.932 INFO:tasks.workunit.client.0.smithi031.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-04-25T01:17:28.932 INFO:tasks.workunit.client.0.smithi031.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-04-25T01:17:28.932 INFO:tasks.workunit.client.0.smithi031.stderr: You might want to install the `Automake' and `Perl' packages. 2024-04-25T01:17:28.932 INFO:tasks.workunit.client.0.smithi031.stderr: Grab them from any GNU archive site. 2024-04-25T01:17:29.022 INFO:tasks.workunit.client.0.smithi031.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-04-25T01:17:29.626 INFO:tasks.workunit.client.0.smithi031.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-04-25T01:17:29.626 INFO:tasks.workunit.client.0.smithi031.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-04-25T01:17:29.626 INFO:tasks.workunit.client.0.smithi031.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-04-25T01:17:29.626 INFO:tasks.workunit.client.0.smithi031.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-04-25T01:17:29.694 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:29 smithi169 ceph-mon[32759]: pgmap v437: 97 pgs: 97 active+clean; 2.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 222 KiB/s wr, 31 op/s 2024-04-25T01:17:29.713 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:29 smithi040 ceph-mon[32518]: pgmap v437: 97 pgs: 97 active+clean; 2.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 222 KiB/s wr, 31 op/s 2024-04-25T01:17:29.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:29 smithi031 ceph-mon[28053]: pgmap v437: 97 pgs: 97 active+clean; 2.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 222 KiB/s wr, 31 op/s 2024-04-25T01:17:29.828 INFO:tasks.workunit.client.0.smithi031.stdout:/bin/sh ./config.status --recheck 2024-04-25T01:17:30.008 INFO:tasks.workunit.client.0.smithi031.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2024-04-25T01:17:30.299 INFO:tasks.workunit.client.0.smithi031.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-04-25T01:17:31.321 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether build environment is sane... yes 2024-04-25T01:17:31.341 INFO:tasks.workunit.client.0.smithi031.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-04-25T01:17:31.342 INFO:tasks.workunit.client.0.smithi031.stdout:checking for gawk... gawk 2024-04-25T01:17:31.373 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether make sets $(MAKE)... yes 2024-04-25T01:17:31.474 INFO:tasks.workunit.client.0.smithi031.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-04-25T01:17:31.475 INFO:tasks.workunit.client.0.smithi031.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-04-25T01:17:31.498 INFO:tasks.workunit.client.0.smithi031.stdout:checking for style of include used by make... GNU 2024-04-25T01:17:31.504 INFO:tasks.workunit.client.0.smithi031.stdout:checking for gcc... gcc 2024-04-25T01:17:31.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:31 smithi169 ceph-mon[32759]: pgmap v438: 97 pgs: 97 active+clean; 4.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 365 KiB/s wr, 42 op/s 2024-04-25T01:17:31.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:31 smithi040 ceph-mon[32518]: pgmap v438: 97 pgs: 97 active+clean; 4.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 365 KiB/s wr, 42 op/s 2024-04-25T01:17:31.702 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether the C compiler works... yes 2024-04-25T01:17:31.702 INFO:tasks.workunit.client.0.smithi031.stdout:checking for C compiler default output file name... a.out 2024-04-25T01:17:31.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:31 smithi031 ceph-mon[28053]: pgmap v438: 97 pgs: 97 active+clean; 4.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 365 KiB/s wr, 42 op/s 2024-04-25T01:17:31.780 INFO:tasks.workunit.client.0.smithi031.stdout:checking for suffix of executables... 2024-04-25T01:17:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.863+0000 7fd7da600640 1 -- 172.21.15.31:0/3846279783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7d41039d0 msgr2=0x7fd7d4103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.863+0000 7fd7da600640 1 --2- 172.21.15.31:0/3846279783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7d41039d0 0x7fd7d4103df0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009ca0 tx=0x7fd7c402a8c0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.863+0000 7fd7da600640 1 -- 172.21.15.31:0/3846279783 shutdown_connections 2024-04-25T01:17:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.863+0000 7fd7da600640 1 --2- 172.21.15.31:0/3846279783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7d4105580 0x7fd7d410be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.863+0000 7fd7da600640 1 --2- 172.21.15.31:0/3846279783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7d4104bc0 0x7fd7d4105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.864+0000 7fd7da600640 1 --2- 172.21.15.31:0/3846279783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7d41039d0 0x7fd7d4103df0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.864+0000 7fd7da600640 1 -- 172.21.15.31:0/3846279783 >> 172.21.15.31:0/3846279783 conn(0x7fd7d40fef60 msgr2=0x7fd7d4101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.864+0000 7fd7da600640 1 -- 172.21.15.31:0/3846279783 shutdown_connections 2024-04-25T01:17:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.864+0000 7fd7da600640 1 -- 172.21.15.31:0/3846279783 wait complete. 2024-04-25T01:17:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.865+0000 7fd7da600640 1 Processor -- start 2024-04-25T01:17:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.865+0000 7fd7da600640 1 -- start start 2024-04-25T01:17:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.865+0000 7fd7da600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7d41039d0 0x7fd7d4199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.865+0000 7fd7da600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7d4104bc0 0x7fd7d419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.865+0000 7fd7da600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7d4105580 0x7fd7d41a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.865+0000 7fd7da600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d410f620 con 0x7fd7d4104bc0 2024-04-25T01:17:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.865+0000 7fd7da600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d410f4a0 con 0x7fd7d4105580 2024-04-25T01:17:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.865+0000 7fd7da600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d410f7a0 con 0x7fd7d41039d0 2024-04-25T01:17:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.866+0000 7fd7d3400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7d41039d0 0x7fd7d4199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.866+0000 7fd7caa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7d4104bc0 0x7fd7d419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.866+0000 7fd7d3e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7d4105580 0x7fd7d41a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.866+0000 7fd7d3400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7d41039d0 0x7fd7d4199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:54910/0 (socket says 172.21.15.31:54910) 2024-04-25T01:17:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.866+0000 7fd7d3400640 1 -- 172.21.15.31:0/803800867 learned_addr learned my addr 172.21.15.31:0/803800867 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:17:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.866+0000 7fd7d3e00640 1 -- 172.21.15.31:0/803800867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7d41039d0 msgr2=0x7fd7d4199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.866+0000 7fd7d3e00640 1 --2- 172.21.15.31:0/803800867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7d41039d0 0x7fd7d4199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.866+0000 7fd7d3e00640 1 -- 172.21.15.31:0/803800867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7d4104bc0 msgr2=0x7fd7d419a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.866+0000 7fd7d3e00640 1 --2- 172.21.15.31:0/803800867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7d4104bc0 0x7fd7d419a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.866+0000 7fd7d3e00640 1 -- 172.21.15.31:0/803800867 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d41a1830 con 0x7fd7d4105580 2024-04-25T01:17:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.866+0000 7fd7caa00640 1 --2- 172.21.15.31:0/803800867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7d4104bc0 0x7fd7d419a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:17:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.867+0000 7fd7d3e00640 1 --2- 172.21.15.31:0/803800867 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7d4105580 0x7fd7d41a1190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00e970 tx=0x7fd7cc00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.867+0000 7fd7d0c00640 1 -- 172.21.15.31:0/803800867 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc00ccc0 con 0x7fd7d4105580 2024-04-25T01:17:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.867+0000 7fd7d0c00640 1 -- 172.21.15.31:0/803800867 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7cc005020 con 0x7fd7d4105580 2024-04-25T01:17:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.867+0000 7fd7d0c00640 1 -- 172.21.15.31:0/803800867 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc004a00 con 0x7fd7d4105580 2024-04-25T01:17:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.867+0000 7fd7da600640 1 -- 172.21.15.31:0/803800867 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d41a1b20 con 0x7fd7d4105580 2024-04-25T01:17:31.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.867+0000 7fd7da600640 1 -- 172.21.15.31:0/803800867 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d41a2070 con 0x7fd7d4105580 2024-04-25T01:17:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.870+0000 7fd7dd56e740 1 Processor -- start 2024-04-25T01:17:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.872+0000 7fd7d0c00640 1 -- 172.21.15.31:0/803800867 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd7cc0051d0 con 0x7fd7d4105580 2024-04-25T01:17:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.872+0000 7fd7dd56e740 1 -- start start 2024-04-25T01:17:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.873+0000 7fd7dd56e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560aee8394b0 0x560aee8398d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.873+0000 7fd7dd56e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee839ea0 0x560aee83e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.873+0000 7fd7dd56e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560aee83f010 0x560aee8414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.873+0000 7fd7dd56e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560aee793ab0 con 0x560aee8394b0 2024-04-25T01:17:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.873+0000 7fd7dd56e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560aee73c640 con 0x560aee83f010 2024-04-25T01:17:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.873+0000 7fd7dd56e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560aee78fa70 con 0x560aee839ea0 2024-04-25T01:17:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.873+0000 7fd7d3400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee839ea0 0x560aee83e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.873+0000 7fd7d3400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee839ea0 0x560aee83e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:54924/0 (socket says 172.21.15.31:54924) 2024-04-25T01:17:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.873+0000 7fd7d3400640 1 -- 172.21.15.31:0/1381429901 learned_addr learned my addr 172.21.15.31:0/1381429901 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:17:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.873+0000 7fd7caa00640 1 --2- 172.21.15.31:0/1381429901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560aee8394b0 0x560aee8398d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.874+0000 7fd7caa00640 1 --2- 172.21.15.31:0/1381429901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560aee83f010 0x560aee8414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.874+0000 7fd7d3400640 1 -- 172.21.15.31:0/1381429901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560aee83f010 msgr2=0x560aee8414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.874+0000 7fd7d3400640 1 --2- 172.21.15.31:0/1381429901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560aee83f010 0x560aee8414a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.874+0000 7fd7d3400640 1 -- 172.21.15.31:0/1381429901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560aee8394b0 msgr2=0x560aee8398d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.874+0000 7fd7d3400640 1 --2- 172.21.15.31:0/1381429901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560aee8394b0 0x560aee8398d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.874+0000 7fd7d3400640 1 -- 172.21.15.31:0/1381429901 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aee4a8210 con 0x560aee839ea0 2024-04-25T01:17:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.874+0000 7fd7caa00640 1 --2- 172.21.15.31:0/1381429901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560aee83f010 0x560aee8414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:17:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.874+0000 7fd7d3400640 1 --2- 172.21.15.31:0/1381429901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee839ea0 0x560aee83e9a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4004ab0 tx=0x7fd7c4009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e77781370fc17a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.874+0000 7fd7d0c00640 1 --2- 172.21.15.31:0/803800867 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd7b80a2cb0 0x7fd7b80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.875+0000 7fd7c8c00640 1 -- 172.21.15.31:0/1381429901 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4049070 con 0x560aee839ea0 2024-04-25T01:17:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.875+0000 7fd7c8c00640 1 -- 172.21.15.31:0/1381429901 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7c4036040 con 0x560aee839ea0 2024-04-25T01:17:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.875+0000 7fd7c8c00640 1 -- 172.21.15.31:0/1381429901 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c402fd10 con 0x560aee839ea0 2024-04-25T01:17:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.875+0000 7fd7d0c00640 1 -- 172.21.15.31:0/803800867 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fd7cc0cefd0 con 0x7fd7d4105580 2024-04-25T01:17:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.875+0000 7fd7caa00640 1 --2- 172.21.15.31:0/803800867 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd7b80a2cb0 0x7fd7b80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.876+0000 7fd7dd56e740 1 -- 172.21.15.31:0/1381429901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee839ea0 msgr2=0x560aee83e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.876+0000 7fd7dd56e740 1 --2- 172.21.15.31:0/1381429901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee839ea0 0x560aee83e9a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4004ab0 tx=0x7fd7c4009360 comp rx=0 tx=0).stop 2024-04-25T01:17:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.876+0000 7fd7caa00640 1 --2- 172.21.15.31:0/803800867 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd7b80a2cb0 0x7fd7b80a5170 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x560aee83a700 tx=0x7fd7bc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.876+0000 7fd7dd56e740 1 -- 172.21.15.31:0/1381429901 shutdown_connections 2024-04-25T01:17:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.876+0000 7fd7dd56e740 1 --2- 172.21.15.31:0/1381429901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560aee83f010 0x560aee8414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.876+0000 7fd7dd56e740 1 --2- 172.21.15.31:0/1381429901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee839ea0 0x560aee83e9a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.876+0000 7fd7dd56e740 1 --2- 172.21.15.31:0/1381429901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560aee8394b0 0x560aee8398d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.876+0000 7fd7dd56e740 1 -- 172.21.15.31:0/1381429901 >> 172.21.15.31:0/1381429901 conn(0x560aee82c310 msgr2=0x560aee82c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.876+0000 7fd7dd56e740 1 -- 172.21.15.31:0/1381429901 shutdown_connections 2024-04-25T01:17:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.876+0000 7fd7dd56e740 1 -- 172.21.15.31:0/1381429901 wait complete. 2024-04-25T01:17:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.876+0000 7fd7dd56e740 1 -- start start 2024-04-25T01:17:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.877+0000 7fd7dd56e740 1 Processor -- start 2024-04-25T01:17:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.877+0000 7fd7dd56e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560aee8394b0 0x560aee8434e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.877+0000 7fd7dd56e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560aee839ea0 0x560aee843a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.877+0000 7fd7dd56e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee83f010 0x560aee84aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.877+0000 7fd7dd56e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560aee841de0 con 0x560aee8394b0 2024-04-25T01:17:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.877+0000 7fd7dd56e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560aee74f990 con 0x560aee839ea0 2024-04-25T01:17:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.877+0000 7fd7dd56e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560aee841f60 con 0x560aee83f010 2024-04-25T01:17:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.877+0000 7fd7dd56e740 10 client.0 _ll_register_callbacks cb 0x560aee751950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:17:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.877+0000 7fd7d3400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560aee8394b0 0x560aee8434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.877+0000 7fd7caa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560aee839ea0 0x560aee843a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.877+0000 7fd7d3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee83f010 0x560aee84aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.877+0000 7fd7d3400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560aee8394b0 0x560aee8434e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52222/0 (socket says 172.21.15.31:52222) 2024-04-25T01:17:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.877+0000 7fd7d3400640 1 -- 172.21.15.31:0/1983732041 learned_addr learned my addr 172.21.15.31:0/1983732041 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:17:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.877+0000 7fd7d3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee83f010 0x560aee84aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:54934/0 (socket says 172.21.15.31:54934) 2024-04-25T01:17:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.877+0000 7fd7caa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560aee839ea0 0x560aee843a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:50006/0 (socket says 172.21.15.31:50006) 2024-04-25T01:17:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.878+0000 7fd7d3e00640 1 -- 172.21.15.31:0/1983732041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560aee839ea0 msgr2=0x560aee843a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.878+0000 7fd7d3e00640 1 --2- 172.21.15.31:0/1983732041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560aee839ea0 0x560aee843a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.878+0000 7fd7d3e00640 1 -- 172.21.15.31:0/1983732041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560aee8394b0 msgr2=0x560aee8434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.878+0000 7fd7d3e00640 1 --2- 172.21.15.31:0/1983732041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560aee8394b0 0x560aee8434e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.878+0000 7fd7d3e00640 1 -- 172.21.15.31:0/1983732041 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aee574620 con 0x560aee83f010 2024-04-25T01:17:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.878+0000 7fd7d3400640 1 --2- 172.21.15.31:0/1983732041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560aee8394b0 0x560aee8434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:17:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.878+0000 7fd7d3e00640 1 --2- 172.21.15.31:0/1983732041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee83f010 0x560aee84aa60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc09b520 tx=0x7fd7cc099a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2295f852bd96f62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.878+0000 7fd7c1600640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:17:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.878+0000 7fd7c1600640 1 -- 172.21.15.31:0/1983732041 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc01c040 con 0x560aee83f010 2024-04-25T01:17:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.878+0000 7fd7c1600640 1 -- 172.21.15.31:0/1983732041 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fd7cc0a4200 con 0x560aee83f010 2024-04-25T01:17:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.878+0000 7fd7dd56e740 1 -- 172.21.15.31:0/1983732041 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7c4036040 con 0x560aee83f010 2024-04-25T01:17:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.879+0000 7fd7c1600640 1 -- 172.21.15.31:0/1983732041 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd7cc0b9050 con 0x560aee83f010 2024-04-25T01:17:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.879+0000 7fd7dd56e740 10 client.25345 fetch_fsmap learned FSMap version 24 2024-04-25T01:17:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.879+0000 7fd7dd56e740 1 -- 172.21.15.31:0/1983732041 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560aee84dad0 con 0x560aee83f010 2024-04-25T01:17:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.879+0000 7fd7c1600640 1 -- 172.21.15.31:0/1983732041 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7c4036040 con 0x560aee83f010 2024-04-25T01:17:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.879+0000 7fd7c1600640 1 -- 172.21.15.31:0/1983732041 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd7cc004ba0 con 0x560aee83f010 2024-04-25T01:17:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.879+0000 7fd7dd56e740 10 client.25345 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T01:17:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.879+0000 7fd7dd56e740 10 client.25345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:17:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.879+0000 7fd7dd56e740 20 client.25345 populate_metadata read hostname 'smithi031' 2024-04-25T01:17:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.879+0000 7fd7dd56e740 1 --2- 172.21.15.31:0/1983732041 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560aee84f210 0x560aee851640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.879+0000 7fd7dd56e740 4 client.25345 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:17:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.879+0000 7fd7dd56e740 1 -- 172.21.15.31:0/1983732041 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560aee7a0ff0 con 0x560aee84f210 2024-04-25T01:17:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.880+0000 7fd7d3400640 1 --2- 172.21.15.31:0/1983732041 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560aee84f210 0x560aee851640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:31.869 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether we are cross compiling... no 2024-04-25T01:17:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.880+0000 7fd7d3400640 1 --2- 172.21.15.31:0/1983732041 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560aee84f210 0x560aee851640 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c851210b6ba133 server_cookie=c1a4decf57afabf4 in_seq=0 out_seq=0 2024-04-25T01:17:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.880+0000 7fd7c1600640 10 client.25345 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:17:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.881+0000 7fd7c1600640 1 -- 172.21.15.31:0/1983732041 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560aee7a0ff0 con 0x560aee84f210 2024-04-25T01:17:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.881+0000 7fd7c1600640 10 client.25345 handle_command_reply: tid=0 2024-04-25T01:17:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.881+0000 7fd7dd56e740 1 client.25345 shutdown 2024-04-25T01:17:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.881+0000 7fd7dd56e740 1 -- 172.21.15.31:0/1983732041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee83f010 msgr2=0x560aee84aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.881+0000 7fd7dd56e740 1 --2- 172.21.15.31:0/1983732041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee83f010 0x560aee84aa60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc09b520 tx=0x7fd7cc099a20 comp rx=0 tx=0).stop 2024-04-25T01:17:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.881+0000 7fd7dd56e740 1 -- 172.21.15.31:0/1983732041 shutdown_connections 2024-04-25T01:17:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.881+0000 7fd7dd56e740 1 --2- 172.21.15.31:0/1983732041 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560aee84f210 0x560aee851640 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.881+0000 7fd7dd56e740 1 --2- 172.21.15.31:0/1983732041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee83f010 0x560aee84aa60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.881+0000 7fd7dd56e740 1 --2- 172.21.15.31:0/1983732041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560aee839ea0 0x560aee843a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.881+0000 7fd7dd56e740 1 --2- 172.21.15.31:0/1983732041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560aee8394b0 0x560aee8434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.881+0000 7fd7dd56e740 1 -- 172.21.15.31:0/1983732041 >> 172.21.15.31:0/1983732041 conn(0x560aee82c310 msgr2=0x560aee82b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:31.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.881+0000 7fd7c1600640 0 client.25345 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:17:31.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.882+0000 7fd7dd56e740 1 -- 172.21.15.31:0/1983732041 shutdown_connections 2024-04-25T01:17:31.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.882+0000 7fd7dd56e740 1 -- 172.21.15.31:0/1983732041 wait complete. 2024-04-25T01:17:31.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.882+0000 7fd7dd56e740 20 client.25345 trim_cache size 0 max 16384 2024-04-25T01:17:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.899+0000 7fd7dd56e740 1 Processor -- start 2024-04-25T01:17:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.899+0000 7fd7dd56e740 1 -- start start 2024-04-25T01:17:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.899+0000 7fd7dd56e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee8394b0 0x560aee856f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.899+0000 7fd7dd56e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560aee839ea0 0x560aee857490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.899+0000 7fd7dd56e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560aee83f010 0x560aee84aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.899+0000 7fd7dd56e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d41a9050 con 0x560aee83f010 2024-04-25T01:17:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.899+0000 7fd7dd56e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d41a6920 con 0x560aee839ea0 2024-04-25T01:17:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.900+0000 7fd7dd56e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d41a4770 con 0x560aee8394b0 2024-04-25T01:17:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.900+0000 7fd7caa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560aee839ea0 0x560aee857490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.900+0000 7fd7d3400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee8394b0 0x560aee856f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.900+0000 7fd7d3e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560aee83f010 0x560aee84aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.900+0000 7fd7caa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560aee839ea0 0x560aee857490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:50008/0 (socket says 172.21.15.31:50008) 2024-04-25T01:17:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.900+0000 7fd7d3400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee8394b0 0x560aee856f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:54942/0 (socket says 172.21.15.31:54942) 2024-04-25T01:17:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.900+0000 7fd7caa00640 1 -- 172.21.15.31:0/1405175940 learned_addr learned my addr 172.21.15.31:0/1405175940 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:17:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.900+0000 7fd7d3400640 1 -- 172.21.15.31:0/1405175940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560aee839ea0 msgr2=0x560aee857490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.900+0000 7fd7d3400640 1 --2- 172.21.15.31:0/1405175940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560aee839ea0 0x560aee857490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.900+0000 7fd7d3400640 1 -- 172.21.15.31:0/1405175940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560aee83f010 msgr2=0x560aee84aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.900+0000 7fd7d3400640 1 --2- 172.21.15.31:0/1405175940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560aee83f010 0x560aee84aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.900+0000 7fd7d3400640 1 -- 172.21.15.31:0/1405175940 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aee82c140 con 0x560aee8394b0 2024-04-25T01:17:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.900+0000 7fd7d3e00640 1 --2- 172.21.15.31:0/1405175940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560aee83f010 0x560aee84aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:17:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.901+0000 7fd7d3400640 1 --2- 172.21.15.31:0/1405175940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee8394b0 0x560aee856f50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009620 tx=0x7fd7c402a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=844b334db8ffd025 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.901+0000 7fd7c3e00640 1 -- 172.21.15.31:0/1405175940 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4049040 con 0x560aee8394b0 2024-04-25T01:17:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.901+0000 7fd7c3e00640 1 -- 172.21.15.31:0/1405175940 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7c4059070 con 0x560aee8394b0 2024-04-25T01:17:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.901+0000 7fd7c3e00640 1 -- 172.21.15.31:0/1405175940 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c40532d0 con 0x560aee8394b0 2024-04-25T01:17:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.902+0000 7fd7dd56e740 1 -- 172.21.15.31:0/1405175940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee8394b0 msgr2=0x560aee856f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.902+0000 7fd7dd56e740 1 --2- 172.21.15.31:0/1405175940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee8394b0 0x560aee856f50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009620 tx=0x7fd7c402a8c0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.902+0000 7fd7dd56e740 1 -- 172.21.15.31:0/1405175940 shutdown_connections 2024-04-25T01:17:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.902+0000 7fd7dd56e740 1 --2- 172.21.15.31:0/1405175940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560aee83f010 0x560aee84aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.902+0000 7fd7dd56e740 1 --2- 172.21.15.31:0/1405175940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560aee839ea0 0x560aee857490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.902+0000 7fd7dd56e740 1 --2- 172.21.15.31:0/1405175940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee8394b0 0x560aee856f50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.902+0000 7fd7dd56e740 1 -- 172.21.15.31:0/1405175940 >> 172.21.15.31:0/1405175940 conn(0x560aee82c310 msgr2=0x560aee826fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.902+0000 7fd7dd56e740 1 -- 172.21.15.31:0/1405175940 shutdown_connections 2024-04-25T01:17:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.902+0000 7fd7dd56e740 1 -- 172.21.15.31:0/1405175940 wait complete. 2024-04-25T01:17:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.903+0000 7fd7dd56e740 1 -- start start 2024-04-25T01:17:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.903+0000 7fd7dd56e740 1 Processor -- start 2024-04-25T01:17:31.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.903+0000 7fd7dd56e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560aee8394b0 0x560aee847fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:31.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.903+0000 7fd7dd56e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560aee839ea0 0x560aee8484f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:31.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.903+0000 7fd7dd56e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee83f010 0x560aee8494b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:31.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.903+0000 7fd7dd56e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d41a4de0 con 0x560aee8394b0 2024-04-25T01:17:31.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.903+0000 7fd7dd56e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d41a6250 con 0x560aee839ea0 2024-04-25T01:17:31.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.903+0000 7fd7dd56e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d41a54b0 con 0x560aee83f010 2024-04-25T01:17:31.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.903+0000 7fd7dd56e740 10 client.0 _ll_register_callbacks cb 0x7fd7cc097370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:17:31.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.903+0000 7fd7d3400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560aee8394b0 0x560aee847fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:31.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.903+0000 7fd7caa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560aee839ea0 0x560aee8484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.904+0000 7fd7d3400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560aee8394b0 0x560aee847fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52228/0 (socket says 172.21.15.31:52228) 2024-04-25T01:17:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.904+0000 7fd7d3400640 1 -- 172.21.15.31:0/656988636 learned_addr learned my addr 172.21.15.31:0/656988636 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:17:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.904+0000 7fd7d3e00640 1 --2- 172.21.15.31:0/656988636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee83f010 0x560aee8494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.904+0000 7fd7caa00640 1 -- 172.21.15.31:0/656988636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee83f010 msgr2=0x560aee8494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.904+0000 7fd7caa00640 1 --2- 172.21.15.31:0/656988636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee83f010 0x560aee8494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.904+0000 7fd7caa00640 1 -- 172.21.15.31:0/656988636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560aee8394b0 msgr2=0x560aee847fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.904+0000 7fd7caa00640 1 --2- 172.21.15.31:0/656988636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560aee8394b0 0x560aee847fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.904+0000 7fd7caa00640 1 -- 172.21.15.31:0/656988636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aee849aa0 con 0x560aee839ea0 2024-04-25T01:17:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.904+0000 7fd7d3400640 1 --2- 172.21.15.31:0/656988636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560aee8394b0 0x560aee847fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:17:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.904+0000 7fd7d3e00640 1 --2- 172.21.15.31:0/656988636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee83f010 0x560aee8494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:17:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.904+0000 7fd7caa00640 1 --2- 172.21.15.31:0/656988636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560aee839ea0 0x560aee8484f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc009fd0 tx=0x7fd7bc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fef6b449f195d30e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.904+0000 7fd7c1600640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T01:17:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.905+0000 7fd7c1600640 1 -- 172.21.15.31:0/656988636 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc003800 con 0x560aee839ea0 2024-04-25T01:17:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.905+0000 7fd7c1600640 1 -- 172.21.15.31:0/656988636 <== mon.2 v2:172.21.15.40:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fd7bc0039a0 con 0x560aee839ea0 2024-04-25T01:17:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.905+0000 7fd7dd56e740 1 -- 172.21.15.31:0/656988636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7c4059070 con 0x560aee839ea0 2024-04-25T01:17:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.905+0000 7fd7c1600640 1 -- 172.21.15.31:0/656988636 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd7bc003ba0 con 0x560aee839ea0 2024-04-25T01:17:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.905+0000 7fd7dd56e740 10 client.25340 fetch_fsmap learned FSMap version 24 2024-04-25T01:17:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.905+0000 7fd7dd56e740 1 -- 172.21.15.31:0/656988636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560aee84ae20 con 0x560aee839ea0 2024-04-25T01:17:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.905+0000 7fd7c1600640 1 -- 172.21.15.31:0/656988636 <== mon.2 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7c4059070 con 0x560aee839ea0 2024-04-25T01:17:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.906+0000 7fd7c1600640 1 -- 172.21.15.31:0/656988636 <== mon.2 v2:172.21.15.40:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd7bc01e410 con 0x560aee839ea0 2024-04-25T01:17:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.906+0000 7fd7dd56e740 10 client.25340 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T01:17:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.906+0000 7fd7dd56e740 10 client.25340 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:17:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.906+0000 7fd7dd56e740 20 client.25340 populate_metadata read hostname 'smithi031' 2024-04-25T01:17:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.906+0000 7fd7dd56e740 1 --2- 172.21.15.31:0/656988636 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560aee8b5fb0 0x560aee8b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.906+0000 7fd7dd56e740 4 client.25340 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:17:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.906+0000 7fd7dd56e740 1 -- 172.21.15.31:0/656988636 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560aee79b340 con 0x560aee8b5fb0 2024-04-25T01:17:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.906+0000 7fd7d3400640 1 --2- 172.21.15.31:0/656988636 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560aee8b5fb0 0x560aee8b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.906+0000 7fd7d3400640 1 --2- 172.21.15.31:0/656988636 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560aee8b5fb0 0x560aee8b83d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51509a24402e960f server_cookie=654d8083f12d275 in_seq=0 out_seq=0 2024-04-25T01:17:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:31.906+0000 7fd7c1600640 10 client.25340 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:17:31.983 INFO:tasks.workunit.client.0.smithi031.stdout:checking for suffix of object files... o 2024-04-25T01:17:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.011+0000 7fd7c1600640 1 -- 172.21.15.31:0/656988636 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560aee79b340 con 0x560aee8b5fb0 2024-04-25T01:17:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.011+0000 7fd7c1600640 10 client.25340 handle_command_reply: tid=0 2024-04-25T01:17:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.011+0000 7fd7dd56e740 1 client.25340 shutdown 2024-04-25T01:17:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.011+0000 7fd7dd56e740 1 -- 172.21.15.31:0/656988636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560aee839ea0 msgr2=0x560aee8484f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.011+0000 7fd7dd56e740 1 --2- 172.21.15.31:0/656988636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560aee839ea0 0x560aee8484f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc009fd0 tx=0x7fd7bc004730 comp rx=0 tx=0).stop 2024-04-25T01:17:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.011+0000 7fd7dd56e740 1 -- 172.21.15.31:0/656988636 shutdown_connections 2024-04-25T01:17:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.011+0000 7fd7dd56e740 1 --2- 172.21.15.31:0/656988636 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560aee8b5fb0 0x560aee8b83d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.011+0000 7fd7dd56e740 1 --2- 172.21.15.31:0/656988636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560aee83f010 0x560aee8494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.011+0000 7fd7dd56e740 1 --2- 172.21.15.31:0/656988636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560aee839ea0 0x560aee8484f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.011+0000 7fd7dd56e740 1 --2- 172.21.15.31:0/656988636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560aee8394b0 0x560aee847fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.011+0000 7fd7c1600640 0 client.25340 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:17:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.011+0000 7fd7dd56e740 1 -- 172.21.15.31:0/656988636 >> 172.21.15.31:0/656988636 conn(0x560aee82c310 msgr2=0x560aee82f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.011+0000 7fd7dd56e740 1 -- 172.21.15.31:0/656988636 shutdown_connections 2024-04-25T01:17:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.011+0000 7fd7dd56e740 1 -- 172.21.15.31:0/656988636 wait complete. 2024-04-25T01:17:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.011+0000 7fd7dd56e740 20 client.25340 trim_cache size 0 max 16384 2024-04-25T01:17:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:17:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:17:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:17:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:17:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.012+0000 7fd7da600640 1 -- 172.21.15.31:0/803800867 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd7b80a2cb0 msgr2=0x7fd7b80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.012+0000 7fd7da600640 1 --2- 172.21.15.31:0/803800867 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd7b80a2cb0 0x7fd7b80a5170 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x560aee83a700 tx=0x7fd7bc005850 comp rx=0 tx=0).stop 2024-04-25T01:17:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.012+0000 7fd7da600640 1 -- 172.21.15.31:0/803800867 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7d4105580 msgr2=0x7fd7d41a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.012+0000 7fd7da600640 1 --2- 172.21.15.31:0/803800867 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7d4105580 0x7fd7d41a1190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00e970 tx=0x7fd7cc00ee30 comp rx=0 tx=0).stop 2024-04-25T01:17:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.012+0000 7fd7da600640 1 -- 172.21.15.31:0/803800867 shutdown_connections 2024-04-25T01:17:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.012+0000 7fd7da600640 1 --2- 172.21.15.31:0/803800867 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd7b80a2cb0 0x7fd7b80a5170 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.012+0000 7fd7da600640 1 --2- 172.21.15.31:0/803800867 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7d4105580 0x7fd7d41a1190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.012+0000 7fd7da600640 1 --2- 172.21.15.31:0/803800867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7d4104bc0 0x7fd7d419a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.012+0000 7fd7da600640 1 --2- 172.21.15.31:0/803800867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7d41039d0 0x7fd7d4199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.012+0000 7fd7da600640 1 -- 172.21.15.31:0/803800867 >> 172.21.15.31:0/803800867 conn(0x7fd7d40fef60 msgr2=0x7fd7d4107e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.012+0000 7fd7da600640 1 -- 172.21.15.31:0/803800867 shutdown_connections 2024-04-25T01:17:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:32.012+0000 7fd7da600640 1 -- 172.21.15.31:0/803800867 wait complete. 2024-04-25T01:17:32.034 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether we are using the GNU C compiler... yes 2024-04-25T01:17:32.084 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether gcc accepts -g... yes 2024-04-25T01:17:32.170 INFO:tasks.workunit.client.0.smithi031.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-25T01:17:32.378 INFO:tasks.workunit.client.0.smithi031.stdout:checking dependency style of gcc... gcc3 2024-04-25T01:17:32.503 INFO:tasks.workunit.client.0.smithi031.stdout:checking how to run the C preprocessor... gcc -E 2024-04-25T01:17:32.620 INFO:tasks.workunit.client.0.smithi031.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-25T01:17:32.625 INFO:tasks.workunit.client.0.smithi031.stdout:checking for egrep... /usr/bin/grep -E 2024-04-25T01:17:32.919 INFO:tasks.workunit.client.0.smithi031.stdout:checking for ANSI C header files... yes 2024-04-25T01:17:32.996 INFO:tasks.workunit.client.0.smithi031.stdout:checking for sys/types.h... yes 2024-04-25T01:17:33.088 INFO:tasks.workunit.client.0.smithi031.stdout:checking for sys/stat.h... yes 2024-04-25T01:17:33.183 INFO:tasks.workunit.client.0.smithi031.stdout:checking for stdlib.h... yes 2024-04-25T01:17:33.280 INFO:tasks.workunit.client.0.smithi031.stdout:checking for string.h... yes 2024-04-25T01:17:33.380 INFO:tasks.workunit.client.0.smithi031.stdout:checking for memory.h... yes 2024-04-25T01:17:33.476 INFO:tasks.workunit.client.0.smithi031.stdout:checking for strings.h... yes 2024-04-25T01:17:33.574 INFO:tasks.workunit.client.0.smithi031.stdout:checking for inttypes.h... yes 2024-04-25T01:17:33.665 INFO:tasks.workunit.client.0.smithi031.stdout:checking for stdint.h... yes 2024-04-25T01:17:33.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:33 smithi169 ceph-mon[32759]: pgmap v439: 97 pgs: 97 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 365 KiB/s wr, 41 op/s 2024-04-25T01:17:33.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:33 smithi040 ceph-mon[32518]: pgmap v439: 97 pgs: 97 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 365 KiB/s wr, 41 op/s 2024-04-25T01:17:33.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:33 smithi031 ceph-mon[28053]: pgmap v439: 97 pgs: 97 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 365 KiB/s wr, 41 op/s 2024-04-25T01:17:33.767 INFO:tasks.workunit.client.0.smithi031.stdout:checking for unistd.h... yes 2024-04-25T01:17:33.865 INFO:tasks.workunit.client.0.smithi031.stdout:checking minix/config.h usability... no 2024-04-25T01:17:33.923 INFO:tasks.workunit.client.0.smithi031.stdout:checking minix/config.h presence... no 2024-04-25T01:17:33.923 INFO:tasks.workunit.client.0.smithi031.stdout:checking for minix/config.h... no 2024-04-25T01:17:34.008 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-04-25T01:17:34.009 INFO:tasks.workunit.client.0.smithi031.stdout:checking for gcc... (cached) gcc 2024-04-25T01:17:34.093 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-25T01:17:34.093 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-25T01:17:34.093 INFO:tasks.workunit.client.0.smithi031.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-25T01:17:34.094 INFO:tasks.workunit.client.0.smithi031.stdout:checking dependency style of gcc... (cached) gcc3 2024-04-25T01:17:34.199 INFO:tasks.workunit.client.0.smithi031.stdout:checking for main in -lm... yes 2024-04-25T01:17:34.300 INFO:tasks.workunit.client.0.smithi031.stdout:checking for main in -lpthread... yes 2024-04-25T01:17:34.304 INFO:tasks.workunit.client.0.smithi031.stdout:checking for ANSI C header files... (cached) yes 2024-04-25T01:17:34.411 INFO:tasks.workunit.client.0.smithi031.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-04-25T01:17:34.498 INFO:tasks.workunit.client.0.smithi031.stdout:checking pthread.h usability... yes 2024-04-25T01:17:34.542 INFO:tasks.workunit.client.0.smithi031.stdout:checking pthread.h presence... yes 2024-04-25T01:17:34.542 INFO:tasks.workunit.client.0.smithi031.stdout:checking for pthread.h... yes 2024-04-25T01:17:34.705 INFO:tasks.workunit.client.0.smithi031.stdout:checking fcntl.h usability... yes 2024-04-25T01:17:34.737 INFO:tasks.workunit.client.0.smithi031.stdout:checking fcntl.h presence... yes 2024-04-25T01:17:34.737 INFO:tasks.workunit.client.0.smithi031.stdout:checking for fcntl.h... yes 2024-04-25T01:17:34.841 INFO:tasks.workunit.client.0.smithi031.stdout:checking limits.h usability... yes 2024-04-25T01:17:34.879 INFO:tasks.workunit.client.0.smithi031.stdout:checking limits.h presence... yes 2024-04-25T01:17:34.879 INFO:tasks.workunit.client.0.smithi031.stdout:checking for limits.h... yes 2024-04-25T01:17:34.908 INFO:tasks.workunit.client.0.smithi031.stdout:checking for stdint.h... (cached) yes 2024-04-25T01:17:34.998 INFO:tasks.workunit.client.0.smithi031.stdout:checking sys/time.h usability... yes 2024-04-25T01:17:35.039 INFO:tasks.workunit.client.0.smithi031.stdout:checking sys/time.h presence... yes 2024-04-25T01:17:35.039 INFO:tasks.workunit.client.0.smithi031.stdout:checking for sys/time.h... yes 2024-04-25T01:17:35.071 INFO:tasks.workunit.client.0.smithi031.stdout:checking for unistd.h... (cached) yes 2024-04-25T01:17:35.162 INFO:tasks.workunit.client.0.smithi031.stdout:checking sys/vfs.h usability... yes 2024-04-25T01:17:35.204 INFO:tasks.workunit.client.0.smithi031.stdout:checking sys/vfs.h presence... yes 2024-04-25T01:17:35.204 INFO:tasks.workunit.client.0.smithi031.stdout:checking for sys/vfs.h... yes 2024-04-25T01:17:35.319 INFO:tasks.workunit.client.0.smithi031.stdout:checking sys/limits.h usability... no 2024-04-25T01:17:35.381 INFO:tasks.workunit.client.0.smithi031.stdout:checking sys/limits.h presence... no 2024-04-25T01:17:35.381 INFO:tasks.workunit.client.0.smithi031.stdout:checking for sys/limits.h... no 2024-04-25T01:17:35.446 INFO:tasks.workunit.client.0.smithi031.stdout:checking for an ANSI C-conforming const... yes 2024-04-25T01:17:35.518 INFO:tasks.workunit.client.0.smithi031.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-04-25T01:17:35.651 INFO:tasks.workunit.client.0.smithi031.stdout:checking for system... yes 2024-04-25T01:17:35.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:35 smithi169 ceph-mon[32759]: pgmap v440: 97 pgs: 97 active+clean; 5.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 445 KiB/s wr, 68 op/s 2024-04-25T01:17:35.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:35 smithi169 ceph-mon[32759]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:17:35.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:35 smithi040 ceph-mon[32518]: pgmap v440: 97 pgs: 97 active+clean; 5.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 445 KiB/s wr, 68 op/s 2024-04-25T01:17:35.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:35 smithi040 ceph-mon[32518]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:17:35.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:35 smithi031 ceph-mon[28053]: pgmap v440: 97 pgs: 97 active+clean; 5.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 445 KiB/s wr, 68 op/s 2024-04-25T01:17:35.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:35 smithi031 ceph-mon[28053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:17:35.783 INFO:tasks.workunit.client.0.smithi031.stdout:checking for gettimeofday... yes 2024-04-25T01:17:35.904 INFO:tasks.workunit.client.0.smithi031.stdout:checking for mkdir... yes 2024-04-25T01:17:36.030 INFO:tasks.workunit.client.0.smithi031.stdout:checking for strerror... yes 2024-04-25T01:17:36.163 INFO:tasks.workunit.client.0.smithi031.stdout:checking for open64... yes 2024-04-25T01:17:36.292 INFO:tasks.workunit.client.0.smithi031.stdout:checking for stat64... yes 2024-04-25T01:17:36.429 INFO:tasks.workunit.client.0.smithi031.stdout:checking for fseeko64... yes 2024-04-25T01:17:36.567 INFO:tasks.workunit.client.0.smithi031.stdout:checking for lrand48_r... yes 2024-04-25T01:17:36.703 INFO:tasks.workunit.client.0.smithi031.stdout:checking for srand48_r... yes 2024-04-25T01:17:36.756 INFO:tasks.workunit.client.0.smithi031.stdout:configure: creating ./config.status 2024-04-25T01:17:36.928 INFO:tasks.workunit.client.0.smithi031.stdout: /bin/sh ./config.status 2024-04-25T01:17:37.071 INFO:tasks.workunit.client.0.smithi031.stdout:config.status: creating Makefile 2024-04-25T01:17:37.107 INFO:tasks.workunit.client.0.smithi031.stdout:config.status: creating config.h 2024-04-25T01:17:37.152 INFO:tasks.workunit.client.0.smithi031.stdout:config.status: executing depfiles commands 2024-04-25T01:17:37.216 INFO:tasks.workunit.client.0.smithi031.stdout:config.status: executing default commands 2024-04-25T01:17:37.258 INFO:tasks.workunit.client.0.smithi031.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-04-25T01:17:37.515 INFO:tasks.workunit.client.0.smithi031.stdout:rm -f stamp-h1 2024-04-25T01:17:37.519 INFO:tasks.workunit.client.0.smithi031.stdout:touch config.h.in 2024-04-25T01:17:37.523 INFO:tasks.workunit.client.0.smithi031.stdout:cd . && /bin/sh ./config.status config.h 2024-04-25T01:17:37.644 INFO:tasks.workunit.client.0.smithi031.stdout:config.status: creating config.h 2024-04-25T01:17:37.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:37 smithi169 ceph-mon[32759]: pgmap v441: 97 pgs: 97 active+clean; 5.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 313 KiB/s wr, 96 op/s 2024-04-25T01:17:37.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:37 smithi040 ceph-mon[32518]: pgmap v441: 97 pgs: 97 active+clean; 5.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 313 KiB/s wr, 96 op/s 2024-04-25T01:17:37.707 INFO:tasks.workunit.client.0.smithi031.stdout:make all-am 2024-04-25T01:17:37.719 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-04-25T01:17:37.719 INFO:tasks.workunit.client.0.smithi031.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-04-25T01:17:37.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:37 smithi031 ceph-mon[28053]: pgmap v441: 97 pgs: 97 active+clean; 5.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 313 KiB/s wr, 96 op/s 2024-04-25T01:17:38.001 INFO:tasks.workunit.client.0.smithi031.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-04-25T01:17:38.007 INFO:tasks.workunit.client.0.smithi031.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-04-25T01:17:38.170 INFO:tasks.workunit.client.0.smithi031.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-04-25T01:17:38.177 INFO:tasks.workunit.client.0.smithi031.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-04-25T01:17:38.312 INFO:tasks.workunit.client.0.smithi031.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-04-25T01:17:38.318 INFO:tasks.workunit.client.0.smithi031.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-04-25T01:17:38.462 INFO:tasks.workunit.client.0.smithi031.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-04-25T01:17:38.469 INFO:tasks.workunit.client.0.smithi031.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-04-25T01:17:38.660 INFO:tasks.workunit.client.0.smithi031.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-04-25T01:17:38.668 INFO:tasks.workunit.client.0.smithi031.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-04-25T01:17:38.760 INFO:tasks.workunit.client.0.smithi031.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-04-25T01:17:38.767 INFO:tasks.workunit.client.0.smithi031.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-04-25T01:17:38.849 INFO:tasks.workunit.client.0.smithi031.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-04-25T01:17:38.856 INFO:tasks.workunit.client.0.smithi031.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-04-25T01:17:38.944 INFO:tasks.workunit.client.0.smithi031.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-04-25T01:17:38.951 INFO:tasks.workunit.client.0.smithi031.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-04-25T01:17:39.234 INFO:tasks.workunit.client.0.smithi031.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-04-25T01:17:39.241 INFO:tasks.workunit.client.0.smithi031.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-04-25T01:17:39.426 INFO:tasks.workunit.client.0.smithi031.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-04-25T01:17:39.432 INFO:tasks.workunit.client.0.smithi031.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-04-25T01:17:39.638 INFO:tasks.workunit.client.0.smithi031.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-04-25T01:17:39.646 INFO:tasks.workunit.client.0.smithi031.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-04-25T01:17:39.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:39 smithi169 ceph-mon[32759]: pgmap v442: 97 pgs: 97 active+clean; 5.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 343 KiB/s wr, 130 op/s 2024-04-25T01:17:39.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:39 smithi040 ceph-mon[32518]: pgmap v442: 97 pgs: 97 active+clean; 5.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 343 KiB/s wr, 130 op/s 2024-04-25T01:17:39.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:39 smithi031 ceph-mon[28053]: pgmap v442: 97 pgs: 97 active+clean; 5.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 343 KiB/s wr, 130 op/s 2024-04-25T01:17:39.762 INFO:tasks.workunit.client.0.smithi031.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-04-25T01:17:39.769 INFO:tasks.workunit.client.0.smithi031.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-04-25T01:17:39.888 INFO:tasks.workunit.client.0.smithi031.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-04-25T01:17:39.894 INFO:tasks.workunit.client.0.smithi031.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-04-25T01:17:40.056 INFO:tasks.workunit.client.0.smithi031.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-04-25T01:17:40.062 INFO:tasks.workunit.client.0.smithi031.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-04-25T01:17:40.121 INFO:tasks.workunit.client.0.smithi031.stderr:In file included from parser.h:22, 2024-04-25T01:17:40.121 INFO:tasks.workunit.client.0.smithi031.stderr: from parser.c:25: 2024-04-25T01:17:40.122 INFO:tasks.workunit.client.0.smithi031.stderr:parser.c: In function ‘init_filesys’: 2024-04-25T01:17:40.122 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-25T01:17:40.122 INFO:tasks.workunit.client.0.smithi031.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-25T01:17:40.122 INFO:tasks.workunit.client.0.smithi031.stderr: | ^ 2024-04-25T01:17:40.122 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-25T01:17:40.122 INFO:tasks.workunit.client.0.smithi031.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-25T01:17:40.122 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~ 2024-04-25T01:17:40.122 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-25T01:17:40.122 INFO:tasks.workunit.client.0.smithi031.stderr: 36 | container_of(ptr, type, member) 2024-04-25T01:17:40.122 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~~~ 2024-04-25T01:17:40.122 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-25T01:17:40.122 INFO:tasks.workunit.client.0.smithi031.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-25T01:17:40.123 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~ 2024-04-25T01:17:40.123 INFO:tasks.workunit.client.0.smithi031.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-25T01:17:40.123 INFO:tasks.workunit.client.0.smithi031.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-04-25T01:17:40.123 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:17:40.123 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-25T01:17:40.123 INFO:tasks.workunit.client.0.smithi031.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-25T01:17:40.123 INFO:tasks.workunit.client.0.smithi031.stderr: | ^ 2024-04-25T01:17:40.123 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-25T01:17:40.123 INFO:tasks.workunit.client.0.smithi031.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-25T01:17:40.123 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~ 2024-04-25T01:17:40.123 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-25T01:17:40.123 INFO:tasks.workunit.client.0.smithi031.stderr: 36 | container_of(ptr, type, member) 2024-04-25T01:17:40.123 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~~~ 2024-04-25T01:17:40.124 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-25T01:17:40.124 INFO:tasks.workunit.client.0.smithi031.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-25T01:17:40.124 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~ 2024-04-25T01:17:40.124 INFO:tasks.workunit.client.0.smithi031.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-25T01:17:40.124 INFO:tasks.workunit.client.0.smithi031.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-04-25T01:17:40.124 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:17:40.124 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-25T01:17:40.124 INFO:tasks.workunit.client.0.smithi031.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-25T01:17:40.124 INFO:tasks.workunit.client.0.smithi031.stderr: | ^ 2024-04-25T01:17:40.124 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-25T01:17:40.124 INFO:tasks.workunit.client.0.smithi031.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-25T01:17:40.124 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~ 2024-04-25T01:17:40.124 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-25T01:17:40.125 INFO:tasks.workunit.client.0.smithi031.stderr: 36 | container_of(ptr, type, member) 2024-04-25T01:17:40.125 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~~~ 2024-04-25T01:17:40.125 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-25T01:17:40.125 INFO:tasks.workunit.client.0.smithi031.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-25T01:17:40.125 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~ 2024-04-25T01:17:40.125 INFO:tasks.workunit.client.0.smithi031.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-25T01:17:40.125 INFO:tasks.workunit.client.0.smithi031.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-04-25T01:17:40.125 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:17:40.125 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-25T01:17:40.125 INFO:tasks.workunit.client.0.smithi031.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-25T01:17:40.125 INFO:tasks.workunit.client.0.smithi031.stderr: | ^ 2024-04-25T01:17:40.125 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-25T01:17:40.125 INFO:tasks.workunit.client.0.smithi031.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-25T01:17:40.126 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~ 2024-04-25T01:17:40.126 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-25T01:17:40.126 INFO:tasks.workunit.client.0.smithi031.stderr: 36 | container_of(ptr, type, member) 2024-04-25T01:17:40.126 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~~~ 2024-04-25T01:17:40.126 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-25T01:17:40.126 INFO:tasks.workunit.client.0.smithi031.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-25T01:17:40.126 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~ 2024-04-25T01:17:40.126 INFO:tasks.workunit.client.0.smithi031.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-25T01:17:40.126 INFO:tasks.workunit.client.0.smithi031.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-04-25T01:17:40.126 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:17:40.126 INFO:tasks.workunit.client.0.smithi031.stderr:parser.c: In function ‘init_tg_stats’: 2024-04-25T01:17:40.126 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-25T01:17:40.126 INFO:tasks.workunit.client.0.smithi031.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-25T01:17:40.126 INFO:tasks.workunit.client.0.smithi031.stderr: | ^ 2024-04-25T01:17:40.127 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-25T01:17:40.127 INFO:tasks.workunit.client.0.smithi031.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-25T01:17:40.127 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~ 2024-04-25T01:17:40.127 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-25T01:17:40.127 INFO:tasks.workunit.client.0.smithi031.stderr: 36 | container_of(ptr, type, member) 2024-04-25T01:17:40.127 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~~~ 2024-04-25T01:17:40.127 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-25T01:17:40.127 INFO:tasks.workunit.client.0.smithi031.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-25T01:17:40.127 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~ 2024-04-25T01:17:40.127 INFO:tasks.workunit.client.0.smithi031.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-25T01:17:40.127 INFO:tasks.workunit.client.0.smithi031.stderr: 971 | list_for_each_entry(tmp_list, 2024-04-25T01:17:40.127 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:17:40.127 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-25T01:17:40.127 INFO:tasks.workunit.client.0.smithi031.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-25T01:17:40.128 INFO:tasks.workunit.client.0.smithi031.stderr: | ^ 2024-04-25T01:17:40.128 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-25T01:17:40.128 INFO:tasks.workunit.client.0.smithi031.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-25T01:17:40.128 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~ 2024-04-25T01:17:40.128 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-25T01:17:40.128 INFO:tasks.workunit.client.0.smithi031.stderr: 36 | container_of(ptr, type, member) 2024-04-25T01:17:40.128 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~~~ 2024-04-25T01:17:40.128 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-25T01:17:40.128 INFO:tasks.workunit.client.0.smithi031.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-25T01:17:40.128 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~ 2024-04-25T01:17:40.128 INFO:tasks.workunit.client.0.smithi031.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-25T01:17:40.128 INFO:tasks.workunit.client.0.smithi031.stderr: 971 | list_for_each_entry(tmp_list, 2024-04-25T01:17:40.128 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:17:40.129 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-25T01:17:40.129 INFO:tasks.workunit.client.0.smithi031.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-25T01:17:40.129 INFO:tasks.workunit.client.0.smithi031.stderr: | ^ 2024-04-25T01:17:40.129 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-25T01:17:40.129 INFO:tasks.workunit.client.0.smithi031.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-25T01:17:40.129 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~ 2024-04-25T01:17:40.129 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-25T01:17:40.129 INFO:tasks.workunit.client.0.smithi031.stderr: 36 | container_of(ptr, type, member) 2024-04-25T01:17:40.129 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~~~ 2024-04-25T01:17:40.129 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-25T01:17:40.129 INFO:tasks.workunit.client.0.smithi031.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-25T01:17:40.129 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~ 2024-04-25T01:17:40.129 INFO:tasks.workunit.client.0.smithi031.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-25T01:17:40.129 INFO:tasks.workunit.client.0.smithi031.stderr: 980 | list_for_each_entry(tmp_list, 2024-04-25T01:17:40.130 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:17:40.130 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-25T01:17:40.130 INFO:tasks.workunit.client.0.smithi031.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-25T01:17:40.130 INFO:tasks.workunit.client.0.smithi031.stderr: | ^ 2024-04-25T01:17:40.130 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-25T01:17:40.130 INFO:tasks.workunit.client.0.smithi031.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-25T01:17:40.130 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~ 2024-04-25T01:17:40.130 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-25T01:17:40.130 INFO:tasks.workunit.client.0.smithi031.stderr: 36 | container_of(ptr, type, member) 2024-04-25T01:17:40.130 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~~~ 2024-04-25T01:17:40.130 INFO:tasks.workunit.client.0.smithi031.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-25T01:17:40.130 INFO:tasks.workunit.client.0.smithi031.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-25T01:17:40.130 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~ 2024-04-25T01:17:40.130 INFO:tasks.workunit.client.0.smithi031.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-25T01:17:40.131 INFO:tasks.workunit.client.0.smithi031.stderr: 980 | list_for_each_entry(tmp_list, 2024-04-25T01:17:40.131 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:17:40.188 INFO:tasks.workunit.client.0.smithi031.stderr:parser.c: In function ‘get_optstr’: 2024-04-25T01:17:40.188 INFO:tasks.workunit.client.0.smithi031.stderr:parser.c:206:23: warning: ‘strnlen’ specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-04-25T01:17:40.188 INFO:tasks.workunit.client.0.smithi031.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-04-25T01:17:40.188 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:17:40.188 INFO:tasks.workunit.client.0.smithi031.stderr:parser.c:200:14: note: source object declared here 2024-04-25T01:17:40.188 INFO:tasks.workunit.client.0.smithi031.stderr: 200 | char temp[BUFSIZE]; 2024-04-25T01:17:40.188 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~ 2024-04-25T01:17:40.195 INFO:tasks.workunit.client.0.smithi031.stderr:parser.c: In function ‘size64_convert’: 2024-04-25T01:17:40.196 INFO:tasks.workunit.client.0.smithi031.stderr:parser.c:119:9: warning: ‘memcpy’ reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-04-25T01:17:40.196 INFO:tasks.workunit.client.0.smithi031.stderr: 119 | memcpy(unit, "\0", 3); 2024-04-25T01:17:40.197 INFO:tasks.workunit.client.0.smithi031.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:17:40.576 INFO:tasks.workunit.client.0.smithi031.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-04-25T01:17:40.585 INFO:tasks.workunit.client.0.smithi031.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-04-25T01:17:40.686 INFO:tasks.workunit.client.0.smithi031.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-04-25T01:17:40.693 INFO:tasks.workunit.client.0.smithi031.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-04-25T01:17:40.831 INFO:tasks.workunit.client.0.smithi031.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-04-25T01:17:40.839 INFO:tasks.workunit.client.0.smithi031.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-04-25T01:17:40.905 INFO:tasks.workunit.client.0.smithi031.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-04-25T01:17:40.913 INFO:tasks.workunit.client.0.smithi031.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-04-25T01:17:40.987 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-04-25T01:17:40.989 INFO:tasks.workunit.client.0.smithi031.stderr:+ cd .. 2024-04-25T01:17:40.989 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir tmp 2024-04-25T01:17:40.995 INFO:tasks.workunit.client.0.smithi031.stderr:+ cd tmp 2024-04-25T01:17:40.995 INFO:tasks.workunit.client.0.smithi031.stderr:+ for f in $mydir/*.ffsb 2024-04-25T01:17:40.995 INFO:tasks.workunit.client.0.smithi031.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/random_write.32.ffsb 2024-04-25T01:17:40.998 INFO:tasks.workunit.client.0.smithi031.stdout:FFSB version 6.0-RC2 started 2024-04-25T01:17:40.998 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-04-25T01:17:41.007 INFO:tasks.workunit.client.0.smithi031.stdout:benchmark time = 300 2024-04-25T01:17:41.007 INFO:tasks.workunit.client.0.smithi031.stdout:ThreadGroup 0 2024-04-25T01:17:41.007 INFO:tasks.workunit.client.0.smithi031.stdout:================ 2024-04-25T01:17:41.007 INFO:tasks.workunit.client.0.smithi031.stdout: num_threads = 32 2024-04-25T01:17:41.007 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-04-25T01:17:41.007 INFO:tasks.workunit.client.0.smithi031.stdout: read_random = off 2024-04-25T01:17:41.008 INFO:tasks.workunit.client.0.smithi031.stdout: read_size = 0 (0B) 2024-04-25T01:17:41.008 INFO:tasks.workunit.client.0.smithi031.stdout: read_blocksize = 0 (0B) 2024-04-25T01:17:41.008 INFO:tasks.workunit.client.0.smithi031.stdout: read_skip = off 2024-04-25T01:17:41.008 INFO:tasks.workunit.client.0.smithi031.stdout: read_skipsize = 0 (0B) 2024-04-25T01:17:41.008 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-04-25T01:17:41.008 INFO:tasks.workunit.client.0.smithi031.stdout: write_random = on 2024-04-25T01:17:41.008 INFO:tasks.workunit.client.0.smithi031.stdout: write_size = 5242880 (5MB) 2024-04-25T01:17:41.008 INFO:tasks.workunit.client.0.smithi031.stdout: fsync_file = 0 2024-04-25T01:17:41.008 INFO:tasks.workunit.client.0.smithi031.stdout: write_blocksize = 4096 (4KB) 2024-04-25T01:17:41.008 INFO:tasks.workunit.client.0.smithi031.stdout: wait time = 0 2024-04-25T01:17:41.008 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-04-25T01:17:41.009 INFO:tasks.workunit.client.0.smithi031.stdout: op weights 2024-04-25T01:17:41.009 INFO:tasks.workunit.client.0.smithi031.stdout: read = 0 (0.00%) 2024-04-25T01:17:41.009 INFO:tasks.workunit.client.0.smithi031.stdout: readall = 0 (0.00%) 2024-04-25T01:17:41.009 INFO:tasks.workunit.client.0.smithi031.stdout: write = 1 (100.00%) 2024-04-25T01:17:41.009 INFO:tasks.workunit.client.0.smithi031.stdout: create = 0 (0.00%) 2024-04-25T01:17:41.009 INFO:tasks.workunit.client.0.smithi031.stdout: append = 0 (0.00%) 2024-04-25T01:17:41.009 INFO:tasks.workunit.client.0.smithi031.stdout: delete = 0 (0.00%) 2024-04-25T01:17:41.009 INFO:tasks.workunit.client.0.smithi031.stdout: metaop = 0 (0.00%) 2024-04-25T01:17:41.009 INFO:tasks.workunit.client.0.smithi031.stdout: createdir = 0 (0.00%) 2024-04-25T01:17:41.009 INFO:tasks.workunit.client.0.smithi031.stdout: stat = 0 (0.00%) 2024-04-25T01:17:41.010 INFO:tasks.workunit.client.0.smithi031.stdout: writeall = 0 (0.00%) 2024-04-25T01:17:41.010 INFO:tasks.workunit.client.0.smithi031.stdout: writeall_fsync = 0 (0.00%) 2024-04-25T01:17:41.010 INFO:tasks.workunit.client.0.smithi031.stdout: open_close = 0 (0.00%) 2024-04-25T01:17:41.010 INFO:tasks.workunit.client.0.smithi031.stdout: write_fsync = 0 (0.00%) 2024-04-25T01:17:41.010 INFO:tasks.workunit.client.0.smithi031.stdout: create_fsync = 0 (0.00%) 2024-04-25T01:17:41.010 INFO:tasks.workunit.client.0.smithi031.stdout: append_fsync = 0 (0.00%) 2024-04-25T01:17:41.010 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-04-25T01:17:41.010 INFO:tasks.workunit.client.0.smithi031.stdout:FileSystem . 2024-04-25T01:17:41.010 INFO:tasks.workunit.client.0.smithi031.stdout:========== 2024-04-25T01:17:41.010 INFO:tasks.workunit.client.0.smithi031.stdout: num_dirs = 0 2024-04-25T01:17:41.010 INFO:tasks.workunit.client.0.smithi031.stdout: starting files = 128 2024-04-25T01:17:41.010 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-04-25T01:17:41.010 INFO:tasks.workunit.client.0.smithi031.stdout: min file size = 104857600 (100MB) 2024-04-25T01:17:41.011 INFO:tasks.workunit.client.0.smithi031.stdout: max file size = 104857600 (100MB) 2024-04-25T01:17:41.011 INFO:tasks.workunit.client.0.smithi031.stdout: directio = off 2024-04-25T01:17:41.011 INFO:tasks.workunit.client.0.smithi031.stdout: alignedio = on 2024-04-25T01:17:41.011 INFO:tasks.workunit.client.0.smithi031.stdout: bufferedio = off 2024-04-25T01:17:41.011 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-04-25T01:17:41.011 INFO:tasks.workunit.client.0.smithi031.stdout: aging is off 2024-04-25T01:17:41.012 INFO:tasks.workunit.client.0.smithi031.stdout: current utilization = 0.00% 2024-04-25T01:17:41.012 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-04-25T01:17:41.012 INFO:tasks.workunit.client.0.smithi031.stdout:checking existing fs: . 2024-04-25T01:17:41.012 INFO:tasks.workunit.client.0.smithi031.stdout:recreating new fileset 2024-04-25T01:17:41.012 INFO:tasks.workunit.client.0.smithi031.stderr:opendir: No such file or directory 2024-04-25T01:17:41.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:41 smithi169 ceph-mon[32759]: pgmap v443: 97 pgs: 97 active+clean; 7.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 462 KiB/s wr, 141 op/s 2024-04-25T01:17:41.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:41 smithi040 ceph-mon[32518]: pgmap v443: 97 pgs: 97 active+clean; 7.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 462 KiB/s wr, 141 op/s 2024-04-25T01:17:41.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:41 smithi031 ceph-mon[28053]: pgmap v443: 97 pgs: 97 active+clean; 7.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 462 KiB/s wr, 141 op/s 2024-04-25T01:17:43.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:43 smithi169 ceph-mon[32759]: pgmap v444: 97 pgs: 97 active+clean; 7.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 319 KiB/s wr, 134 op/s 2024-04-25T01:17:43.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:43 smithi040 ceph-mon[32518]: pgmap v444: 97 pgs: 97 active+clean; 7.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 319 KiB/s wr, 134 op/s 2024-04-25T01:17:43.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:43 smithi031 ceph-mon[28053]: pgmap v444: 97 pgs: 97 active+clean; 7.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 319 KiB/s wr, 134 op/s 2024-04-25T01:17:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.267+0000 7fc95ac00640 1 -- 172.21.15.31:0/1820560509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc954076100 msgr2=0x7fc954076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.267+0000 7fc95ac00640 1 --2- 172.21.15.31:0/1820560509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc954076100 0x7fc954076520 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc944009900 tx=0x7fc94402a410 comp rx=0 tx=0).stop 2024-04-25T01:17:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.267+0000 7fc95ac00640 1 -- 172.21.15.31:0/1820560509 shutdown_connections 2024-04-25T01:17:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.267+0000 7fc95ac00640 1 --2- 172.21.15.31:0/1820560509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc954077ce0 0x7fc954078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.267+0000 7fc95ac00640 1 --2- 172.21.15.31:0/1820560509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc954077340 0x7fc9540777a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.267+0000 7fc95ac00640 1 --2- 172.21.15.31:0/1820560509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc954076100 0x7fc954076520 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.267+0000 7fc95ac00640 1 -- 172.21.15.31:0/1820560509 >> 172.21.15.31:0/1820560509 conn(0x7fc9540fef30 msgr2=0x7fc954101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.267+0000 7fc95ac00640 1 -- 172.21.15.31:0/1820560509 shutdown_connections 2024-04-25T01:17:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.267+0000 7fc95ac00640 1 -- 172.21.15.31:0/1820560509 wait complete. 2024-04-25T01:17:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.268+0000 7fc95ac00640 1 Processor -- start 2024-04-25T01:17:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.268+0000 7fc95ac00640 1 -- start start 2024-04-25T01:17:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.268+0000 7fc95ac00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc954076100 0x7fc95419e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.268+0000 7fc95ac00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc954077340 0x7fc95419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.268+0000 7fc95ac00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc954077ce0 0x7fc9541a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.268+0000 7fc95ac00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc954113a50 con 0x7fc954077340 2024-04-25T01:17:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.268+0000 7fc95ac00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc9541138d0 con 0x7fc954076100 2024-04-25T01:17:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.268+0000 7fc95ac00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc954113bd0 con 0x7fc954077ce0 2024-04-25T01:17:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.269+0000 7fc953e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc954076100 0x7fc95419e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.269+0000 7fc953e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc954076100 0x7fc95419e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:34712/0 (socket says 172.21.15.31:34712) 2024-04-25T01:17:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.269+0000 7fc953e00640 1 -- 172.21.15.31:0/3150977092 learned_addr learned my addr 172.21.15.31:0/3150977092 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:17:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.269+0000 7fc94be00640 1 --2- 172.21.15.31:0/3150977092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc954077340 0x7fc95419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.269+0000 7fc958c00640 1 --2- 172.21.15.31:0/3150977092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc954077ce0 0x7fc9541a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.269+0000 7fc953e00640 1 -- 172.21.15.31:0/3150977092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc954077ce0 msgr2=0x7fc9541a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.269+0000 7fc953e00640 1 --2- 172.21.15.31:0/3150977092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc954077ce0 0x7fc9541a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.269+0000 7fc953e00640 1 -- 172.21.15.31:0/3150977092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc954077340 msgr2=0x7fc95419e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.269+0000 7fc953e00640 1 --2- 172.21.15.31:0/3150977092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc954077340 0x7fc95419e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.269+0000 7fc953e00640 1 -- 172.21.15.31:0/3150977092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9541a5c10 con 0x7fc954076100 2024-04-25T01:17:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.269+0000 7fc953e00640 1 --2- 172.21.15.31:0/3150977092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc954076100 0x7fc95419e040 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc944009230 tx=0x7fc944031b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.270+0000 7fc951600640 1 -- 172.21.15.31:0/3150977092 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94403f030 con 0x7fc954076100 2024-04-25T01:17:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.270+0000 7fc95ac00640 1 -- 172.21.15.31:0/3150977092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9541a5ea0 con 0x7fc954076100 2024-04-25T01:17:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.270+0000 7fc951600640 1 -- 172.21.15.31:0/3150977092 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc944003ad0 con 0x7fc954076100 2024-04-25T01:17:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.270+0000 7fc951600640 1 -- 172.21.15.31:0/3150977092 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc944035070 con 0x7fc954076100 2024-04-25T01:17:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.270+0000 7fc95ac00640 1 -- 172.21.15.31:0/3150977092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9541a6300 con 0x7fc954076100 2024-04-25T01:17:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.273+0000 7fc951600640 1 -- 172.21.15.31:0/3150977092 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc944042030 con 0x7fc954076100 2024-04-25T01:17:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.273+0000 7fc95d968740 1 Processor -- start 2024-04-25T01:17:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.273+0000 7fc95d968740 1 -- start start 2024-04-25T01:17:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.274+0000 7fc95d968740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bb3c01ca0 0x559bb3c020c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.274+0000 7fc95d968740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb3c02690 0x559bb3c07190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.274+0000 7fc95d968740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559bb3c07800 0x559bb3c09c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.274+0000 7fc95d968740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559bb3b64ab0 con 0x559bb3c02690 2024-04-25T01:17:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.274+0000 7fc95d968740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x559bb3b0d640 con 0x559bb3c07800 2024-04-25T01:17:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.274+0000 7fc95d968740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559bb3b60a70 con 0x559bb3c01ca0 2024-04-25T01:17:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.274+0000 7fc953e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb3c02690 0x559bb3c07190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.274+0000 7fc958c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bb3c01ca0 0x559bb3c020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.274+0000 7fc958c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bb3c01ca0 0x559bb3c020c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:58440/0 (socket says 172.21.15.31:58440) 2024-04-25T01:17:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.274+0000 7fc958c00640 1 -- 172.21.15.31:0/3871670054 learned_addr learned my addr 172.21.15.31:0/3871670054 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:17:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.274+0000 7fc94be00640 1 --2- 172.21.15.31:0/3871670054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559bb3c07800 0x559bb3c09c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.274+0000 7fc958c00640 1 -- 172.21.15.31:0/3871670054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559bb3c07800 msgr2=0x559bb3c09c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.274+0000 7fc958c00640 1 --2- 172.21.15.31:0/3871670054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559bb3c07800 0x559bb3c09c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.274+0000 7fc958c00640 1 -- 172.21.15.31:0/3871670054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb3c02690 msgr2=0x559bb3c07190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.274+0000 7fc958c00640 1 --2- 172.21.15.31:0/3871670054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb3c02690 0x559bb3c07190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.274+0000 7fc958c00640 1 -- 172.21.15.31:0/3871670054 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bb3879210 con 0x559bb3c01ca0 2024-04-25T01:17:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.274+0000 7fc953e00640 1 --2- 172.21.15.31:0/3871670054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb3c02690 0x559bb3c07190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:17:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.275+0000 7fc958c00640 1 --2- 172.21.15.31:0/3871670054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bb3c01ca0 0x559bb3c020c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc94c010370 tx=0x7fc94c010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a1f7eda4b07781 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.275+0000 7fc951600640 1 --2- 172.21.15.31:0/3150977092 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc9380a2c30 0x7fc9380a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.275+0000 7fc949200640 1 -- 172.21.15.31:0/3871670054 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c016070 con 0x559bb3c01ca0 2024-04-25T01:17:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.275+0000 7fc951600640 1 -- 172.21.15.31:0/3150977092 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fc9440ef520 con 0x7fc954076100 2024-04-25T01:17:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.275+0000 7fc949200640 1 -- 172.21.15.31:0/3871670054 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc94c004d20 con 0x559bb3c01ca0 2024-04-25T01:17:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.275+0000 7fc949200640 1 -- 172.21.15.31:0/3871670054 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c005480 con 0x559bb3c01ca0 2024-04-25T01:17:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.275+0000 7fc953e00640 1 --2- 172.21.15.31:0/3150977092 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc9380a2c30 0x7fc9380a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.276+0000 7fc95d968740 1 -- 172.21.15.31:0/3871670054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bb3c01ca0 msgr2=0x559bb3c020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.276+0000 7fc95d968740 1 --2- 172.21.15.31:0/3871670054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bb3c01ca0 0x559bb3c020c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc94c010370 tx=0x7fc94c010830 comp rx=0 tx=0).stop 2024-04-25T01:17:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.276+0000 7fc953e00640 1 --2- 172.21.15.31:0/3150977092 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc9380a2c30 0x7fc9380a50f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc9440bbb20 tx=0x7fc944009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.276+0000 7fc95d968740 1 -- 172.21.15.31:0/3871670054 shutdown_connections 2024-04-25T01:17:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.276+0000 7fc95d968740 1 --2- 172.21.15.31:0/3871670054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559bb3c07800 0x559bb3c09c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.276+0000 7fc95d968740 1 --2- 172.21.15.31:0/3871670054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb3c02690 0x559bb3c07190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.276+0000 7fc95d968740 1 --2- 172.21.15.31:0/3871670054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bb3c01ca0 0x559bb3c020c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.276+0000 7fc95d968740 1 -- 172.21.15.31:0/3871670054 >> 172.21.15.31:0/3871670054 conn(0x559bb3bfd310 msgr2=0x559bb3bfd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.276+0000 7fc95d968740 1 -- 172.21.15.31:0/3871670054 shutdown_connections 2024-04-25T01:17:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.276+0000 7fc95d968740 1 -- 172.21.15.31:0/3871670054 wait complete. 2024-04-25T01:17:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.277+0000 7fc95d968740 1 -- start start 2024-04-25T01:17:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.277+0000 7fc95d968740 1 Processor -- start 2024-04-25T01:17:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.277+0000 7fc95d968740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb3c01ca0 0x559bb3c0bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.277+0000 7fc95d968740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bb3c02690 0x559bb3c0c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.277+0000 7fc95d968740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559bb3c07800 0x559bb3c134c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.277+0000 7fc95d968740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559bb3c0a7b0 con 0x559bb3c01ca0 2024-04-25T01:17:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.277+0000 7fc95d968740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x559bb3b20990 con 0x559bb3c07800 2024-04-25T01:17:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.277+0000 7fc95d968740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559bb3c0a930 con 0x559bb3c02690 2024-04-25T01:17:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.277+0000 7fc95d968740 10 client.0 _ll_register_callbacks cb 0x559bb3b22950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:17:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.278+0000 7fc958c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb3c01ca0 0x559bb3c0bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.278+0000 7fc94be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bb3c02690 0x559bb3c0c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.278+0000 7fc958c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb3c01ca0 0x559bb3c0bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50928/0 (socket says 172.21.15.31:50928) 2024-04-25T01:17:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.278+0000 7fc958c00640 1 -- 172.21.15.31:0/2735621763 learned_addr learned my addr 172.21.15.31:0/2735621763 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:17:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.278+0000 7fc958c00640 1 --2- 172.21.15.31:0/2735621763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559bb3c07800 0x559bb3c134c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.278+0000 7fc94be00640 1 -- 172.21.15.31:0/2735621763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559bb3c07800 msgr2=0x559bb3c134c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.278+0000 7fc94be00640 1 --2- 172.21.15.31:0/2735621763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559bb3c07800 0x559bb3c134c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.278+0000 7fc94be00640 1 -- 172.21.15.31:0/2735621763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb3c01ca0 msgr2=0x559bb3c0bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.278+0000 7fc94be00640 1 --2- 172.21.15.31:0/2735621763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb3c01ca0 0x559bb3c0bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.278+0000 7fc94be00640 1 -- 172.21.15.31:0/2735621763 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bb3945620 con 0x559bb3c02690 2024-04-25T01:17:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.278+0000 7fc94be00640 1 --2- 172.21.15.31:0/2735621763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bb3c02690 0x559bb3c0c480 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc95419f500 tx=0x7fc93c012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1705249c49390401 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.278+0000 7fc941600640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:17:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.279+0000 7fc941600640 1 -- 172.21.15.31:0/2735621763 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c004050 con 0x559bb3c02690 2024-04-25T01:17:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.279+0000 7fc95d968740 1 -- 172.21.15.31:0/2735621763 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc94c004d20 con 0x559bb3c02690 2024-04-25T01:17:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.279+0000 7fc941600640 1 -- 172.21.15.31:0/2735621763 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fc93c0041f0 con 0x559bb3c02690 2024-04-25T01:17:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.279+0000 7fc941600640 1 -- 172.21.15.31:0/2735621763 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc93c0043f0 con 0x559bb3c02690 2024-04-25T01:17:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.279+0000 7fc95d968740 10 client.25363 fetch_fsmap learned FSMap version 25 2024-04-25T01:17:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.279+0000 7fc95d968740 1 -- 172.21.15.31:0/2735621763 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559bb3c165c0 con 0x559bb3c02690 2024-04-25T01:17:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.279+0000 7fc941600640 1 -- 172.21.15.31:0/2735621763 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc94c004d20 con 0x559bb3c02690 2024-04-25T01:17:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.279+0000 7fc941600640 1 -- 172.21.15.31:0/2735621763 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 25) ==== 8342+0+0 (secure 0 0 0) 0x7fc93c003b60 con 0x559bb3c02690 2024-04-25T01:17:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.280+0000 7fc95d968740 10 client.25363 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T01:17:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.280+0000 7fc95d968740 10 client.25363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:17:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.280+0000 7fc95d968740 20 client.25363 populate_metadata read hostname 'smithi031' 2024-04-25T01:17:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.280+0000 7fc95d968740 1 --2- 172.21.15.31:0/2735621763 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x559bb3c17cc0 0x559bb3c1a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.280+0000 7fc95d968740 4 client.25363 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:17:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.280+0000 7fc95d968740 1 -- 172.21.15.31:0/2735621763 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bb3b71ff0 con 0x559bb3c17cc0 2024-04-25T01:17:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.280+0000 7fc958c00640 1 --2- 172.21.15.31:0/2735621763 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x559bb3c17cc0 0x559bb3c1a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.281+0000 7fc958c00640 1 --2- 172.21.15.31:0/2735621763 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x559bb3c17cc0 0x559bb3c1a160 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=399c6851b098dc19 server_cookie=41ed0d125c37476c in_seq=0 out_seq=0 2024-04-25T01:17:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.281+0000 7fc941600640 10 client.25363 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:17:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.281+0000 7fc941600640 1 -- 172.21.15.31:0/2735621763 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559bb3b71ff0 con 0x559bb3c17cc0 2024-04-25T01:17:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.281+0000 7fc941600640 10 client.25363 handle_command_reply: tid=0 2024-04-25T01:17:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.281+0000 7fc95d968740 1 client.25363 shutdown 2024-04-25T01:17:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.282+0000 7fc95d968740 1 -- 172.21.15.31:0/2735621763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bb3c02690 msgr2=0x559bb3c0c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.282+0000 7fc95d968740 1 --2- 172.21.15.31:0/2735621763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bb3c02690 0x559bb3c0c480 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc95419f500 tx=0x7fc93c012b60 comp rx=0 tx=0).stop 2024-04-25T01:17:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.282+0000 7fc95d968740 1 -- 172.21.15.31:0/2735621763 shutdown_connections 2024-04-25T01:17:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.282+0000 7fc95d968740 1 --2- 172.21.15.31:0/2735621763 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x559bb3c17cc0 0x559bb3c1a160 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.282+0000 7fc95d968740 1 --2- 172.21.15.31:0/2735621763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559bb3c07800 0x559bb3c134c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.282+0000 7fc941600640 0 client.25363 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:17:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.282+0000 7fc95d968740 1 --2- 172.21.15.31:0/2735621763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bb3c02690 0x559bb3c0c480 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.282+0000 7fc95d968740 1 --2- 172.21.15.31:0/2735621763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb3c01ca0 0x559bb3c0bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.282+0000 7fc95d968740 1 -- 172.21.15.31:0/2735621763 >> 172.21.15.31:0/2735621763 conn(0x559bb3bfd310 msgr2=0x559bb3bfc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.282+0000 7fc95d968740 1 -- 172.21.15.31:0/2735621763 shutdown_connections 2024-04-25T01:17:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.282+0000 7fc95d968740 1 -- 172.21.15.31:0/2735621763 wait complete. 2024-04-25T01:17:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.282+0000 7fc95d968740 20 client.25363 trim_cache size 0 max 16384 2024-04-25T01:17:45.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.298+0000 7fc95d968740 1 Processor -- start 2024-04-25T01:17:45.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.298+0000 7fc95d968740 1 -- start start 2024-04-25T01:17:45.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.298+0000 7fc95d968740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bb3c01ca0 0x559bb3c0b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:45.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.298+0000 7fc95d968740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559bb3c02690 0x559bb3c13480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.298+0000 7fc95d968740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb3c07800 0x559bb3c00700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.298+0000 7fc95d968740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc9541b18f0 con 0x559bb3c07800 2024-04-25T01:17:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.298+0000 7fc95d968740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc9541af1c0 con 0x559bb3c02690 2024-04-25T01:17:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.298+0000 7fc95d968740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc9541ad010 con 0x559bb3c01ca0 2024-04-25T01:17:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.298+0000 7fc958c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bb3c01ca0 0x559bb3c0b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.298+0000 7fc958c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bb3c01ca0 0x559bb3c0b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:58448/0 (socket says 172.21.15.31:58448) 2024-04-25T01:17:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.298+0000 7fc958c00640 1 -- 172.21.15.31:0/743292573 learned_addr learned my addr 172.21.15.31:0/743292573 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:17:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.298+0000 7fc958c00640 1 --2- 172.21.15.31:0/743292573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb3c07800 0x559bb3c00700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.298+0000 7fc94be00640 1 --2- 172.21.15.31:0/743292573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559bb3c02690 0x559bb3c13480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.299+0000 7fc94be00640 1 -- 172.21.15.31:0/743292573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bb3c01ca0 msgr2=0x559bb3c0b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.299+0000 7fc94be00640 1 --2- 172.21.15.31:0/743292573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bb3c01ca0 0x559bb3c0b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.299+0000 7fc94be00640 1 -- 172.21.15.31:0/743292573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb3c07800 msgr2=0x559bb3c00700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.299+0000 7fc94be00640 1 --2- 172.21.15.31:0/743292573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb3c07800 0x559bb3c00700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.299+0000 7fc94be00640 1 -- 172.21.15.31:0/743292573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bb3bfd140 con 0x559bb3c02690 2024-04-25T01:17:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.299+0000 7fc94be00640 1 --2- 172.21.15.31:0/743292573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559bb3c02690 0x559bb3c13480 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc93c010480 tx=0x7fc93c010880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9eaa83bb1edcf2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.299+0000 7fc943e00640 1 -- 172.21.15.31:0/743292573 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c003fd0 con 0x559bb3c02690 2024-04-25T01:17:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.299+0000 7fc943e00640 1 -- 172.21.15.31:0/743292573 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc93c004170 con 0x559bb3c02690 2024-04-25T01:17:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.299+0000 7fc943e00640 1 -- 172.21.15.31:0/743292573 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c00f790 con 0x559bb3c02690 2024-04-25T01:17:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.300+0000 7fc95d968740 1 -- 172.21.15.31:0/743292573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559bb3c02690 msgr2=0x559bb3c13480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.300+0000 7fc95d968740 1 --2- 172.21.15.31:0/743292573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559bb3c02690 0x559bb3c13480 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc93c010480 tx=0x7fc93c010880 comp rx=0 tx=0).stop 2024-04-25T01:17:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.300+0000 7fc95d968740 1 -- 172.21.15.31:0/743292573 shutdown_connections 2024-04-25T01:17:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.300+0000 7fc95d968740 1 --2- 172.21.15.31:0/743292573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb3c07800 0x559bb3c00700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.300+0000 7fc95d968740 1 --2- 172.21.15.31:0/743292573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559bb3c02690 0x559bb3c13480 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.300+0000 7fc95d968740 1 --2- 172.21.15.31:0/743292573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bb3c01ca0 0x559bb3c0b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.300+0000 7fc95d968740 1 -- 172.21.15.31:0/743292573 >> 172.21.15.31:0/743292573 conn(0x559bb3bfd310 msgr2=0x559bb3bffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.300+0000 7fc95d968740 1 -- 172.21.15.31:0/743292573 shutdown_connections 2024-04-25T01:17:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.300+0000 7fc95d968740 1 -- 172.21.15.31:0/743292573 wait complete. 2024-04-25T01:17:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.301+0000 7fc95d968740 1 -- start start 2024-04-25T01:17:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.301+0000 7fc95d968740 1 Processor -- start 2024-04-25T01:17:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.301+0000 7fc95d968740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb3c01ca0 0x559bb3c11190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.301+0000 7fc95d968740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559bb3c02690 0x559bb3c116d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.301+0000 7fc95d968740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bb3c07800 0x559bb3c126e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.301+0000 7fc95d968740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc9541ad680 con 0x559bb3c01ca0 2024-04-25T01:17:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.301+0000 7fc95d968740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc9541aeaf0 con 0x559bb3c02690 2024-04-25T01:17:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.301+0000 7fc95d968740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc9541add50 con 0x559bb3c07800 2024-04-25T01:17:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.301+0000 7fc95d968740 10 client.0 _ll_register_callbacks cb 0x7fc93c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:17:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.302+0000 7fc94be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559bb3c02690 0x559bb3c116d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.302+0000 7fc958c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb3c01ca0 0x559bb3c11190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.302+0000 7fc94be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559bb3c02690 0x559bb3c116d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:34742/0 (socket says 172.21.15.31:34742) 2024-04-25T01:17:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.302+0000 7fc958c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb3c01ca0 0x559bb3c11190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50944/0 (socket says 172.21.15.31:50944) 2024-04-25T01:17:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.302+0000 7fc94be00640 1 -- 172.21.15.31:0/1707639731 learned_addr learned my addr 172.21.15.31:0/1707639731 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:17:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.302+0000 7fc958c00640 1 --2- 172.21.15.31:0/1707639731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bb3c07800 0x559bb3c126e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.302+0000 7fc94be00640 1 -- 172.21.15.31:0/1707639731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bb3c07800 msgr2=0x559bb3c126e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.302+0000 7fc94be00640 1 --2- 172.21.15.31:0/1707639731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bb3c07800 0x559bb3c126e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.302+0000 7fc94be00640 1 -- 172.21.15.31:0/1707639731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb3c01ca0 msgr2=0x559bb3c11190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.302+0000 7fc94be00640 1 --2- 172.21.15.31:0/1707639731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb3c01ca0 0x559bb3c11190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.302+0000 7fc94be00640 1 -- 172.21.15.31:0/1707639731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bb3c12cd0 con 0x559bb3c02690 2024-04-25T01:17:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.302+0000 7fc958c00640 1 --2- 172.21.15.31:0/1707639731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bb3c07800 0x559bb3c126e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:17:45.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.302+0000 7fc94be00640 1 --2- 172.21.15.31:0/1707639731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559bb3c02690 0x559bb3c116d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0105f0 tx=0x7fc93c005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fe45e9e146fde47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:45.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.302+0000 7fc941600640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T01:17:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.303+0000 7fc941600640 1 -- 172.21.15.31:0/1707639731 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c003fd0 con 0x559bb3c02690 2024-04-25T01:17:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.303+0000 7fc95d968740 1 -- 172.21.15.31:0/1707639731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc93c004170 con 0x559bb3c02690 2024-04-25T01:17:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.303+0000 7fc941600640 1 -- 172.21.15.31:0/1707639731 <== mon.2 v2:172.21.15.40:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fc93c00f990 con 0x559bb3c02690 2024-04-25T01:17:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.303+0000 7fc941600640 1 -- 172.21.15.31:0/1707639731 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc93c00fb90 con 0x559bb3c02690 2024-04-25T01:17:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.303+0000 7fc95d968740 10 client.25358 fetch_fsmap learned FSMap version 25 2024-04-25T01:17:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.303+0000 7fc95d968740 1 -- 172.21.15.31:0/1707639731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559bb3c7e8d0 con 0x559bb3c02690 2024-04-25T01:17:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.303+0000 7fc941600640 1 -- 172.21.15.31:0/1707639731 <== mon.2 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc93c004170 con 0x559bb3c02690 2024-04-25T01:17:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.303+0000 7fc941600640 1 -- 172.21.15.31:0/1707639731 <== mon.2 v2:172.21.15.40:3300/0 5 ==== fsmap(e 25) ==== 8342+0+0 (secure 0 0 0) 0x7fc93c01c420 con 0x559bb3c02690 2024-04-25T01:17:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.304+0000 7fc95d968740 10 client.25358 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T01:17:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.304+0000 7fc95d968740 10 client.25358 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:17:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.304+0000 7fc95d968740 20 client.25358 populate_metadata read hostname 'smithi031' 2024-04-25T01:17:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.304+0000 7fc95d968740 1 --2- 172.21.15.31:0/1707639731 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x559bb3c7f240 0x559bb3c81660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.304+0000 7fc95d968740 4 client.25358 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:17:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.304+0000 7fc95d968740 1 -- 172.21.15.31:0/1707639731 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559bb3b6c340 con 0x559bb3c7f240 2024-04-25T01:17:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.304+0000 7fc958c00640 1 --2- 172.21.15.31:0/1707639731 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x559bb3c7f240 0x559bb3c81660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.304+0000 7fc958c00640 1 --2- 172.21.15.31:0/1707639731 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x559bb3c7f240 0x559bb3c81660 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1582a80613bffb73 server_cookie=62ad7a5acbc35b1f in_seq=0 out_seq=0 2024-04-25T01:17:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.304+0000 7fc941600640 10 client.25358 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:17:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.362+0000 7fc941600640 1 -- 172.21.15.31:0/1707639731 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559bb3b6c340 con 0x559bb3c7f240 2024-04-25T01:17:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.362+0000 7fc941600640 10 client.25358 handle_command_reply: tid=0 2024-04-25T01:17:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.362+0000 7fc95d968740 1 client.25358 shutdown 2024-04-25T01:17:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.363+0000 7fc95d968740 1 -- 172.21.15.31:0/1707639731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559bb3c02690 msgr2=0x559bb3c116d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.363+0000 7fc95d968740 1 --2- 172.21.15.31:0/1707639731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559bb3c02690 0x559bb3c116d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0105f0 tx=0x7fc93c005970 comp rx=0 tx=0).stop 2024-04-25T01:17:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.363+0000 7fc95d968740 1 -- 172.21.15.31:0/1707639731 shutdown_connections 2024-04-25T01:17:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.363+0000 7fc95d968740 1 --2- 172.21.15.31:0/1707639731 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x559bb3c7f240 0x559bb3c81660 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.363+0000 7fc95d968740 1 --2- 172.21.15.31:0/1707639731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bb3c07800 0x559bb3c126e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.363+0000 7fc95d968740 1 --2- 172.21.15.31:0/1707639731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559bb3c02690 0x559bb3c116d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.363+0000 7fc95d968740 1 --2- 172.21.15.31:0/1707639731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb3c01ca0 0x559bb3c11190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.363+0000 7fc95d968740 1 -- 172.21.15.31:0/1707639731 >> 172.21.15.31:0/1707639731 conn(0x559bb3bfd310 msgr2=0x559bb3c00880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.363+0000 7fc941600640 0 client.25358 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:17:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.363+0000 7fc95d968740 1 -- 172.21.15.31:0/1707639731 shutdown_connections 2024-04-25T01:17:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.363+0000 7fc95d968740 1 -- 172.21.15.31:0/1707639731 wait complete. 2024-04-25T01:17:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.363+0000 7fc95d968740 20 client.25358 trim_cache size 0 max 16384 2024-04-25T01:17:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:17:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:17:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:17:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:17:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.364+0000 7fc95ac00640 1 -- 172.21.15.31:0/3150977092 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc9380a2c30 msgr2=0x7fc9380a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.364+0000 7fc95ac00640 1 --2- 172.21.15.31:0/3150977092 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc9380a2c30 0x7fc9380a50f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc9440bbb20 tx=0x7fc944009260 comp rx=0 tx=0).stop 2024-04-25T01:17:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.364+0000 7fc95ac00640 1 -- 172.21.15.31:0/3150977092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc954076100 msgr2=0x7fc95419e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.364+0000 7fc95ac00640 1 --2- 172.21.15.31:0/3150977092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc954076100 0x7fc95419e040 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc944009230 tx=0x7fc944031b60 comp rx=0 tx=0).stop 2024-04-25T01:17:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.364+0000 7fc95ac00640 1 -- 172.21.15.31:0/3150977092 shutdown_connections 2024-04-25T01:17:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.364+0000 7fc95ac00640 1 --2- 172.21.15.31:0/3150977092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc954077ce0 0x7fc9541a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.364+0000 7fc95ac00640 1 --2- 172.21.15.31:0/3150977092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc954077340 0x7fc95419e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.364+0000 7fc95ac00640 1 --2- 172.21.15.31:0/3150977092 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc9380a2c30 0x7fc9380a50f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.364+0000 7fc95ac00640 1 --2- 172.21.15.31:0/3150977092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc954076100 0x7fc95419e040 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.364+0000 7fc95ac00640 1 -- 172.21.15.31:0/3150977092 >> 172.21.15.31:0/3150977092 conn(0x7fc9540fef30 msgr2=0x7fc954100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.364+0000 7fc95ac00640 1 -- 172.21.15.31:0/3150977092 shutdown_connections 2024-04-25T01:17:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:45.364+0000 7fc95ac00640 1 -- 172.21.15.31:0/3150977092 wait complete. 2024-04-25T01:17:45.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:45 smithi169 ceph-mon[32759]: pgmap v445: 97 pgs: 97 active+clean; 7.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 319 KiB/s wr, 135 op/s 2024-04-25T01:17:45.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:45 smithi040 ceph-mon[32518]: pgmap v445: 97 pgs: 97 active+clean; 7.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 319 KiB/s wr, 135 op/s 2024-04-25T01:17:45.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:45 smithi031 ceph-mon[28053]: pgmap v445: 97 pgs: 97 active+clean; 7.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 319 KiB/s wr, 135 op/s 2024-04-25T01:17:47.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:47 smithi169 ceph-mon[32759]: pgmap v446: 97 pgs: 97 active+clean; 7.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 291 KiB/s wr, 118 op/s 2024-04-25T01:17:47.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:47 smithi040 ceph-mon[32518]: pgmap v446: 97 pgs: 97 active+clean; 7.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 291 KiB/s wr, 118 op/s 2024-04-25T01:17:47.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:47 smithi031 ceph-mon[28053]: pgmap v446: 97 pgs: 97 active+clean; 7.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 291 KiB/s wr, 118 op/s 2024-04-25T01:17:49.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:49 smithi169 ceph-mon[32759]: pgmap v447: 97 pgs: 97 active+clean; 44 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 3.3 MiB/s wr, 101 op/s 2024-04-25T01:17:49.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:49 smithi040 ceph-mon[32518]: pgmap v447: 97 pgs: 97 active+clean; 44 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 3.3 MiB/s wr, 101 op/s 2024-04-25T01:17:49.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:49 smithi031 ceph-mon[28053]: pgmap v447: 97 pgs: 97 active+clean; 44 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 3.3 MiB/s wr, 101 op/s 2024-04-25T01:17:51.694 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:51 smithi169 ceph-mon[32759]: pgmap v448: 97 pgs: 97 active+clean; 177 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 15 MiB/s wr, 89 op/s 2024-04-25T01:17:51.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:51 smithi040 ceph-mon[32518]: pgmap v448: 97 pgs: 97 active+clean; 177 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 15 MiB/s wr, 89 op/s 2024-04-25T01:17:51.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:51 smithi031 ceph-mon[28053]: pgmap v448: 97 pgs: 97 active+clean; 177 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 15 MiB/s wr, 89 op/s 2024-04-25T01:17:53.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:53 smithi169 ceph-mon[32759]: pgmap v449: 97 pgs: 97 active+clean; 501 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 42 MiB/s wr, 77 op/s 2024-04-25T01:17:53.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:53 smithi040 ceph-mon[32518]: pgmap v449: 97 pgs: 97 active+clean; 501 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 42 MiB/s wr, 77 op/s 2024-04-25T01:17:53.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:53 smithi031 ceph-mon[28053]: pgmap v449: 97 pgs: 97 active+clean; 501 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 42 MiB/s wr, 77 op/s 2024-04-25T01:17:55.693 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:55 smithi169 ceph-mon[32759]: pgmap v450: 97 pgs: 97 active+clean; 849 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 71 MiB/s wr, 82 op/s 2024-04-25T01:17:55.696 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:55 smithi040 ceph-mon[32518]: pgmap v450: 97 pgs: 97 active+clean; 849 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 71 MiB/s wr, 82 op/s 2024-04-25T01:17:55.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:55 smithi031 ceph-mon[28053]: pgmap v450: 97 pgs: 97 active+clean; 849 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 71 MiB/s wr, 82 op/s 2024-04-25T01:17:57.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:57 smithi169 ceph-mon[32759]: pgmap v451: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 105 MiB/s wr, 100 op/s 2024-04-25T01:17:57.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:57 smithi040 ceph-mon[32518]: pgmap v451: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 105 MiB/s wr, 100 op/s 2024-04-25T01:17:57.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:57 smithi031 ceph-mon[28053]: pgmap v451: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 105 MiB/s wr, 100 op/s 2024-04-25T01:17:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.575+0000 7fdc76c00640 1 -- 172.21.15.31:0/743198150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc701031b0 msgr2=0x7fdc70109a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.575+0000 7fdc76c00640 1 --2- 172.21.15.31:0/743198150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc701031b0 0x7fdc70109a40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009be0 tx=0x7fdc6802a6d0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.576+0000 7fdc76c00640 1 -- 172.21.15.31:0/743198150 shutdown_connections 2024-04-25T01:17:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.576+0000 7fdc76c00640 1 --2- 172.21.15.31:0/743198150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc701031b0 0x7fdc70109a40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.576+0000 7fdc76c00640 1 --2- 172.21.15.31:0/743198150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc701027f0 0x7fdc70102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.576+0000 7fdc76c00640 1 --2- 172.21.15.31:0/743198150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc70101600 0x7fdc70101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.576+0000 7fdc76c00640 1 -- 172.21.15.31:0/743198150 >> 172.21.15.31:0/743198150 conn(0x7fdc700fcb70 msgr2=0x7fdc700fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.576+0000 7fdc76c00640 1 -- 172.21.15.31:0/743198150 shutdown_connections 2024-04-25T01:17:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.576+0000 7fdc76c00640 1 -- 172.21.15.31:0/743198150 wait complete. 2024-04-25T01:17:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.578+0000 7fdc76c00640 1 Processor -- start 2024-04-25T01:17:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.578+0000 7fdc76c00640 1 -- start start 2024-04-25T01:17:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.579+0000 7fdc76c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc70101600 0x7fdc701a72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.579+0000 7fdc76c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc701027f0 0x7fdc701a7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.579+0000 7fdc76c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc701031b0 0x7fdc701a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.579+0000 7fdc76c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7010d260 con 0x7fdc701031b0 2024-04-25T01:17:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.579+0000 7fdc76c00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7010d0e0 con 0x7fdc70101600 2024-04-25T01:17:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.579+0000 7fdc76c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7010d3e0 con 0x7fdc701027f0 2024-04-25T01:17:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.579+0000 7fdc75800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc70101600 0x7fdc701a72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.579+0000 7fdc75800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc70101600 0x7fdc701a72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:60934/0 (socket says 172.21.15.31:60934) 2024-04-25T01:17:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.580+0000 7fdc75800640 1 -- 172.21.15.31:0/1685892214 learned_addr learned my addr 172.21.15.31:0/1685892214 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:17:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.580+0000 7fdc74e00640 1 --2- 172.21.15.31:0/1685892214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc701027f0 0x7fdc701a7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.580+0000 7fdc76200640 1 --2- 172.21.15.31:0/1685892214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc701031b0 0x7fdc701a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.580+0000 7fdc75800640 1 -- 172.21.15.31:0/1685892214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc701027f0 msgr2=0x7fdc701a7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.580+0000 7fdc75800640 1 --2- 172.21.15.31:0/1685892214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc701027f0 0x7fdc701a7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.580+0000 7fdc75800640 1 -- 172.21.15.31:0/1685892214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc701031b0 msgr2=0x7fdc701a3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.580+0000 7fdc75800640 1 --2- 172.21.15.31:0/1685892214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc701031b0 0x7fdc701a3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.580+0000 7fdc75800640 1 -- 172.21.15.31:0/1685892214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc701a46f0 con 0x7fdc70101600 2024-04-25T01:17:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.580+0000 7fdc76200640 1 --2- 172.21.15.31:0/1685892214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc701031b0 0x7fdc701a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:17:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.581+0000 7fdc75800640 1 --2- 172.21.15.31:0/1685892214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc70101600 0x7fdc701a72c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000e9f0 tx=0x7fdc6000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.581+0000 7fdc6e000640 1 -- 172.21.15.31:0/1685892214 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6000ccc0 con 0x7fdc70101600 2024-04-25T01:17:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.581+0000 7fdc6e000640 1 -- 172.21.15.31:0/1685892214 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc600035f0 con 0x7fdc70101600 2024-04-25T01:17:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.581+0000 7fdc6e000640 1 -- 172.21.15.31:0/1685892214 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6000ce60 con 0x7fdc70101600 2024-04-25T01:17:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.581+0000 7fdc76c00640 1 -- 172.21.15.31:0/1685892214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc70073cd0 con 0x7fdc70101600 2024-04-25T01:17:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.582+0000 7fdc76c00640 1 -- 172.21.15.31:0/1685892214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc700741d0 con 0x7fdc70101600 2024-04-25T01:17:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.587+0000 7fdc7b001740 1 Processor -- start 2024-04-25T01:17:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.587+0000 7fdc7b001740 1 -- start start 2024-04-25T01:17:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.587+0000 7fdc7b001740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bd9e50 0x5621e1bda270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.587+0000 7fdc7b001740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621e1bda840 0x5621e1bdf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.587+0000 7fdc7b001740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621e1bdf9b0 0x5621e1be1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.587+0000 7fdc7b001740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5621e1b3cab0 con 0x5621e1bdf9b0 2024-04-25T01:17:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.587+0000 7fdc7b001740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5621e1ae5640 con 0x5621e1bda840 2024-04-25T01:17:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.587+0000 7fdc7b001740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5621e1b38a70 con 0x5621e1bd9e50 2024-04-25T01:17:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.588+0000 7fdc74e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621e1bdf9b0 0x5621e1be1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.588+0000 7fdc76200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bd9e50 0x5621e1bda270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.588+0000 7fdc76200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bd9e50 0x5621e1bda270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:55092/0 (socket says 172.21.15.31:55092) 2024-04-25T01:17:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.588+0000 7fdc76200640 1 -- 172.21.15.31:0/2646851946 learned_addr learned my addr 172.21.15.31:0/2646851946 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:17:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.588+0000 7fdc74e00640 1 -- 172.21.15.31:0/2646851946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bd9e50 msgr2=0x5621e1bda270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.588+0000 7fdc74e00640 1 --2- 172.21.15.31:0/2646851946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bd9e50 0x5621e1bda270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.588+0000 7fdc74e00640 1 -- 172.21.15.31:0/2646851946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621e1bda840 msgr2=0x5621e1bdf340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:17:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.588+0000 7fdc74e00640 1 --2- 172.21.15.31:0/2646851946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621e1bda840 0x5621e1bdf340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.588+0000 7fdc74e00640 1 -- 172.21.15.31:0/2646851946 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621e1851210 con 0x5621e1bdf9b0 2024-04-25T01:17:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.589+0000 7fdc74e00640 1 --2- 172.21.15.31:0/2646851946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621e1bdf9b0 0x5621e1be1e40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdc6400bbc0 tx=0x7fdc640107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f973eb0c6e71b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.589+0000 7fdc5ea00640 1 -- 172.21.15.31:0/2646851946 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc64004030 con 0x5621e1bdf9b0 2024-04-25T01:17:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.589+0000 7fdc5ea00640 1 -- 172.21.15.31:0/2646851946 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc640041d0 con 0x5621e1bdf9b0 2024-04-25T01:17:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.589+0000 7fdc5ea00640 1 -- 172.21.15.31:0/2646851946 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc640116a0 con 0x5621e1bdf9b0 2024-04-25T01:17:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.590+0000 7fdc7b001740 1 -- 172.21.15.31:0/2646851946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621e1bdf9b0 msgr2=0x5621e1be1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.590+0000 7fdc7b001740 1 --2- 172.21.15.31:0/2646851946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621e1bdf9b0 0x5621e1be1e40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdc6400bbc0 tx=0x7fdc640107a0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.591+0000 7fdc7b001740 1 -- 172.21.15.31:0/2646851946 shutdown_connections 2024-04-25T01:17:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.591+0000 7fdc7b001740 1 --2- 172.21.15.31:0/2646851946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621e1bdf9b0 0x5621e1be1e40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.591+0000 7fdc7b001740 1 --2- 172.21.15.31:0/2646851946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621e1bda840 0x5621e1bdf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.591+0000 7fdc7b001740 1 --2- 172.21.15.31:0/2646851946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bd9e50 0x5621e1bda270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.591+0000 7fdc7b001740 1 -- 172.21.15.31:0/2646851946 >> 172.21.15.31:0/2646851946 conn(0x5621e1bd54c0 msgr2=0x5621e1bd58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.602+0000 7fdc6e000640 1 -- 172.21.15.31:0/1685892214 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdc600037a0 con 0x7fdc70101600 2024-04-25T01:17:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.602+0000 7fdc7b001740 1 -- 172.21.15.31:0/2646851946 shutdown_connections 2024-04-25T01:17:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.602+0000 7fdc7b001740 1 -- 172.21.15.31:0/2646851946 wait complete. 2024-04-25T01:17:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.603+0000 7fdc7b001740 1 -- start start 2024-04-25T01:17:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.603+0000 7fdc7b001740 1 Processor -- start 2024-04-25T01:17:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.604+0000 7fdc7b001740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621e1bd9e50 0x5621e1be4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.604+0000 7fdc7b001740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bdf9b0 0x5621e1be4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.604+0000 7fdc7b001740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621e1be97b0 0x5621e1bebba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.604+0000 7fdc7b001740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5621e1be2960 con 0x5621e1bd9e50 2024-04-25T01:17:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.604+0000 7fdc7b001740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5621e1af8990 con 0x5621e1be97b0 2024-04-25T01:17:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.604+0000 7fdc7b001740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5621e1be2ae0 con 0x5621e1bdf9b0 2024-04-25T01:17:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.604+0000 7fdc7b001740 10 client.0 _ll_register_callbacks cb 0x5621e1afa950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:17:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.604+0000 7fdc6e000640 1 --2- 172.21.15.31:0/1685892214 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fdc580a2b10 0x7fdc580a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.604+0000 7fdc6e000640 1 -- 172.21.15.31:0/1685892214 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fdc600cf230 con 0x7fdc70101600 2024-04-25T01:17:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.605+0000 7fdc76200640 1 --2- 172.21.15.31:0/1685892214 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fdc580a2b10 0x7fdc580a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.605+0000 7fdc75800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bdf9b0 0x5621e1be4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.605+0000 7fdc75800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bdf9b0 0x5621e1be4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:55104/0 (socket says 172.21.15.31:55104) 2024-04-25T01:17:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.605+0000 7fdc75800640 1 -- 172.21.15.31:0/3826325668 learned_addr learned my addr 172.21.15.31:0/3826325668 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:17:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.605+0000 7fdc76200640 1 --2- 172.21.15.31:0/3826325668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621e1bd9e50 0x5621e1be4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.605+0000 7fdc74e00640 1 --2- 172.21.15.31:0/3826325668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621e1be97b0 0x5621e1bebba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.605+0000 7fdc75800640 1 -- 172.21.15.31:0/3826325668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621e1be97b0 msgr2=0x5621e1bebba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.605+0000 7fdc75800640 1 --2- 172.21.15.31:0/3826325668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621e1be97b0 0x5621e1bebba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.605+0000 7fdc75800640 1 -- 172.21.15.31:0/3826325668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621e1bd9e50 msgr2=0x5621e1be4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.605+0000 7fdc75800640 1 --2- 172.21.15.31:0/3826325668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621e1bd9e50 0x5621e1be4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.605+0000 7fdc75800640 1 -- 172.21.15.31:0/3826325668 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621e191d620 con 0x5621e1bdf9b0 2024-04-25T01:17:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.605+0000 7fdc74e00640 1 --2- 172.21.15.31:0/3826325668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621e1be97b0 0x5621e1bebba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:17:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.606+0000 7fdc76200640 1 --2- 172.21.15.31:0/3826325668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621e1bd9e50 0x5621e1be4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:17:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.606+0000 7fdc75800640 1 --2- 172.21.15.31:0/3826325668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bdf9b0 0x5621e1be4780 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdc6009bab0 tx=0x7fdc6009c220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2060394e992460b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.606+0000 7fdc53400640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:17:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.606+0000 7fdc53400640 1 -- 172.21.15.31:0/3826325668 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6009fed0 con 0x5621e1bdf9b0 2024-04-25T01:17:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.606+0000 7fdc7b001740 1 -- 172.21.15.31:0/3826325668 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc640041d0 con 0x5621e1bdf9b0 2024-04-25T01:17:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.606+0000 7fdc53400640 1 -- 172.21.15.31:0/3826325668 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fdc60099e20 con 0x5621e1bdf9b0 2024-04-25T01:17:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.607+0000 7fdc53400640 1 -- 172.21.15.31:0/3826325668 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdc600a4790 con 0x5621e1bdf9b0 2024-04-25T01:17:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.607+0000 7fdc7b001740 10 client.25381 fetch_fsmap learned FSMap version 25 2024-04-25T01:17:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.607+0000 7fdc7b001740 1 -- 172.21.15.31:0/3826325668 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5621e1beec40 con 0x5621e1bdf9b0 2024-04-25T01:17:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.607+0000 7fdc53400640 1 -- 172.21.15.31:0/3826325668 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc640041d0 con 0x5621e1bdf9b0 2024-04-25T01:17:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.610+0000 7fdc76200640 1 --2- 172.21.15.31:0/1685892214 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fdc580a2b10 0x7fdc580a4fd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdc68030960 tx=0x7fdc68033040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.612+0000 7fdc53400640 1 -- 172.21.15.31:0/3826325668 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 25) ==== 8342+0+0 (secure 0 0 0) 0x7fdc600b7290 con 0x5621e1bdf9b0 2024-04-25T01:17:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.612+0000 7fdc7b001740 10 client.25381 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T01:17:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.612+0000 7fdc7b001740 10 client.25381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:17:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.612+0000 7fdc7b001740 20 client.25381 populate_metadata read hostname 'smithi031' 2024-04-25T01:17:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.612+0000 7fdc7b001740 1 --2- 172.21.15.31:0/3826325668 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5621e1bf02c0 0x5621e1bf26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.612+0000 7fdc7b001740 4 client.25381 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:17:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.612+0000 7fdc7b001740 1 -- 172.21.15.31:0/3826325668 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621e1b49ff0 con 0x5621e1bf02c0 2024-04-25T01:17:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.612+0000 7fdc74e00640 1 --2- 172.21.15.31:0/3826325668 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5621e1bf02c0 0x5621e1bf26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.613+0000 7fdc53400640 10 client.25381 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:17:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.613+0000 7fdc74e00640 1 --2- 172.21.15.31:0/3826325668 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5621e1bf02c0 0x5621e1bf26f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2a7e2e1cda4a3e6 server_cookie=964de9dbd1339350 in_seq=0 out_seq=0 2024-04-25T01:17:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.614+0000 7fdc53400640 1 -- 172.21.15.31:0/3826325668 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5621e1b49ff0 con 0x5621e1bf02c0 2024-04-25T01:17:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.614+0000 7fdc53400640 10 client.25381 handle_command_reply: tid=0 2024-04-25T01:17:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.616+0000 7fdc7b001740 1 client.25381 shutdown 2024-04-25T01:17:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.616+0000 7fdc7b001740 1 -- 172.21.15.31:0/3826325668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bdf9b0 msgr2=0x5621e1be4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.616+0000 7fdc7b001740 1 --2- 172.21.15.31:0/3826325668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bdf9b0 0x5621e1be4780 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdc6009bab0 tx=0x7fdc6009c220 comp rx=0 tx=0).stop 2024-04-25T01:17:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.616+0000 7fdc7b001740 1 -- 172.21.15.31:0/3826325668 shutdown_connections 2024-04-25T01:17:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.616+0000 7fdc7b001740 1 --2- 172.21.15.31:0/3826325668 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5621e1bf02c0 0x5621e1bf26f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.616+0000 7fdc7b001740 1 --2- 172.21.15.31:0/3826325668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621e1be97b0 0x5621e1bebba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.616+0000 7fdc7b001740 1 --2- 172.21.15.31:0/3826325668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bdf9b0 0x5621e1be4780 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.616+0000 7fdc7b001740 1 --2- 172.21.15.31:0/3826325668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621e1bd9e50 0x5621e1be4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.616+0000 7fdc53400640 0 client.25381 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:17:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.616+0000 7fdc7b001740 1 -- 172.21.15.31:0/3826325668 >> 172.21.15.31:0/3826325668 conn(0x5621e1bd54c0 msgr2=0x5621e1bd4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.617+0000 7fdc7b001740 1 -- 172.21.15.31:0/3826325668 shutdown_connections 2024-04-25T01:17:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.617+0000 7fdc7b001740 1 -- 172.21.15.31:0/3826325668 wait complete. 2024-04-25T01:17:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.617+0000 7fdc7b001740 20 client.25381 trim_cache size 0 max 16384 2024-04-25T01:17:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.658+0000 7fdc7b001740 1 Processor -- start 2024-04-25T01:17:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.661+0000 7fdc7b001740 1 -- start start 2024-04-25T01:17:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.661+0000 7fdc7b001740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621e1bd54c0 0x5621e1bd83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.662+0000 7fdc7b001740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bd9e50 0x5621e1bd8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.662+0000 7fdc7b001740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621e1bdf9b0 0x5621e1bf6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.662+0000 7fdc7b001740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701c2850 con 0x5621e1bdf9b0 2024-04-25T01:17:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.662+0000 7fdc7b001740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701af110 con 0x5621e1bd54c0 2024-04-25T01:17:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.662+0000 7fdc7b001740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701acf60 con 0x5621e1bd9e50 2024-04-25T01:17:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.662+0000 7fdc74e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bd9e50 0x5621e1bd8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.662+0000 7fdc74e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bd9e50 0x5621e1bd8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:55110/0 (socket says 172.21.15.31:55110) 2024-04-25T01:17:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.662+0000 7fdc74e00640 1 -- 172.21.15.31:0/268439491 learned_addr learned my addr 172.21.15.31:0/268439491 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:17:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.662+0000 7fdc76200640 1 --2- 172.21.15.31:0/268439491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621e1bdf9b0 0x5621e1bf6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.662+0000 7fdc74e00640 1 -- 172.21.15.31:0/268439491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621e1bd54c0 msgr2=0x5621e1bd83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.662+0000 7fdc75800640 1 --2- 172.21.15.31:0/268439491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621e1bd54c0 0x5621e1bd83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.662+0000 7fdc74e00640 1 --2- 172.21.15.31:0/268439491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621e1bd54c0 0x5621e1bd83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.662+0000 7fdc74e00640 1 -- 172.21.15.31:0/268439491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621e1bdf9b0 msgr2=0x5621e1bf6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.662+0000 7fdc74e00640 1 --2- 172.21.15.31:0/268439491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621e1bdf9b0 0x5621e1bf6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.662+0000 7fdc74e00640 1 -- 172.21.15.31:0/268439491 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621e1bd52f0 con 0x5621e1bd9e50 2024-04-25T01:17:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.662+0000 7fdc75800640 1 --2- 172.21.15.31:0/268439491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621e1bd54c0 0x5621e1bd83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:17:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.663+0000 7fdc76200640 1 --2- 172.21.15.31:0/268439491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621e1bdf9b0 0x5621e1bf6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:17:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.663+0000 7fdc74e00640 1 --2- 172.21.15.31:0/268439491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bd9e50 0x5621e1bd8910 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdc640058c0 tx=0x7fdc64003b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e3b819fd0890af9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.663+0000 7fdc5e000640 1 -- 172.21.15.31:0/268439491 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc64003df0 con 0x5621e1bd9e50 2024-04-25T01:17:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.663+0000 7fdc5e000640 1 -- 172.21.15.31:0/268439491 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc64005160 con 0x5621e1bd9e50 2024-04-25T01:17:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.663+0000 7fdc5e000640 1 -- 172.21.15.31:0/268439491 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc64028450 con 0x5621e1bd9e50 2024-04-25T01:17:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.664+0000 7fdc7b001740 1 -- 172.21.15.31:0/268439491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bd9e50 msgr2=0x5621e1bd8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.664+0000 7fdc7b001740 1 --2- 172.21.15.31:0/268439491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bd9e50 0x5621e1bd8910 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdc640058c0 tx=0x7fdc64003b80 comp rx=0 tx=0).stop 2024-04-25T01:17:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.664+0000 7fdc7b001740 1 -- 172.21.15.31:0/268439491 shutdown_connections 2024-04-25T01:17:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.664+0000 7fdc7b001740 1 --2- 172.21.15.31:0/268439491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621e1bdf9b0 0x5621e1bf6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.664+0000 7fdc7b001740 1 --2- 172.21.15.31:0/268439491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bd9e50 0x5621e1bd8910 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.664+0000 7fdc7b001740 1 --2- 172.21.15.31:0/268439491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621e1bd54c0 0x5621e1bd83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.664+0000 7fdc7b001740 1 -- 172.21.15.31:0/268439491 >> 172.21.15.31:0/268439491 conn(0x5621e1bdb440 msgr2=0x5621e1bdbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.665+0000 7fdc7b001740 1 -- 172.21.15.31:0/268439491 shutdown_connections 2024-04-25T01:17:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.665+0000 7fdc7b001740 1 -- 172.21.15.31:0/268439491 wait complete. 2024-04-25T01:17:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.665+0000 7fdc7b001740 1 -- start start 2024-04-25T01:17:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.665+0000 7fdc7b001740 1 Processor -- start 2024-04-25T01:17:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.666+0000 7fdc7b001740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bd54c0 0x5621e1c034e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.666+0000 7fdc7b001740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621e1bd9e50 0x5621e1c03a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.666+0000 7fdc7b001740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621e1bdf9b0 0x5621e1c01b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.667+0000 7fdc74e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621e1bd9e50 0x5621e1c03a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.667+0000 7fdc74e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621e1bd9e50 0x5621e1c03a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:60966/0 (socket says 172.21.15.31:60966) 2024-04-25T01:17:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.667+0000 7fdc75800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bd54c0 0x5621e1c034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.667+0000 7fdc75800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bd54c0 0x5621e1c034e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:55114/0 (socket says 172.21.15.31:55114) 2024-04-25T01:17:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.667+0000 7fdc76200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621e1bdf9b0 0x5621e1c01b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.667+0000 7fdc76200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621e1bdf9b0 0x5621e1c01b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51634/0 (socket says 172.21.15.31:51634) 2024-04-25T01:17:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.667+0000 7fdc75800640 1 -- 172.21.15.31:0/2714540742 learned_addr learned my addr 172.21.15.31:0/2714540742 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:17:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.668+0000 7fdc7b001740 1 -- 172.21.15.31:0/2714540742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701ad5d0 con 0x5621e1bdf9b0 2024-04-25T01:17:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.668+0000 7fdc7b001740 1 -- 172.21.15.31:0/2714540742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701aea40 con 0x5621e1bd9e50 2024-04-25T01:17:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.668+0000 7fdc7b001740 1 -- 172.21.15.31:0/2714540742 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701adca0 con 0x5621e1bd54c0 2024-04-25T01:17:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.668+0000 7fdc7b001740 10 client.0 _ll_register_callbacks cb 0x7fdc60003e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:17:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.668+0000 7fdc74e00640 1 -- 172.21.15.31:0/2714540742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bd54c0 msgr2=0x5621e1c034e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.668+0000 7fdc74e00640 1 --2- 172.21.15.31:0/2714540742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bd54c0 0x5621e1c034e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.668+0000 7fdc74e00640 1 -- 172.21.15.31:0/2714540742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621e1bdf9b0 msgr2=0x5621e1c01b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:17:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.668+0000 7fdc74e00640 1 --2- 172.21.15.31:0/2714540742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621e1bdf9b0 0x5621e1c01b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.668+0000 7fdc74e00640 1 -- 172.21.15.31:0/2714540742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621e1c02390 con 0x5621e1bd9e50 2024-04-25T01:17:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.668+0000 7fdc75800640 1 --2- 172.21.15.31:0/2714540742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bd54c0 0x5621e1c034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:17:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.669+0000 7fdc74e00640 1 --2- 172.21.15.31:0/2714540742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621e1bd9e50 0x5621e1c03a20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdc64005890 tx=0x7fdc6400bdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6100fbe50b30aaf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:17:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.669+0000 7fdc53400640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T01:17:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.669+0000 7fdc53400640 1 -- 172.21.15.31:0/2714540742 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc64005310 con 0x5621e1bd9e50 2024-04-25T01:17:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.669+0000 7fdc7b001740 1 -- 172.21.15.31:0/2714540742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc64005160 con 0x5621e1bd9e50 2024-04-25T01:17:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.670+0000 7fdc53400640 1 -- 172.21.15.31:0/2714540742 <== mon.2 v2:172.21.15.40:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fdc6402e040 con 0x5621e1bd9e50 2024-04-25T01:17:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.670+0000 7fdc53400640 1 -- 172.21.15.31:0/2714540742 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdc640054b0 con 0x5621e1bd9e50 2024-04-25T01:17:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.670+0000 7fdc7b001740 10 client.25367 fetch_fsmap learned FSMap version 25 2024-04-25T01:17:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.670+0000 7fdc7b001740 1 -- 172.21.15.31:0/2714540742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5621e1c56d90 con 0x5621e1bd9e50 2024-04-25T01:17:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.670+0000 7fdc53400640 1 -- 172.21.15.31:0/2714540742 <== mon.2 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc64005160 con 0x5621e1bd9e50 2024-04-25T01:17:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.674+0000 7fdc53400640 1 -- 172.21.15.31:0/2714540742 <== mon.2 v2:172.21.15.40:3300/0 5 ==== fsmap(e 25) ==== 8342+0+0 (secure 0 0 0) 0x7fdc64028790 con 0x5621e1bd9e50 2024-04-25T01:17:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.674+0000 7fdc7b001740 10 client.25367 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T01:17:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.674+0000 7fdc7b001740 10 client.25367 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:17:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.674+0000 7fdc7b001740 20 client.25367 populate_metadata read hostname 'smithi031' 2024-04-25T01:17:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.674+0000 7fdc7b001740 1 --2- 172.21.15.31:0/2714540742 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5621e1be97b0 0x5621e1c594b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:17:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.674+0000 7fdc7b001740 4 client.25367 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:17:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.674+0000 7fdc7b001740 1 -- 172.21.15.31:0/2714540742 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5621e1b44340 con 0x5621e1be97b0 2024-04-25T01:17:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.675+0000 7fdc75800640 1 --2- 172.21.15.31:0/2714540742 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5621e1be97b0 0x5621e1c594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:17:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.678+0000 7fdc75800640 1 --2- 172.21.15.31:0/2714540742 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5621e1be97b0 0x5621e1c594b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cee417a2862b75e server_cookie=f18406305f586564 in_seq=0 out_seq=0 2024-04-25T01:17:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.678+0000 7fdc53400640 10 client.25367 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:17:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.700+0000 7fdc53400640 1 -- 172.21.15.31:0/2714540742 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5621e1b44340 con 0x5621e1be97b0 2024-04-25T01:17:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.700+0000 7fdc53400640 10 client.25367 handle_command_reply: tid=0 2024-04-25T01:17:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.702+0000 7fdc7b001740 1 client.25367 shutdown 2024-04-25T01:17:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.702+0000 7fdc7b001740 1 -- 172.21.15.31:0/2714540742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621e1bd9e50 msgr2=0x5621e1c03a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.702+0000 7fdc7b001740 1 --2- 172.21.15.31:0/2714540742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621e1bd9e50 0x5621e1c03a20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdc64005890 tx=0x7fdc6400bdf0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.702+0000 7fdc7b001740 1 -- 172.21.15.31:0/2714540742 shutdown_connections 2024-04-25T01:17:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.702+0000 7fdc7b001740 1 --2- 172.21.15.31:0/2714540742 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5621e1be97b0 0x5621e1c594b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.702+0000 7fdc7b001740 1 --2- 172.21.15.31:0/2714540742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5621e1bdf9b0 0x5621e1c01b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.702+0000 7fdc7b001740 1 --2- 172.21.15.31:0/2714540742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5621e1bd9e50 0x5621e1c03a20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.702+0000 7fdc7b001740 1 --2- 172.21.15.31:0/2714540742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5621e1bd54c0 0x5621e1c034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.702+0000 7fdc7b001740 1 -- 172.21.15.31:0/2714540742 >> 172.21.15.31:0/2714540742 conn(0x5621e1bdb440 msgr2=0x5621e1bd83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.702+0000 7fdc53400640 0 client.25367 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:17:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.702+0000 7fdc7b001740 1 -- 172.21.15.31:0/2714540742 shutdown_connections 2024-04-25T01:17:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.702+0000 7fdc7b001740 1 -- 172.21.15.31:0/2714540742 wait complete. 2024-04-25T01:17:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.703+0000 7fdc7b001740 20 client.25367 trim_cache size 0 max 16384 2024-04-25T01:17:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:17:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:17:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:17:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:17:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.704+0000 7fdc76c00640 1 -- 172.21.15.31:0/1685892214 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fdc580a2b10 msgr2=0x7fdc580a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.704+0000 7fdc76c00640 1 --2- 172.21.15.31:0/1685892214 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fdc580a2b10 0x7fdc580a4fd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdc68030960 tx=0x7fdc68033040 comp rx=0 tx=0).stop 2024-04-25T01:17:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.704+0000 7fdc76c00640 1 -- 172.21.15.31:0/1685892214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc70101600 msgr2=0x7fdc701a72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:17:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.704+0000 7fdc76c00640 1 --2- 172.21.15.31:0/1685892214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc70101600 0x7fdc701a72c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000e9f0 tx=0x7fdc6000eeb0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.705+0000 7fdc76c00640 1 -- 172.21.15.31:0/1685892214 shutdown_connections 2024-04-25T01:17:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.705+0000 7fdc76c00640 1 --2- 172.21.15.31:0/1685892214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc701031b0 0x7fdc701a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.705+0000 7fdc76c00640 1 --2- 172.21.15.31:0/1685892214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc701027f0 0x7fdc701a7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.705+0000 7fdc76c00640 1 --2- 172.21.15.31:0/1685892214 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fdc580a2b10 0x7fdc580a4fd0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.705+0000 7fdc76c00640 1 --2- 172.21.15.31:0/1685892214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc70101600 0x7fdc701a72c0 secure :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000e9f0 tx=0x7fdc6000eeb0 comp rx=0 tx=0).stop 2024-04-25T01:17:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.705+0000 7fdc76c00640 1 -- 172.21.15.31:0/1685892214 >> 172.21.15.31:0/1685892214 conn(0x7fdc700fcb70 msgr2=0x7fdc70105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:17:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.706+0000 7fdc76c00640 1 -- 172.21.15.31:0/1685892214 shutdown_connections 2024-04-25T01:17:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:17:58.706+0000 7fdc76c00640 1 -- 172.21.15.31:0/1685892214 wait complete. 2024-04-25T01:17:59.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:17:59 smithi169 ceph-mon[32759]: pgmap v452: 97 pgs: 97 active+clean; 1.7 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 146 MiB/s wr, 100 op/s 2024-04-25T01:17:59.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:17:59 smithi040 ceph-mon[32518]: pgmap v452: 97 pgs: 97 active+clean; 1.7 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 146 MiB/s wr, 100 op/s 2024-04-25T01:17:59.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:17:59 smithi031 ceph-mon[28053]: pgmap v452: 97 pgs: 97 active+clean; 1.7 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 146 MiB/s wr, 100 op/s 2024-04-25T01:18:01.692 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:01 smithi169 ceph-mon[32759]: pgmap v453: 97 pgs: 97 active+clean; 2.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 188 MiB/s wr, 105 op/s 2024-04-25T01:18:01.695 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:01 smithi040 ceph-mon[32518]: pgmap v453: 97 pgs: 97 active+clean; 2.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 188 MiB/s wr, 105 op/s 2024-04-25T01:18:01.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:01 smithi031 ceph-mon[28053]: pgmap v453: 97 pgs: 97 active+clean; 2.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 188 MiB/s wr, 105 op/s 2024-04-25T01:18:03.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:03 smithi031 ceph-mon[28053]: pgmap v454: 97 pgs: 97 active+clean; 2.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 229 MiB/s wr, 88 op/s 2024-04-25T01:18:03.942 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:03 smithi169 ceph-mon[32759]: pgmap v454: 97 pgs: 97 active+clean; 2.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 229 MiB/s wr, 88 op/s 2024-04-25T01:18:03.945 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:03 smithi040 ceph-mon[32518]: pgmap v454: 97 pgs: 97 active+clean; 2.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 229 MiB/s wr, 88 op/s 2024-04-25T01:18:05.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:05 smithi031 ceph-mon[28053]: pgmap v455: 97 pgs: 97 active+clean; 3.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 226 MiB/s wr, 86 op/s 2024-04-25T01:18:05.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:05 smithi169 ceph-mon[32759]: pgmap v455: 97 pgs: 97 active+clean; 3.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 226 MiB/s wr, 86 op/s 2024-04-25T01:18:05.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:05 smithi040 ceph-mon[32518]: pgmap v455: 97 pgs: 97 active+clean; 3.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 226 MiB/s wr, 86 op/s 2024-04-25T01:18:07.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:07 smithi031 ceph-mon[28053]: pgmap v456: 97 pgs: 97 active+clean; 3.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 237 MiB/s wr, 99 op/s 2024-04-25T01:18:07.942 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:07 smithi169 ceph-mon[32759]: pgmap v456: 97 pgs: 97 active+clean; 3.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 237 MiB/s wr, 99 op/s 2024-04-25T01:18:07.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:07 smithi040 ceph-mon[32518]: pgmap v456: 97 pgs: 97 active+clean; 3.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 237 MiB/s wr, 99 op/s 2024-04-25T01:18:08.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:08 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:18:08.942 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:08 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:18:08.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:08 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:18:09.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:09 smithi031 ceph-mon[28053]: pgmap v457: 97 pgs: 97 active+clean; 4.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 237 MiB/s wr, 87 op/s 2024-04-25T01:18:09.942 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:09 smithi169 ceph-mon[32759]: pgmap v457: 97 pgs: 97 active+clean; 4.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 237 MiB/s wr, 87 op/s 2024-04-25T01:18:09.945 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:09 smithi040 ceph-mon[32518]: pgmap v457: 97 pgs: 97 active+clean; 4.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 237 MiB/s wr, 87 op/s 2024-04-25T01:18:10.942 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:10 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:18:10.942 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:10 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:18:10.942 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:10 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:18:10.945 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:10 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:18:10.945 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:10 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:18:10.945 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:10 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:18:11.003 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:10 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:18:11.003 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:10 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:18:11.003 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:10 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:18:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.954+0000 7fa02a800640 1 -- 172.21.15.31:0/2499643678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0241027f0 msgr2=0x7fa024102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.954+0000 7fa02a800640 1 --2- 172.21.15.31:0/2499643678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0241027f0 0x7fa024102c70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa014009900 tx=0x7fa01402a3f0 comp rx=0 tx=0).stop 2024-04-25T01:18:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.954+0000 7fa02a800640 1 -- 172.21.15.31:0/2499643678 shutdown_connections 2024-04-25T01:18:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.954+0000 7fa02a800640 1 --2- 172.21.15.31:0/2499643678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0241031b0 0x7fa024109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.954+0000 7fa02a800640 1 --2- 172.21.15.31:0/2499643678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0241027f0 0x7fa024102c70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.954+0000 7fa02a800640 1 --2- 172.21.15.31:0/2499643678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa024101600 0x7fa024101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.955+0000 7fa02a800640 1 -- 172.21.15.31:0/2499643678 >> 172.21.15.31:0/2499643678 conn(0x7fa0240fcb70 msgr2=0x7fa0240fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:18:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.955+0000 7fa02a800640 1 -- 172.21.15.31:0/2499643678 shutdown_connections 2024-04-25T01:18:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.955+0000 7fa02a800640 1 -- 172.21.15.31:0/2499643678 wait complete. 2024-04-25T01:18:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.956+0000 7fa02a800640 1 Processor -- start 2024-04-25T01:18:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.956+0000 7fa02a800640 1 -- start start 2024-04-25T01:18:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.956+0000 7fa02a800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa024101600 0x7fa0241109e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.956+0000 7fa02a800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0241027f0 0x7fa024110f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.957+0000 7fa02a800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0241031b0 0x7fa0241adec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.957+0000 7fa028a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0241027f0 0x7fa024110f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.957+0000 7fa029e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0241031b0 0x7fa0241adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.957+0000 7fa028a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0241027f0 0x7fa024110f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49314/0 (socket says 172.21.15.31:49314) 2024-04-25T01:18:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.957+0000 7fa029e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0241031b0 0x7fa0241adec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:58676/0 (socket says 172.21.15.31:58676) 2024-04-25T01:18:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.957+0000 7fa029400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa024101600 0x7fa0241109e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.957+0000 7fa029400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa024101600 0x7fa0241109e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:38216/0 (socket says 172.21.15.31:38216) 2024-04-25T01:18:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.957+0000 7fa02a800640 1 -- 172.21.15.31:0/904416877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa0240756d0 con 0x7fa0241027f0 2024-04-25T01:18:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.957+0000 7fa028a00640 1 -- 172.21.15.31:0/904416877 learned_addr learned my addr 172.21.15.31:0/904416877 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:18:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.957+0000 7fa02a800640 1 -- 172.21.15.31:0/904416877 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa024075550 con 0x7fa0241031b0 2024-04-25T01:18:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.957+0000 7fa02a800640 1 -- 172.21.15.31:0/904416877 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa024075850 con 0x7fa024101600 2024-04-25T01:18:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.958+0000 7fa029400640 1 -- 172.21.15.31:0/904416877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0241031b0 msgr2=0x7fa0241adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.958+0000 7fa029400640 1 --2- 172.21.15.31:0/904416877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0241031b0 0x7fa0241adec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.958+0000 7fa029400640 1 -- 172.21.15.31:0/904416877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0241027f0 msgr2=0x7fa024110f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.958+0000 7fa029400640 1 --2- 172.21.15.31:0/904416877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0241027f0 0x7fa024110f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.958+0000 7fa029400640 1 -- 172.21.15.31:0/904416877 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0241ae400 con 0x7fa024101600 2024-04-25T01:18:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.958+0000 7fa028a00640 1 --2- 172.21.15.31:0/904416877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0241027f0 0x7fa024110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:18:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.958+0000 7fa029400640 1 --2- 172.21.15.31:0/904416877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa024101600 0x7fa0241109e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa01800c940 tx=0x7fa01800ce00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:18:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.958+0000 7fa022000640 1 -- 172.21.15.31:0/904416877 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa018004220 con 0x7fa024101600 2024-04-25T01:18:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.959+0000 7fa02a800640 1 -- 172.21.15.31:0/904416877 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0241ae630 con 0x7fa024101600 2024-04-25T01:18:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.959+0000 7fa02a800640 1 -- 172.21.15.31:0/904416877 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0241aeb80 con 0x7fa024101600 2024-04-25T01:18:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.959+0000 7fa022000640 1 -- 172.21.15.31:0/904416877 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0180043c0 con 0x7fa024101600 2024-04-25T01:18:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.959+0000 7fa022000640 1 -- 172.21.15.31:0/904416877 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa018004a00 con 0x7fa024101600 2024-04-25T01:18:11.943 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:11 smithi169 ceph-mon[32759]: pgmap v458: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 241 MiB/s wr, 99 op/s 2024-04-25T01:18:11.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:11 smithi040 ceph-mon[32518]: pgmap v458: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 241 MiB/s wr, 99 op/s 2024-04-25T01:18:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.962+0000 7fa02ec22740 1 Processor -- start 2024-04-25T01:18:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.963+0000 7fa02ec22740 1 -- start start 2024-04-25T01:18:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.964+0000 7fa02ec22740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5630fb0efe50 0x5630fb0f0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.964+0000 7fa02ec22740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630fb0f0840 0x5630fb0f5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.964+0000 7fa02ec22740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0f59b0 0x5630fb0f7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.964+0000 7fa029e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5630fb0efe50 0x5630fb0f0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.964+0000 7fa029e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5630fb0efe50 0x5630fb0f0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:58684/0 (socket says 172.21.15.31:58684) 2024-04-25T01:18:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.964+0000 7fa02ec22740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5630fb052ab0 con 0x5630fb0f0840 2024-04-25T01:18:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.964+0000 7fa02ec22740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5630faffb640 con 0x5630fb0efe50 2024-04-25T01:18:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.964+0000 7fa02ec22740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5630fb04ea70 con 0x5630fb0f59b0 2024-04-25T01:18:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.964+0000 7fa029e00640 1 -- 172.21.15.31:0/1460845968 learned_addr learned my addr 172.21.15.31:0/1460845968 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:18:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.965+0000 7fa028a00640 1 --2- 172.21.15.31:0/1460845968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0f59b0 0x5630fb0f7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.965+0000 7fa029e00640 1 -- 172.21.15.31:0/1460845968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0f59b0 msgr2=0x5630fb0f7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.965+0000 7fa029e00640 1 --2- 172.21.15.31:0/1460845968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0f59b0 0x5630fb0f7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.965+0000 7fa029e00640 1 -- 172.21.15.31:0/1460845968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630fb0f0840 msgr2=0x5630fb0f5340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:18:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.965+0000 7fa029e00640 1 --2- 172.21.15.31:0/1460845968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630fb0f0840 0x5630fb0f5340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.965+0000 7fa029e00640 1 -- 172.21.15.31:0/1460845968 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630fad67210 con 0x5630fb0efe50 2024-04-25T01:18:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.965+0000 7fa028a00640 1 --2- 172.21.15.31:0/1460845968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0f59b0 0x5630fb0f7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:18:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.965+0000 7fa029e00640 1 --2- 172.21.15.31:0/1460845968 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5630fb0efe50 0x5630fb0f0270 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00ae80 tx=0x7fa01c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a7753c0365a13f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:18:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.966+0000 7fa012600640 1 -- 172.21.15.31:0/1460845968 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01c004d20 con 0x5630fb0efe50 2024-04-25T01:18:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.966+0000 7fa012600640 1 -- 172.21.15.31:0/1460845968 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa01c010b20 con 0x5630fb0efe50 2024-04-25T01:18:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.966+0000 7fa012600640 1 -- 172.21.15.31:0/1460845968 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01c005790 con 0x5630fb0efe50 2024-04-25T01:18:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.966+0000 7fa022000640 1 -- 172.21.15.31:0/904416877 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa018004ba0 con 0x7fa024101600 2024-04-25T01:18:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.967+0000 7fa02ec22740 1 -- 172.21.15.31:0/1460845968 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5630fb0efe50 msgr2=0x5630fb0f0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.967+0000 7fa02ec22740 1 --2- 172.21.15.31:0/1460845968 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5630fb0efe50 0x5630fb0f0270 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00ae80 tx=0x7fa01c00f790 comp rx=0 tx=0).stop 2024-04-25T01:18:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.967+0000 7fa02ec22740 1 -- 172.21.15.31:0/1460845968 shutdown_connections 2024-04-25T01:18:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.967+0000 7fa02ec22740 1 --2- 172.21.15.31:0/1460845968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0f59b0 0x5630fb0f7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.967+0000 7fa02ec22740 1 --2- 172.21.15.31:0/1460845968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630fb0f0840 0x5630fb0f5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.967+0000 7fa02ec22740 1 --2- 172.21.15.31:0/1460845968 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5630fb0efe50 0x5630fb0f0270 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.967+0000 7fa02ec22740 1 -- 172.21.15.31:0/1460845968 >> 172.21.15.31:0/1460845968 conn(0x5630fb0eb4c0 msgr2=0x5630fb0eb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:18:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.968+0000 7fa02ec22740 1 -- 172.21.15.31:0/1460845968 shutdown_connections 2024-04-25T01:18:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.968+0000 7fa02ec22740 1 -- 172.21.15.31:0/1460845968 wait complete. 2024-04-25T01:18:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.968+0000 7fa022000640 1 --2- 172.21.15.31:0/904416877 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa00c0a2b10 0x7fa00c0a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.968+0000 7fa022000640 1 -- 172.21.15.31:0/904416877 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fa0180ceb20 con 0x7fa024101600 2024-04-25T01:18:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.968+0000 7fa02ec22740 1 -- start start 2024-04-25T01:18:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.969+0000 7fa02ec22740 1 Processor -- start 2024-04-25T01:18:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.969+0000 7fa028a00640 1 --2- 172.21.15.31:0/904416877 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa00c0a2b10 0x7fa00c0a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.970+0000 7fa02ec22740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0efe50 0x5630fb0fa110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.970+0000 7fa02ec22740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5630fb0f0840 0x5630fb0fa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.970+0000 7fa02ec22740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630fb0f59b0 0x5630fb1016b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.970+0000 7fa02ec22740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5630fb0f8960 con 0x5630fb0f59b0 2024-04-25T01:18:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.970+0000 7fa02ec22740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5630fb00e990 con 0x5630fb0f0840 2024-04-25T01:18:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.970+0000 7fa02ec22740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5630fb0f8ae0 con 0x5630fb0efe50 2024-04-25T01:18:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.970+0000 7fa02ec22740 10 client.0 _ll_register_callbacks cb 0x5630fb010950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:18:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.970+0000 7fa029400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0efe50 0x5630fb0fa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.970+0000 7fa029400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0efe50 0x5630fb0fa110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:38240/0 (socket says 172.21.15.31:38240) 2024-04-25T01:18:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.970+0000 7fa029400640 1 -- 172.21.15.31:0/1180962664 learned_addr learned my addr 172.21.15.31:0/1180962664 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:18:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.970+0000 7fa029e00640 1 --2- 172.21.15.31:0/1180962664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630fb0f59b0 0x5630fb1016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.970+0000 7fa029400640 1 -- 172.21.15.31:0/1180962664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5630fb0f0840 msgr2=0x5630fb0fa650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T01:18:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.970+0000 7fa029400640 1 --2- 172.21.15.31:0/1180962664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5630fb0f0840 0x5630fb0fa650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.970+0000 7fa029400640 1 -- 172.21.15.31:0/1180962664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630fb0f59b0 msgr2=0x5630fb1016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.970+0000 7fa029400640 1 --2- 172.21.15.31:0/1180962664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630fb0f59b0 0x5630fb1016b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.971+0000 7fa029400640 1 -- 172.21.15.31:0/1180962664 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630fae33620 con 0x5630fb0efe50 2024-04-25T01:18:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.971+0000 7fa029400640 1 --2- 172.21.15.31:0/1180962664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0efe50 0x5630fb0fa110 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa018097920 tx=0x7fa01809b380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e27f67a1aadf53a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:18:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.971+0000 7fa006a00640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:18:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.971+0000 7fa006a00640 1 -- 172.21.15.31:0/1180962664 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01801d040 con 0x5630fb0efe50 2024-04-25T01:18:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.971+0000 7fa006a00640 1 -- 172.21.15.31:0/1180962664 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fa01809a910 con 0x5630fb0efe50 2024-04-25T01:18:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.971+0000 7fa02ec22740 1 -- 172.21.15.31:0/1180962664 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa01c010b20 con 0x5630fb0efe50 2024-04-25T01:18:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.971+0000 7fa028a00640 1 --2- 172.21.15.31:0/904416877 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa00c0a2b10 0x7fa00c0a4fd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa024111ef0 tx=0x7fa0140315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:18:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.971+0000 7fa006a00640 1 -- 172.21.15.31:0/1180962664 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa01809f7c0 con 0x5630fb0efe50 2024-04-25T01:18:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.971+0000 7fa02ec22740 10 client.25399 fetch_fsmap learned FSMap version 25 2024-04-25T01:18:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.972+0000 7fa02ec22740 1 -- 172.21.15.31:0/1180962664 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5630fb104760 con 0x5630fb0efe50 2024-04-25T01:18:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.972+0000 7fa006a00640 1 -- 172.21.15.31:0/1180962664 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa01c010b20 con 0x5630fb0efe50 2024-04-25T01:18:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.972+0000 7fa006a00640 1 -- 172.21.15.31:0/1180962664 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 25) ==== 8342+0+0 (secure 0 0 0) 0x7fa0180b41e0 con 0x5630fb0efe50 2024-04-25T01:18:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.972+0000 7fa02ec22740 10 client.25399 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T01:18:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.972+0000 7fa02ec22740 10 client.25399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:18:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.972+0000 7fa02ec22740 20 client.25399 populate_metadata read hostname 'smithi031' 2024-04-25T01:18:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.972+0000 7fa02ec22740 1 --2- 172.21.15.31:0/1180962664 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5630fb105de0 0x5630fb108200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.972+0000 7fa02ec22740 4 client.25399 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:18:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.972+0000 7fa02ec22740 1 -- 172.21.15.31:0/1180962664 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630fb05fff0 con 0x5630fb105de0 2024-04-25T01:18:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.972+0000 7fa028a00640 1 --2- 172.21.15.31:0/1180962664 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5630fb105de0 0x5630fb108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.973+0000 7fa028a00640 1 --2- 172.21.15.31:0/1180962664 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5630fb105de0 0x5630fb108200 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=353808ea0ec30b87 server_cookie=8e7db37f83327f0f in_seq=0 out_seq=0 2024-04-25T01:18:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.973+0000 7fa006a00640 10 client.25399 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:18:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.974+0000 7fa006a00640 1 -- 172.21.15.31:0/1180962664 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5630fb05fff0 con 0x5630fb105de0 2024-04-25T01:18:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.974+0000 7fa006a00640 10 client.25399 handle_command_reply: tid=0 2024-04-25T01:18:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.974+0000 7fa02ec22740 1 client.25399 shutdown 2024-04-25T01:18:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.974+0000 7fa02ec22740 1 -- 172.21.15.31:0/1180962664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0efe50 msgr2=0x5630fb0fa110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.974+0000 7fa02ec22740 1 --2- 172.21.15.31:0/1180962664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0efe50 0x5630fb0fa110 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa018097920 tx=0x7fa01809b380 comp rx=0 tx=0).stop 2024-04-25T01:18:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.974+0000 7fa02ec22740 1 -- 172.21.15.31:0/1180962664 shutdown_connections 2024-04-25T01:18:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.974+0000 7fa02ec22740 1 --2- 172.21.15.31:0/1180962664 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5630fb105de0 0x5630fb108200 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.974+0000 7fa02ec22740 1 --2- 172.21.15.31:0/1180962664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630fb0f59b0 0x5630fb1016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.974+0000 7fa02ec22740 1 --2- 172.21.15.31:0/1180962664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5630fb0f0840 0x5630fb0fa650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.974+0000 7fa02ec22740 1 --2- 172.21.15.31:0/1180962664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0efe50 0x5630fb0fa110 secure :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa018097920 tx=0x7fa01809b380 comp rx=0 tx=0).stop 2024-04-25T01:18:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.974+0000 7fa02ec22740 1 -- 172.21.15.31:0/1180962664 >> 172.21.15.31:0/1180962664 conn(0x5630fb0eb4c0 msgr2=0x5630fb0ea620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:18:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.974+0000 7fa006a00640 0 client.25399 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:18:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.975+0000 7fa02ec22740 1 -- 172.21.15.31:0/1180962664 shutdown_connections 2024-04-25T01:18:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.975+0000 7fa02ec22740 1 -- 172.21.15.31:0/1180962664 wait complete. 2024-04-25T01:18:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.975+0000 7fa02ec22740 20 client.25399 trim_cache size 0 max 16384 2024-04-25T01:18:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.998+0000 7fa02ec22740 1 Processor -- start 2024-04-25T01:18:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.999+0000 7fa02ec22740 1 -- start start 2024-04-25T01:18:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.999+0000 7fa02ec22740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630fb0f0840 0x5630fb0fa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.999+0000 7fa02ec22740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0f59b0 0x5630fb0e9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.999+0000 7fa02ec22740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5630fb0e6570 0x5630fb0ea2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.999+0000 7fa02ec22740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa008010bf0 con 0x5630fb0f0840 2024-04-25T01:18:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.999+0000 7fa02ec22740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa00800e4c0 con 0x5630fb0e6570 2024-04-25T01:18:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:11.999+0000 7fa02ec22740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa00800c310 con 0x5630fb0f59b0 2024-04-25T01:18:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.000+0000 7fa028a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0f59b0 0x5630fb0e9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.000+0000 7fa028a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0f59b0 0x5630fb0e9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:38252/0 (socket says 172.21.15.31:38252) 2024-04-25T01:18:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.000+0000 7fa028a00640 1 -- 172.21.15.31:0/878654921 learned_addr learned my addr 172.21.15.31:0/878654921 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:18:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.000+0000 7fa029e00640 1 --2- 172.21.15.31:0/878654921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5630fb0e6570 0x5630fb0ea2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.000+0000 7fa028a00640 1 -- 172.21.15.31:0/878654921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5630fb0e6570 msgr2=0x5630fb0ea2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.000+0000 7fa028a00640 1 --2- 172.21.15.31:0/878654921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5630fb0e6570 0x5630fb0ea2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.000+0000 7fa028a00640 1 -- 172.21.15.31:0/878654921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630fb0f0840 msgr2=0x5630fb0fa450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:18:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.000+0000 7fa028a00640 1 --2- 172.21.15.31:0/878654921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630fb0f0840 0x5630fb0fa450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.000+0000 7fa028a00640 1 -- 172.21.15.31:0/878654921 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630fb0efa90 con 0x5630fb0f59b0 2024-04-25T01:18:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.000+0000 7fa028a00640 1 --2- 172.21.15.31:0/878654921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0f59b0 0x5630fb0e9d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa014031440 tx=0x7fa01403f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d640fc88b00686dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:18:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.001+0000 7fa011c00640 1 -- 172.21.15.31:0/878654921 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014056070 con 0x5630fb0f59b0 2024-04-25T01:18:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.001+0000 7fa011c00640 1 -- 172.21.15.31:0/878654921 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa014031e10 con 0x5630fb0f59b0 2024-04-25T01:18:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.001+0000 7fa011c00640 1 -- 172.21.15.31:0/878654921 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014035040 con 0x5630fb0f59b0 2024-04-25T01:18:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.002+0000 7fa02ec22740 1 -- 172.21.15.31:0/878654921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0f59b0 msgr2=0x5630fb0e9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.002+0000 7fa02ec22740 1 --2- 172.21.15.31:0/878654921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0f59b0 0x5630fb0e9d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa014031440 tx=0x7fa01403f000 comp rx=0 tx=0).stop 2024-04-25T01:18:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.002+0000 7fa02ec22740 1 -- 172.21.15.31:0/878654921 shutdown_connections 2024-04-25T01:18:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.002+0000 7fa02ec22740 1 --2- 172.21.15.31:0/878654921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5630fb0e6570 0x5630fb0ea2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.002+0000 7fa02ec22740 1 --2- 172.21.15.31:0/878654921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0f59b0 0x5630fb0e9d90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.002+0000 7fa02ec22740 1 --2- 172.21.15.31:0/878654921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630fb0f0840 0x5630fb0fa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.002+0000 7fa02ec22740 1 -- 172.21.15.31:0/878654921 >> 172.21.15.31:0/878654921 conn(0x5630fb0eb4c0 msgr2=0x5630fb0f9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:18:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.006+0000 7fa02ec22740 1 -- 172.21.15.31:0/878654921 shutdown_connections 2024-04-25T01:18:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.006+0000 7fa02ec22740 1 -- 172.21.15.31:0/878654921 wait complete. 2024-04-25T01:18:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.006+0000 7fa02ec22740 1 -- start start 2024-04-25T01:18:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.007+0000 7fa02ec22740 1 Processor -- start 2024-04-25T01:18:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.007+0000 7fa02ec22740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5630fb1003b0 0x5630fb1007f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.007+0000 7fa02ec22740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630fb100d30 0x5630fb0fea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.007+0000 7fa02ec22740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0ff000 0x5630fb0ff480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.007+0000 7fa02ec22740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa00800c980 con 0x5630fb100d30 2024-04-25T01:18:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.007+0000 7fa02ec22740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa00800ddf0 con 0x5630fb1003b0 2024-04-25T01:18:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.007+0000 7fa02ec22740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa00800d050 con 0x5630fb0ff000 2024-04-25T01:18:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.007+0000 7fa02ec22740 10 client.0 _ll_register_callbacks cb 0x7fa01800cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:18:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.008+0000 7fa029e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0ff000 0x5630fb0ff480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.008+0000 7fa029e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0ff000 0x5630fb0ff480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:38266/0 (socket says 172.21.15.31:38266) 2024-04-25T01:18:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.008+0000 7fa029e00640 1 -- 172.21.15.31:0/2462786939 learned_addr learned my addr 172.21.15.31:0/2462786939 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:18:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.008+0000 7fa029400640 1 --2- 172.21.15.31:0/2462786939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5630fb1003b0 0x5630fb1007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.008+0000 7fa029e00640 1 -- 172.21.15.31:0/2462786939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5630fb1003b0 msgr2=0x5630fb1007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.008+0000 7fa029e00640 1 --2- 172.21.15.31:0/2462786939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5630fb1003b0 0x5630fb1007f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.008+0000 7fa029e00640 1 -- 172.21.15.31:0/2462786939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630fb100d30 msgr2=0x5630fb0fea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.008+0000 7fa029e00640 1 --2- 172.21.15.31:0/2462786939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630fb100d30 0x5630fb0fea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.008+0000 7fa029e00640 1 -- 172.21.15.31:0/2462786939 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630fb16c540 con 0x5630fb0ff000 2024-04-25T01:18:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.009+0000 7fa029e00640 1 --2- 172.21.15.31:0/2462786939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0ff000 0x5630fb0ff480 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x5630fb0f10a0 tx=0x7fa01c00a8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dc8cf159e04e109 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:18:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.009+0000 7fa006a00640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:18:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.009+0000 7fa006a00640 1 -- 172.21.15.31:0/2462786939 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01c027020 con 0x5630fb0ff000 2024-04-25T01:18:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.009+0000 7fa006a00640 1 -- 172.21.15.31:0/2462786939 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fa01c010a50 con 0x5630fb0ff000 2024-04-25T01:18:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.009+0000 7fa02ec22740 1 -- 172.21.15.31:0/2462786939 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa014031e10 con 0x5630fb0ff000 2024-04-25T01:18:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.010+0000 7fa006a00640 1 -- 172.21.15.31:0/2462786939 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa01c01d040 con 0x5630fb0ff000 2024-04-25T01:18:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.011+0000 7fa02ec22740 10 client.25405 fetch_fsmap learned FSMap version 25 2024-04-25T01:18:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.011+0000 7fa02ec22740 1 -- 172.21.15.31:0/2462786939 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5630fb16d8f0 con 0x5630fb0ff000 2024-04-25T01:18:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.014+0000 7fa006a00640 1 -- 172.21.15.31:0/2462786939 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa014031e10 con 0x5630fb0ff000 2024-04-25T01:18:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.014+0000 7fa006a00640 1 -- 172.21.15.31:0/2462786939 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 25) ==== 8342+0+0 (secure 0 0 0) 0x7fa01c018440 con 0x5630fb0ff000 2024-04-25T01:18:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.014+0000 7fa02ec22740 10 client.25405 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T01:18:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.014+0000 7fa02ec22740 10 client.25405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:18:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.014+0000 7fa02ec22740 20 client.25405 populate_metadata read hostname 'smithi031' 2024-04-25T01:18:11.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.014+0000 7fa02ec22740 1 --2- 172.21.15.31:0/2462786939 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5630fb16df90 0x5630fb1703b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.014+0000 7fa02ec22740 4 client.25405 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:18:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.014+0000 7fa02ec22740 1 -- 172.21.15.31:0/2462786939 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630fb05a340 con 0x5630fb16df90 2024-04-25T01:18:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.015+0000 7fa029400640 1 --2- 172.21.15.31:0/2462786939 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5630fb16df90 0x5630fb1703b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.015+0000 7fa006a00640 10 client.25405 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:18:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.016+0000 7fa029400640 1 --2- 172.21.15.31:0/2462786939 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5630fb16df90 0x5630fb1703b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b28a24d64cd52e2 server_cookie=db78ecc852227d12 in_seq=0 out_seq=0 2024-04-25T01:18:12.003 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:11 smithi031 ceph-mon[28053]: pgmap v458: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 241 MiB/s wr, 99 op/s 2024-04-25T01:18:12.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.030+0000 7fa006a00640 1 -- 172.21.15.31:0/2462786939 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5630fb05a340 con 0x5630fb16df90 2024-04-25T01:18:12.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.031+0000 7fa006a00640 10 client.25405 handle_command_reply: tid=0 2024-04-25T01:18:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.031+0000 7fa02ec22740 1 client.25405 shutdown 2024-04-25T01:18:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.031+0000 7fa02ec22740 1 -- 172.21.15.31:0/2462786939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0ff000 msgr2=0x5630fb0ff480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:12.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.032+0000 7fa02ec22740 1 --2- 172.21.15.31:0/2462786939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0ff000 0x5630fb0ff480 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x5630fb0f10a0 tx=0x7fa01c00a8d0 comp rx=0 tx=0).stop 2024-04-25T01:18:12.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.033+0000 7fa02ec22740 1 -- 172.21.15.31:0/2462786939 shutdown_connections 2024-04-25T01:18:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.033+0000 7fa02ec22740 1 --2- 172.21.15.31:0/2462786939 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5630fb16df90 0x5630fb1703b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.033+0000 7fa02ec22740 1 --2- 172.21.15.31:0/2462786939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630fb0ff000 0x5630fb0ff480 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.033+0000 7fa02ec22740 1 --2- 172.21.15.31:0/2462786939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630fb100d30 0x5630fb0fea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.033+0000 7fa006a00640 0 client.25405 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:18:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.033+0000 7fa02ec22740 1 --2- 172.21.15.31:0/2462786939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5630fb1003b0 0x5630fb1007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.033+0000 7fa02ec22740 1 -- 172.21.15.31:0/2462786939 >> 172.21.15.31:0/2462786939 conn(0x5630fb0eb4c0 msgr2=0x5630fb0ee1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:18:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.034+0000 7fa02ec22740 1 -- 172.21.15.31:0/2462786939 shutdown_connections 2024-04-25T01:18:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.034+0000 7fa02ec22740 1 -- 172.21.15.31:0/2462786939 wait complete. 2024-04-25T01:18:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.035+0000 7fa02ec22740 20 client.25405 trim_cache size 0 max 16384 2024-04-25T01:18:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:18:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:18:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:18:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:18:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.036+0000 7fa02a800640 1 -- 172.21.15.31:0/904416877 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa00c0a2b10 msgr2=0x7fa00c0a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.036+0000 7fa02a800640 1 --2- 172.21.15.31:0/904416877 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa00c0a2b10 0x7fa00c0a4fd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa024111ef0 tx=0x7fa0140315b0 comp rx=0 tx=0).stop 2024-04-25T01:18:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.037+0000 7fa02a800640 1 -- 172.21.15.31:0/904416877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa024101600 msgr2=0x7fa0241109e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.037+0000 7fa02a800640 1 --2- 172.21.15.31:0/904416877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa024101600 0x7fa0241109e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa01800c940 tx=0x7fa01800ce00 comp rx=0 tx=0).stop 2024-04-25T01:18:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.037+0000 7fa02a800640 1 -- 172.21.15.31:0/904416877 shutdown_connections 2024-04-25T01:18:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.037+0000 7fa02a800640 1 --2- 172.21.15.31:0/904416877 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa00c0a2b10 0x7fa00c0a4fd0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.037+0000 7fa02a800640 1 --2- 172.21.15.31:0/904416877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0241031b0 0x7fa0241adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.037+0000 7fa02a800640 1 --2- 172.21.15.31:0/904416877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0241027f0 0x7fa024110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.037+0000 7fa02a800640 1 --2- 172.21.15.31:0/904416877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa024101600 0x7fa0241109e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.037+0000 7fa02a800640 1 -- 172.21.15.31:0/904416877 >> 172.21.15.31:0/904416877 conn(0x7fa0240fcb70 msgr2=0x7fa024105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:18:12.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.037+0000 7fa02a800640 1 -- 172.21.15.31:0/904416877 shutdown_connections 2024-04-25T01:18:12.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:12.038+0000 7fa02a800640 1 -- 172.21.15.31:0/904416877 wait complete. 2024-04-25T01:18:13.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:13 smithi169 ceph-mon[32759]: pgmap v459: 97 pgs: 97 active+clean; 5.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 247 MiB/s wr, 88 op/s 2024-04-25T01:18:13.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:13 smithi040 ceph-mon[32518]: pgmap v459: 97 pgs: 97 active+clean; 5.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 247 MiB/s wr, 88 op/s 2024-04-25T01:18:14.003 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:13 smithi031 ceph-mon[28053]: pgmap v459: 97 pgs: 97 active+clean; 5.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 247 MiB/s wr, 88 op/s 2024-04-25T01:18:14.942 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:14 smithi169 ceph-mon[32759]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:18:14.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:14 smithi040 ceph-mon[32518]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:18:15.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:14 smithi031 ceph-mon[28053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:18:15.942 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:15 smithi169 ceph-mon[32759]: pgmap v460: 97 pgs: 97 active+clean; 5.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 226 MiB/s wr, 83 op/s 2024-04-25T01:18:15.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:15 smithi040 ceph-mon[32518]: pgmap v460: 97 pgs: 97 active+clean; 5.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 226 MiB/s wr, 83 op/s 2024-04-25T01:18:16.003 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:15 smithi031 ceph-mon[28053]: pgmap v460: 97 pgs: 97 active+clean; 5.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 226 MiB/s wr, 83 op/s 2024-04-25T01:18:17.942 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:17 smithi169 ceph-mon[32759]: pgmap v461: 97 pgs: 97 active+clean; 5.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 237 MiB/s wr, 97 op/s 2024-04-25T01:18:17.945 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:17 smithi040 ceph-mon[32518]: pgmap v461: 97 pgs: 97 active+clean; 5.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 237 MiB/s wr, 97 op/s 2024-04-25T01:18:18.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:17 smithi031 ceph-mon[28053]: pgmap v461: 97 pgs: 97 active+clean; 5.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 237 MiB/s wr, 97 op/s 2024-04-25T01:18:19.942 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:19 smithi169 ceph-mon[32759]: pgmap v462: 97 pgs: 97 active+clean; 6.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 237 MiB/s wr, 87 op/s 2024-04-25T01:18:19.945 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:19 smithi040 ceph-mon[32518]: pgmap v462: 97 pgs: 97 active+clean; 6.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 237 MiB/s wr, 87 op/s 2024-04-25T01:18:20.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:19 smithi031 ceph-mon[28053]: pgmap v462: 97 pgs: 97 active+clean; 6.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 237 MiB/s wr, 87 op/s 2024-04-25T01:18:21.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:21 smithi169 ceph-mon[32759]: pgmap v463: 97 pgs: 97 active+clean; 6.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 244 MiB/s wr, 100 op/s 2024-04-25T01:18:21.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:21 smithi040 ceph-mon[32518]: pgmap v463: 97 pgs: 97 active+clean; 6.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 244 MiB/s wr, 100 op/s 2024-04-25T01:18:22.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:21 smithi031 ceph-mon[28053]: pgmap v463: 97 pgs: 97 active+clean; 6.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 244 MiB/s wr, 100 op/s 2024-04-25T01:18:23.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:23 smithi169 ceph-mon[32759]: pgmap v464: 97 pgs: 97 active+clean; 7.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 244 MiB/s wr, 90 op/s 2024-04-25T01:18:23.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:23 smithi040 ceph-mon[32518]: pgmap v464: 97 pgs: 97 active+clean; 7.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 244 MiB/s wr, 90 op/s 2024-04-25T01:18:24.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:23 smithi031 ceph-mon[28053]: pgmap v464: 97 pgs: 97 active+clean; 7.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 244 MiB/s wr, 90 op/s 2024-04-25T01:18:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.247+0000 7f1aaa200640 1 -- 172.21.15.31:0/1985548945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1aa41027f0 msgr2=0x7f1aa4102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.247+0000 7f1aaa200640 1 --2- 172.21.15.31:0/1985548945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1aa41027f0 0x7f1aa4102c70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1a90009860 tx=0x7f1a9002a360 comp rx=0 tx=0).stop 2024-04-25T01:18:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.248+0000 7f1aaa200640 1 -- 172.21.15.31:0/1985548945 shutdown_connections 2024-04-25T01:18:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.248+0000 7f1aaa200640 1 --2- 172.21.15.31:0/1985548945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1aa41031b0 0x7f1aa4109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.248+0000 7f1aaa200640 1 --2- 172.21.15.31:0/1985548945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1aa41027f0 0x7f1aa4102c70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.248+0000 7f1aaa200640 1 --2- 172.21.15.31:0/1985548945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa4101600 0x7f1aa4101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.248+0000 7f1aaa200640 1 -- 172.21.15.31:0/1985548945 >> 172.21.15.31:0/1985548945 conn(0x7f1aa40fcb70 msgr2=0x7f1aa40fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:18:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.249+0000 7f1aaa200640 1 -- 172.21.15.31:0/1985548945 shutdown_connections 2024-04-25T01:18:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.249+0000 7f1aaa200640 1 -- 172.21.15.31:0/1985548945 wait complete. 2024-04-25T01:18:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.251+0000 7f1aaa200640 1 Processor -- start 2024-04-25T01:18:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.252+0000 7f1aaa200640 1 -- start start 2024-04-25T01:18:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.253+0000 7f1aaa200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1aa4101600 0x7f1aa419df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.253+0000 7f1aaa200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa41027f0 0x7f1aa419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.253+0000 7f1aaa200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1aa41031b0 0x7f1aa41a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.253+0000 7f1aaa200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa410d260 con 0x7f1aa41031b0 2024-04-25T01:18:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.253+0000 7f1aaa200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa410d0e0 con 0x7f1aa41027f0 2024-04-25T01:18:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.253+0000 7f1aaa200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa410d3e0 con 0x7f1aa4101600 2024-04-25T01:18:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.254+0000 7f1aa8e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1aa4101600 0x7f1aa419df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.254+0000 7f1aa8e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1aa4101600 0x7f1aa419df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:38944/0 (socket says 172.21.15.31:38944) 2024-04-25T01:18:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.254+0000 7f1aa8e00640 1 -- 172.21.15.31:0/1642137983 learned_addr learned my addr 172.21.15.31:0/1642137983 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:18:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.254+0000 7f1aa3e00640 1 --2- 172.21.15.31:0/1642137983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa41027f0 0x7f1aa419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.255+0000 7f1aa8e00640 1 -- 172.21.15.31:0/1642137983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa41027f0 msgr2=0x7f1aa419e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.255+0000 7f1aa8e00640 1 --2- 172.21.15.31:0/1642137983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa41027f0 0x7f1aa419e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.255+0000 7f1aa8e00640 1 -- 172.21.15.31:0/1642137983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1aa41031b0 msgr2=0x7f1aa41a54c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:18:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.255+0000 7f1aa9800640 1 --2- 172.21.15.31:0/1642137983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1aa41031b0 0x7f1aa41a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.255+0000 7f1aa8e00640 1 --2- 172.21.15.31:0/1642137983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1aa41031b0 0x7f1aa41a54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.255+0000 7f1aa8e00640 1 -- 172.21.15.31:0/1642137983 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa41a5b30 con 0x7f1aa4101600 2024-04-25T01:18:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.256+0000 7f1aa8e00640 1 --2- 172.21.15.31:0/1642137983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1aa4101600 0x7f1aa419df10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800d770 tx=0x7f1a9800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:18:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.256+0000 7f1aa1600640 1 -- 172.21.15.31:0/1642137983 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a98017070 con 0x7f1aa4101600 2024-04-25T01:18:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.256+0000 7f1aaa200640 1 -- 172.21.15.31:0/1642137983 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa41a5e20 con 0x7f1aa4101600 2024-04-25T01:18:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.256+0000 7f1aaa200640 1 -- 172.21.15.31:0/1642137983 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa41a6290 con 0x7f1aa4101600 2024-04-25T01:18:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.256+0000 7f1aa1600640 1 -- 172.21.15.31:0/1642137983 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a98009c30 con 0x7f1aa4101600 2024-04-25T01:18:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.256+0000 7f1aa1600640 1 -- 172.21.15.31:0/1642137983 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a98003dd0 con 0x7f1aa4101600 2024-04-25T01:18:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.263+0000 7f1aa1600640 1 -- 172.21.15.31:0/1642137983 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1a98004020 con 0x7f1aa4101600 2024-04-25T01:18:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.264+0000 7f1aae79d740 1 Processor -- start 2024-04-25T01:18:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.265+0000 7f1aae79d740 1 -- start start 2024-04-25T01:18:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.265+0000 7f1aae79d740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f896ccee60 0x55f896ccf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.265+0000 7f1aae79d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896ccf850 0x55f896cd4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.266+0000 7f1aa8e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896ccf850 0x55f896cd4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.266+0000 7f1aa9800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f896ccee60 0x55f896ccf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.266+0000 7f1aae79d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f896cd49c0 0x55f896cd6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.266+0000 7f1aae79d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f896c20ab0 con 0x55f896ccf850 2024-04-25T01:18:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.266+0000 7f1aa8e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896ccf850 0x55f896cd4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45968/0 (socket says 172.21.15.31:45968) 2024-04-25T01:18:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.266+0000 7f1aa8e00640 1 -- 172.21.15.31:0/1600998146 learned_addr learned my addr 172.21.15.31:0/1600998146 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:18:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.267+0000 7f1aae79d740 1 -- 172.21.15.31:0/1600998146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f896bc9640 con 0x55f896ccee60 2024-04-25T01:18:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.267+0000 7f1aae79d740 1 -- 172.21.15.31:0/1600998146 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f896c1ca70 con 0x55f896cd49c0 2024-04-25T01:18:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.267+0000 7f1aa3e00640 1 --2- 172.21.15.31:0/1600998146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f896cd49c0 0x55f896cd6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.268+0000 7f1aa8e00640 1 -- 172.21.15.31:0/1600998146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f896cd49c0 msgr2=0x55f896cd6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.268+0000 7f1aa8e00640 1 --2- 172.21.15.31:0/1600998146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f896cd49c0 0x55f896cd6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.268+0000 7f1aa8e00640 1 -- 172.21.15.31:0/1600998146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f896ccee60 msgr2=0x55f896ccf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.268+0000 7f1aa8e00640 1 --2- 172.21.15.31:0/1600998146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f896ccee60 0x55f896ccf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.268+0000 7f1aa8e00640 1 -- 172.21.15.31:0/1600998146 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f896935210 con 0x55f896ccf850 2024-04-25T01:18:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.269+0000 7f1aa8e00640 1 --2- 172.21.15.31:0/1600998146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896ccf850 0x55f896cd4350 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1a980f6e10 tx=0x7f1a980f8680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d62b6948209595 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:18:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.270+0000 7f1a96000640 1 -- 172.21.15.31:0/1600998146 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a980ff070 con 0x55f896ccf850 2024-04-25T01:18:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.270+0000 7f1a96000640 1 -- 172.21.15.31:0/1600998146 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a980f8e50 con 0x55f896ccf850 2024-04-25T01:18:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.270+0000 7f1a96000640 1 -- 172.21.15.31:0/1600998146 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a980faa50 con 0x55f896ccf850 2024-04-25T01:18:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.271+0000 7f1aae79d740 1 -- 172.21.15.31:0/1600998146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896ccf850 msgr2=0x55f896cd4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.271+0000 7f1aae79d740 1 --2- 172.21.15.31:0/1600998146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896ccf850 0x55f896cd4350 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1a980f6e10 tx=0x7f1a980f8680 comp rx=0 tx=0).stop 2024-04-25T01:18:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.272+0000 7f1aae79d740 1 -- 172.21.15.31:0/1600998146 shutdown_connections 2024-04-25T01:18:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.272+0000 7f1aae79d740 1 --2- 172.21.15.31:0/1600998146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f896cd49c0 0x55f896cd6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.272+0000 7f1aae79d740 1 --2- 172.21.15.31:0/1600998146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896ccf850 0x55f896cd4350 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.272+0000 7f1aae79d740 1 --2- 172.21.15.31:0/1600998146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f896ccee60 0x55f896ccf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.272+0000 7f1aae79d740 1 -- 172.21.15.31:0/1600998146 >> 172.21.15.31:0/1600998146 conn(0x55f896cb94c0 msgr2=0x55f896cb98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:18:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.272+0000 7f1aae79d740 1 -- 172.21.15.31:0/1600998146 shutdown_connections 2024-04-25T01:18:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.272+0000 7f1aae79d740 1 -- 172.21.15.31:0/1600998146 wait complete. 2024-04-25T01:18:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.272+0000 7f1aa1600640 1 --2- 172.21.15.31:0/1642137983 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1a8c0a2b10 0x7f1a8c0a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.273+0000 7f1aae79d740 1 -- start start 2024-04-25T01:18:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.273+0000 7f1aa1600640 1 -- 172.21.15.31:0/1642137983 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f1a980ce960 con 0x7f1aa4101600 2024-04-25T01:18:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.273+0000 7f1aae79d740 1 Processor -- start 2024-04-25T01:18:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.274+0000 7f1aa3e00640 1 --2- 172.21.15.31:0/1642137983 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1a8c0a2b10 0x7f1a8c0a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.275+0000 7f1aae79d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f896ccee60 0x55f896cbf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.275+0000 7f1aae79d740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f896ccf850 0x55f896cbfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.275+0000 7f1aae79d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896cd49c0 0x55f896cc6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.276+0000 7f1aa9800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896cd49c0 0x55f896cc6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.276+0000 7f1aa3e00640 1 --2- 172.21.15.31:0/1642137983 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1a8c0a2b10 0x7f1a8c0a4fd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1a90030420 tx=0x7f1a900304f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:18:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.276+0000 7f1aa9800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896cd49c0 0x55f896cc6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45976/0 (socket says 172.21.15.31:45976) 2024-04-25T01:18:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.276+0000 7f1aa9800640 1 -- 172.21.15.31:0/2976256917 learned_addr learned my addr 172.21.15.31:0/2976256917 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:18:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.276+0000 7f1aa3e00640 1 --2- 172.21.15.31:0/2976256917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f896ccf850 0x55f896cbfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.276+0000 7f1aae79d740 1 -- 172.21.15.31:0/2976256917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f896cbdd10 con 0x55f896cd49c0 2024-04-25T01:18:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.276+0000 7f1aae79d740 1 -- 172.21.15.31:0/2976256917 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f896bdc990 con 0x55f896ccf850 2024-04-25T01:18:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.276+0000 7f1aae79d740 1 -- 172.21.15.31:0/2976256917 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f896cbde90 con 0x55f896ccee60 2024-04-25T01:18:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.277+0000 7f1aa8e00640 1 --2- 172.21.15.31:0/2976256917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f896ccee60 0x55f896cbf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.277+0000 7f1aa3e00640 1 -- 172.21.15.31:0/2976256917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f896ccee60 msgr2=0x55f896cbf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.277+0000 7f1aae79d740 10 client.0 _ll_register_callbacks cb 0x55f896bde950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:18:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.277+0000 7f1aa3e00640 1 --2- 172.21.15.31:0/2976256917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f896ccee60 0x55f896cbf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.277+0000 7f1aa3e00640 1 -- 172.21.15.31:0/2976256917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896cd49c0 msgr2=0x55f896cc6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.277+0000 7f1aa3e00640 1 --2- 172.21.15.31:0/2976256917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896cd49c0 0x55f896cc6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.277+0000 7f1aa3e00640 1 -- 172.21.15.31:0/2976256917 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f896a01620 con 0x55f896ccf850 2024-04-25T01:18:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.277+0000 7f1aa9800640 1 --2- 172.21.15.31:0/2976256917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896cd49c0 0x55f896cc6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:18:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.278+0000 7f1a8aa00640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T01:18:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.278+0000 7f1aa3e00640 1 --2- 172.21.15.31:0/2976256917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f896ccf850 0x55f896cbfac0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1a9002a880 tx=0x7f1a9002a360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36f9dc6940046eb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:18:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.278+0000 7f1aa8e00640 1 --2- 172.21.15.31:0/2976256917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f896ccee60 0x55f896cbf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:18:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.278+0000 7f1aae79d740 1 -- 172.21.15.31:0/2976256917 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a980f8e50 con 0x55f896ccf850 2024-04-25T01:18:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.278+0000 7f1a8aa00640 1 -- 172.21.15.31:0/2976256917 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9004c020 con 0x55f896ccf850 2024-04-25T01:18:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.278+0000 7f1a8aa00640 1 -- 172.21.15.31:0/2976256917 <== mon.2 v2:172.21.15.40:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f1a90030b70 con 0x55f896ccf850 2024-04-25T01:18:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.279+0000 7f1a8aa00640 1 -- 172.21.15.31:0/2976256917 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1a90044040 con 0x55f896ccf850 2024-04-25T01:18:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.279+0000 7f1aae79d740 10 client.25388 fetch_fsmap learned FSMap version 26 2024-04-25T01:18:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.279+0000 7f1aae79d740 1 -- 172.21.15.31:0/2976256917 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55f896cc9ab0 con 0x55f896ccf850 2024-04-25T01:18:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.279+0000 7f1a8aa00640 1 -- 172.21.15.31:0/2976256917 <== mon.2 v2:172.21.15.40:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f1a90047030 con 0x55f896ccf850 2024-04-25T01:18:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.279+0000 7f1aae79d740 10 client.25388 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T01:18:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.279+0000 7f1aae79d740 10 client.25388 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:18:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.279+0000 7f1aae79d740 20 client.25388 populate_metadata read hostname 'smithi031' 2024-04-25T01:18:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.279+0000 7f1aae79d740 1 --2- 172.21.15.31:0/2976256917 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55f896ccb580 0x55f896ccda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.279+0000 7f1aae79d740 4 client.25388 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:18:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.279+0000 7f1aae79d740 1 -- 172.21.15.31:0/2976256917 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f896c2dff0 con 0x55f896ccb580 2024-04-25T01:18:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.280+0000 7f1aa8e00640 1 --2- 172.21.15.31:0/2976256917 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55f896ccb580 0x55f896ccda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.280+0000 7f1aa8e00640 1 --2- 172.21.15.31:0/2976256917 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55f896ccb580 0x55f896ccda40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35f888c247d3ca62 server_cookie=c556ed07866eb594 in_seq=0 out_seq=0 2024-04-25T01:18:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.280+0000 7f1a8aa00640 10 client.25388 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:18:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.282+0000 7f1a8aa00640 1 -- 172.21.15.31:0/2976256917 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f896c2dff0 con 0x55f896ccb580 2024-04-25T01:18:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.282+0000 7f1a8aa00640 10 client.25388 handle_command_reply: tid=0 2024-04-25T01:18:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.282+0000 7f1aae79d740 1 client.25388 shutdown 2024-04-25T01:18:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.283+0000 7f1aae79d740 1 -- 172.21.15.31:0/2976256917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f896ccf850 msgr2=0x55f896cbfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.283+0000 7f1aae79d740 1 --2- 172.21.15.31:0/2976256917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f896ccf850 0x55f896cbfac0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1a9002a880 tx=0x7f1a9002a360 comp rx=0 tx=0).stop 2024-04-25T01:18:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.283+0000 7f1aae79d740 1 -- 172.21.15.31:0/2976256917 shutdown_connections 2024-04-25T01:18:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.283+0000 7f1aae79d740 1 --2- 172.21.15.31:0/2976256917 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55f896ccb580 0x55f896ccda40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.283+0000 7f1a8aa00640 0 client.25388 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:18:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.283+0000 7f1aae79d740 1 --2- 172.21.15.31:0/2976256917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896cd49c0 0x55f896cc6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.283+0000 7f1aae79d740 1 --2- 172.21.15.31:0/2976256917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f896ccf850 0x55f896cbfac0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.283+0000 7f1aae79d740 1 --2- 172.21.15.31:0/2976256917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f896ccee60 0x55f896cbf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.283+0000 7f1aae79d740 1 -- 172.21.15.31:0/2976256917 >> 172.21.15.31:0/2976256917 conn(0x55f896cb94c0 msgr2=0x55f896cb8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:18:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.283+0000 7f1aae79d740 1 -- 172.21.15.31:0/2976256917 shutdown_connections 2024-04-25T01:18:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.283+0000 7f1aae79d740 1 -- 172.21.15.31:0/2976256917 wait complete. 2024-04-25T01:18:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.283+0000 7f1aae79d740 20 client.25388 trim_cache size 0 max 16384 2024-04-25T01:18:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.302+0000 7f1aae79d740 1 Processor -- start 2024-04-25T01:18:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.303+0000 7f1aae79d740 1 -- start start 2024-04-25T01:18:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.303+0000 7f1aae79d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f896cb94c0 0x55f896cbea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.303+0000 7f1aae79d740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f896ccee60 0x55f896cc6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.303+0000 7f1aae79d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896ccf850 0x55f896cb7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.303+0000 7f1aae79d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1a84010b90 con 0x55f896ccf850 2024-04-25T01:18:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.303+0000 7f1aae79d740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8400e430 con 0x55f896ccee60 2024-04-25T01:18:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.303+0000 7f1aae79d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8400c280 con 0x55f896cb94c0 2024-04-25T01:18:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.303+0000 7f1aa8e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f896cb94c0 0x55f896cbea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.303+0000 7f1aa8e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f896cb94c0 0x55f896cbea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:38972/0 (socket says 172.21.15.31:38972) 2024-04-25T01:18:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.303+0000 7f1aa8e00640 1 -- 172.21.15.31:0/2515453538 learned_addr learned my addr 172.21.15.31:0/2515453538 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:18:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.303+0000 7f1aa3e00640 1 --2- 172.21.15.31:0/2515453538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f896ccee60 0x55f896cc6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.303+0000 7f1aa9800640 1 --2- 172.21.15.31:0/2515453538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896ccf850 0x55f896cb7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.304+0000 7f1aa8e00640 1 -- 172.21.15.31:0/2515453538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f896ccee60 msgr2=0x55f896cc6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.304+0000 7f1aa8e00640 1 --2- 172.21.15.31:0/2515453538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f896ccee60 0x55f896cc6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.304+0000 7f1aa8e00640 1 -- 172.21.15.31:0/2515453538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896ccf850 msgr2=0x55f896cb7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.304+0000 7f1aa8e00640 1 --2- 172.21.15.31:0/2515453538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896ccf850 0x55f896cb7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.304+0000 7f1aa8e00640 1 -- 172.21.15.31:0/2515453538 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f896cd00b0 con 0x55f896cb94c0 2024-04-25T01:18:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.304+0000 7f1aa9800640 1 --2- 172.21.15.31:0/2515453538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896ccf850 0x55f896cb7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:18:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.304+0000 7f1aa3e00640 1 --2- 172.21.15.31:0/2515453538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f896ccee60 0x55f896cc6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:18:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.304+0000 7f1aa8e00640 1 --2- 172.21.15.31:0/2515453538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f896cb94c0 0x55f896cbea20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1a980976b0 tx=0x7f1a980f6540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3107958b8db8526a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:18:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.304+0000 7f1a95600640 1 -- 172.21.15.31:0/2515453538 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a98020070 con 0x55f896cb94c0 2024-04-25T01:18:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.305+0000 7f1a95600640 1 -- 172.21.15.31:0/2515453538 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a98009de0 con 0x55f896cb94c0 2024-04-25T01:18:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.305+0000 7f1a95600640 1 -- 172.21.15.31:0/2515453538 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a980ff070 con 0x55f896cb94c0 2024-04-25T01:18:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.305+0000 7f1aae79d740 1 -- 172.21.15.31:0/2515453538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f896cb94c0 msgr2=0x55f896cbea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.305+0000 7f1aae79d740 1 --2- 172.21.15.31:0/2515453538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f896cb94c0 0x55f896cbea20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1a980976b0 tx=0x7f1a980f6540 comp rx=0 tx=0).stop 2024-04-25T01:18:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.305+0000 7f1aae79d740 1 -- 172.21.15.31:0/2515453538 shutdown_connections 2024-04-25T01:18:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.305+0000 7f1aae79d740 1 --2- 172.21.15.31:0/2515453538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896ccf850 0x55f896cb7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.305+0000 7f1aae79d740 1 --2- 172.21.15.31:0/2515453538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f896ccee60 0x55f896cc6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.305+0000 7f1aae79d740 1 --2- 172.21.15.31:0/2515453538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f896cb94c0 0x55f896cbea20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.305+0000 7f1aae79d740 1 -- 172.21.15.31:0/2515453538 >> 172.21.15.31:0/2515453538 conn(0x55f896cd44c0 msgr2=0x55f896cbe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:18:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.305+0000 7f1aae79d740 1 -- 172.21.15.31:0/2515453538 shutdown_connections 2024-04-25T01:18:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.305+0000 7f1aae79d740 1 -- 172.21.15.31:0/2515453538 wait complete. 2024-04-25T01:18:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.306+0000 7f1aae79d740 1 -- start start 2024-04-25T01:18:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.306+0000 7f1aae79d740 1 Processor -- start 2024-04-25T01:18:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.306+0000 7f1aae79d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f896cb94c0 0x55f896cc3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.306+0000 7f1aae79d740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f896ccee60 0x55f896cc3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.306+0000 7f1aae79d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896ccf850 0x55f896d38fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.306+0000 7f1aae79d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8400c8f0 con 0x55f896ccf850 2024-04-25T01:18:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.306+0000 7f1aae79d740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8400dd60 con 0x55f896ccee60 2024-04-25T01:18:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.306+0000 7f1aae79d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8400cfc0 con 0x55f896cb94c0 2024-04-25T01:18:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.306+0000 7f1aae79d740 10 client.0 _ll_register_callbacks cb 0x7f1a90035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:18:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.307+0000 7f1aa3e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f896ccee60 0x55f896cc3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.307+0000 7f1aa3e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f896ccee60 0x55f896cc3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:60928/0 (socket says 172.21.15.31:60928) 2024-04-25T01:18:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.307+0000 7f1aa3e00640 1 -- 172.21.15.31:0/3298353677 learned_addr learned my addr 172.21.15.31:0/3298353677 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:18:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.307+0000 7f1aa8e00640 1 --2- 172.21.15.31:0/3298353677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f896cb94c0 0x55f896cc3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.307+0000 7f1aa9800640 1 --2- 172.21.15.31:0/3298353677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896ccf850 0x55f896d38fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.307+0000 7f1aa3e00640 1 -- 172.21.15.31:0/3298353677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f896cb94c0 msgr2=0x55f896cc3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.307+0000 7f1aa3e00640 1 --2- 172.21.15.31:0/3298353677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f896cb94c0 0x55f896cc3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.307+0000 7f1aa3e00640 1 -- 172.21.15.31:0/3298353677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896ccf850 msgr2=0x55f896d38fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.307+0000 7f1aa3e00640 1 --2- 172.21.15.31:0/3298353677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896ccf850 0x55f896d38fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.307+0000 7f1aa3e00640 1 -- 172.21.15.31:0/3298353677 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f896cc6c30 con 0x55f896ccee60 2024-04-25T01:18:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.307+0000 7f1aa8e00640 1 --2- 172.21.15.31:0/3298353677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f896cb94c0 0x55f896cc3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:18:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.307+0000 7f1aa9800640 1 --2- 172.21.15.31:0/3298353677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896ccf850 0x55f896d38fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:18:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.308+0000 7f1aa3e00640 1 --2- 172.21.15.31:0/3298353677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f896ccee60 0x55f896cc3b60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1a900045c0 tx=0x7f1a9003f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e84c1015123b257c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:18:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.308+0000 7f1a8aa00640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T01:18:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.310+0000 7f1a8aa00640 1 -- 172.21.15.31:0/3298353677 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9004c040 con 0x55f896ccee60 2024-04-25T01:18:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.310+0000 7f1a8aa00640 1 -- 172.21.15.31:0/3298353677 <== mon.2 v2:172.21.15.40:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f1a900035f0 con 0x55f896ccee60 2024-04-25T01:18:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.310+0000 7f1aae79d740 1 -- 172.21.15.31:0/3298353677 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a98009de0 con 0x55f896ccee60 2024-04-25T01:18:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.310+0000 7f1a8aa00640 1 -- 172.21.15.31:0/3298353677 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1a90069050 con 0x55f896ccee60 2024-04-25T01:18:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.310+0000 7f1aae79d740 10 client.25394 fetch_fsmap learned FSMap version 26 2024-04-25T01:18:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.310+0000 7f1aae79d740 1 -- 172.21.15.31:0/3298353677 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f896d3a390 con 0x55f896ccee60 2024-04-25T01:18:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.310+0000 7f1a8aa00640 1 -- 172.21.15.31:0/3298353677 <== mon.2 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a98009de0 con 0x55f896ccee60 2024-04-25T01:18:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.310+0000 7f1a8aa00640 1 -- 172.21.15.31:0/3298353677 <== mon.2 v2:172.21.15.40:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f1a90047040 con 0x55f896ccee60 2024-04-25T01:18:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.310+0000 7f1aae79d740 10 client.25394 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T01:18:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.311+0000 7f1aae79d740 10 client.25394 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:18:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.311+0000 7f1aae79d740 20 client.25394 populate_metadata read hostname 'smithi031' 2024-04-25T01:18:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.311+0000 7f1aae79d740 1 --2- 172.21.15.31:0/3298353677 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55f896cd49c0 0x55f896d3cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.311+0000 7f1aae79d740 4 client.25394 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:18:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.311+0000 7f1aae79d740 1 -- 172.21.15.31:0/3298353677 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f896c28340 con 0x55f896cd49c0 2024-04-25T01:18:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.311+0000 7f1aa8e00640 1 --2- 172.21.15.31:0/3298353677 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55f896cd49c0 0x55f896d3cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.312+0000 7f1aa8e00640 1 --2- 172.21.15.31:0/3298353677 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55f896cd49c0 0x55f896d3cd50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad4dc39de39724a1 server_cookie=26ff34245d7429b4 in_seq=0 out_seq=0 2024-04-25T01:18:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.312+0000 7f1a8aa00640 10 client.25394 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:18:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.337+0000 7f1a8aa00640 1 -- 172.21.15.31:0/3298353677 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f896c28340 con 0x55f896cd49c0 2024-04-25T01:18:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.337+0000 7f1a8aa00640 10 client.25394 handle_command_reply: tid=0 2024-04-25T01:18:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.337+0000 7f1aae79d740 1 client.25394 shutdown 2024-04-25T01:18:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.337+0000 7f1aae79d740 1 -- 172.21.15.31:0/3298353677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f896ccee60 msgr2=0x55f896cc3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.337+0000 7f1aae79d740 1 --2- 172.21.15.31:0/3298353677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f896ccee60 0x55f896cc3b60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1a900045c0 tx=0x7f1a9003f000 comp rx=0 tx=0).stop 2024-04-25T01:18:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.337+0000 7f1aae79d740 1 -- 172.21.15.31:0/3298353677 shutdown_connections 2024-04-25T01:18:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.337+0000 7f1aae79d740 1 --2- 172.21.15.31:0/3298353677 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55f896cd49c0 0x55f896d3cd50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.337+0000 7f1a8aa00640 0 client.25394 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:18:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.337+0000 7f1aae79d740 1 --2- 172.21.15.31:0/3298353677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f896ccf850 0x55f896d38fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.337+0000 7f1aae79d740 1 --2- 172.21.15.31:0/3298353677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f896ccee60 0x55f896cc3b60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.337+0000 7f1aae79d740 1 --2- 172.21.15.31:0/3298353677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f896cb94c0 0x55f896cc3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.337+0000 7f1aae79d740 1 -- 172.21.15.31:0/3298353677 >> 172.21.15.31:0/3298353677 conn(0x55f896cd44c0 msgr2=0x55f896cbc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:18:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.337+0000 7f1aae79d740 1 -- 172.21.15.31:0/3298353677 shutdown_connections 2024-04-25T01:18:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.338+0000 7f1aae79d740 1 -- 172.21.15.31:0/3298353677 wait complete. 2024-04-25T01:18:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.338+0000 7f1aae79d740 20 client.25394 trim_cache size 0 max 16384 2024-04-25T01:18:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:18:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:18:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:18:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:18:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.339+0000 7f1aaa200640 1 -- 172.21.15.31:0/1642137983 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1a8c0a2b10 msgr2=0x7f1a8c0a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.339+0000 7f1aaa200640 1 --2- 172.21.15.31:0/1642137983 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1a8c0a2b10 0x7f1a8c0a4fd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1a90030420 tx=0x7f1a900304f0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.339+0000 7f1aaa200640 1 -- 172.21.15.31:0/1642137983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1aa4101600 msgr2=0x7f1aa419df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.339+0000 7f1aaa200640 1 --2- 172.21.15.31:0/1642137983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1aa4101600 0x7f1aa419df10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800d770 tx=0x7f1a9800dc30 comp rx=0 tx=0).stop 2024-04-25T01:18:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.339+0000 7f1aaa200640 1 -- 172.21.15.31:0/1642137983 shutdown_connections 2024-04-25T01:18:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.339+0000 7f1aaa200640 1 --2- 172.21.15.31:0/1642137983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1aa41031b0 0x7f1aa41a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.339+0000 7f1aaa200640 1 --2- 172.21.15.31:0/1642137983 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f1a8c0a2b10 0x7f1a8c0a4fd0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.339+0000 7f1aaa200640 1 --2- 172.21.15.31:0/1642137983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa41027f0 0x7f1aa419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.339+0000 7f1aaa200640 1 --2- 172.21.15.31:0/1642137983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1aa4101600 0x7f1aa419df10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.339+0000 7f1aaa200640 1 -- 172.21.15.31:0/1642137983 >> 172.21.15.31:0/1642137983 conn(0x7f1aa40fcb70 msgr2=0x7f1aa4103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:18:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.339+0000 7f1aaa200640 1 -- 172.21.15.31:0/1642137983 shutdown_connections 2024-04-25T01:18:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:25.340+0000 7f1aaa200640 1 -- 172.21.15.31:0/1642137983 wait complete. 2024-04-25T01:18:25.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:25 smithi169 ceph-mon[32759]: pgmap v465: 97 pgs: 97 active+clean; 7.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 226 MiB/s wr, 86 op/s 2024-04-25T01:18:25.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:25 smithi040 ceph-mon[32518]: pgmap v465: 97 pgs: 97 active+clean; 7.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 226 MiB/s wr, 86 op/s 2024-04-25T01:18:26.003 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:25 smithi031 ceph-mon[28053]: pgmap v465: 97 pgs: 97 active+clean; 7.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 226 MiB/s wr, 86 op/s 2024-04-25T01:18:27.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:27 smithi169 ceph-mon[32759]: pgmap v466: 97 pgs: 97 active+clean; 8.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 238 MiB/s wr, 98 op/s 2024-04-25T01:18:27.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:27 smithi040 ceph-mon[32518]: pgmap v466: 97 pgs: 97 active+clean; 8.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 238 MiB/s wr, 98 op/s 2024-04-25T01:18:28.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:27 smithi031 ceph-mon[28053]: pgmap v466: 97 pgs: 97 active+clean; 8.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 238 MiB/s wr, 98 op/s 2024-04-25T01:18:29.045 INFO:tasks.workunit.client.0.smithi031.stdout:fs setup took 48 secs 2024-04-25T01:18:29.942 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:29 smithi169 ceph-mon[32759]: pgmap v467: 97 pgs: 97 active+clean; 8.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 245 MiB/s wr, 91 op/s 2024-04-25T01:18:29.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:29 smithi040 ceph-mon[32518]: pgmap v467: 97 pgs: 97 active+clean; 8.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 245 MiB/s wr, 91 op/s 2024-04-25T01:18:30.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:29 smithi031 ceph-mon[28053]: pgmap v467: 97 pgs: 97 active+clean; 8.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 245 MiB/s wr, 91 op/s 2024-04-25T01:18:31.944 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:31 smithi169 ceph-mon[32759]: pgmap v468: 97 pgs: 97 active+clean; 9.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 249 MiB/s wr, 106 op/s 2024-04-25T01:18:31.945 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:31 smithi040 ceph-mon[32518]: pgmap v468: 97 pgs: 97 active+clean; 9.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 249 MiB/s wr, 106 op/s 2024-04-25T01:18:32.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:31 smithi031 ceph-mon[28053]: pgmap v468: 97 pgs: 97 active+clean; 9.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 249 MiB/s wr, 106 op/s 2024-04-25T01:18:33.945 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:33 smithi040 ceph-mon[32518]: pgmap v469: 97 pgs: 97 active+clean; 9.9 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 261 MiB/s wr, 99 op/s 2024-04-25T01:18:34.003 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:33 smithi031 ceph-mon[28053]: pgmap v469: 97 pgs: 97 active+clean; 9.9 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 261 MiB/s wr, 99 op/s 2024-04-25T01:18:34.191 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:33 smithi169 ceph-mon[32759]: pgmap v469: 97 pgs: 97 active+clean; 9.9 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 261 MiB/s wr, 99 op/s 2024-04-25T01:18:36.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:35 smithi031 ceph-mon[28053]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 243 MiB/s wr, 94 op/s 2024-04-25T01:18:36.191 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:35 smithi169 ceph-mon[32759]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 243 MiB/s wr, 94 op/s 2024-04-25T01:18:36.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:35 smithi040 ceph-mon[32518]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 243 MiB/s wr, 94 op/s 2024-04-25T01:18:38.017 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:37 smithi031 ceph-mon[28053]: pgmap v471: 97 pgs: 97 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 261 MiB/s wr, 100 op/s 2024-04-25T01:18:38.191 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:37 smithi169 ceph-mon[32759]: pgmap v471: 97 pgs: 97 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 261 MiB/s wr, 100 op/s 2024-04-25T01:18:38.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:37 smithi040 ceph-mon[32518]: pgmap v471: 97 pgs: 97 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 261 MiB/s wr, 100 op/s 2024-04-25T01:18:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.552+0000 7f3708400640 1 -- 172.21.15.31:0/3186891571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3700103350 msgr2=0x7f3700109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.552+0000 7f3708400640 1 --2- 172.21.15.31:0/3186891571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3700103350 0x7f3700109be0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f36ec009900 tx=0x7f36ec02a410 comp rx=0 tx=0).stop 2024-04-25T01:18:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.552+0000 7f3708400640 1 -- 172.21.15.31:0/3186891571 shutdown_connections 2024-04-25T01:18:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.552+0000 7f3708400640 1 --2- 172.21.15.31:0/3186891571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3700103350 0x7f3700109be0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.552+0000 7f3708400640 1 --2- 172.21.15.31:0/3186891571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3700102990 0x7f3700102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.552+0000 7f3708400640 1 --2- 172.21.15.31:0/3186891571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37001017a0 0x7f3700101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.552+0000 7f3708400640 1 -- 172.21.15.31:0/3186891571 >> 172.21.15.31:0/3186891571 conn(0x7f37000fcd10 msgr2=0x7f37000ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:18:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.554+0000 7f3708400640 1 -- 172.21.15.31:0/3186891571 shutdown_connections 2024-04-25T01:18:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.554+0000 7f3708400640 1 -- 172.21.15.31:0/3186891571 wait complete. 2024-04-25T01:18:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.556+0000 7f3708400640 1 Processor -- start 2024-04-25T01:18:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.556+0000 7f3708400640 1 -- start start 2024-04-25T01:18:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.557+0000 7f3708400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37001017a0 0x7f37001a7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.557+0000 7f3708400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3700102990 0x7f37001a79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.557+0000 7f3708400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3700103350 0x7f37001a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.557+0000 7f3708400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f370010d400 con 0x7f3700102990 2024-04-25T01:18:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.557+0000 7f3708400640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f370010d280 con 0x7f37001017a0 2024-04-25T01:18:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.557+0000 7f3708400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f370010d580 con 0x7f3700103350 2024-04-25T01:18:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.558+0000 7f3705a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37001017a0 0x7f37001a7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.558+0000 7f3705a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37001017a0 0x7f37001a7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:47558/0 (socket says 172.21.15.31:47558) 2024-04-25T01:18:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.558+0000 7f3705a00640 1 -- 172.21.15.31:0/758831238 learned_addr learned my addr 172.21.15.31:0/758831238 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:18:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.558+0000 7f3706400640 1 --2- 172.21.15.31:0/758831238 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3700103350 0x7f37001a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.558+0000 7f3705000640 1 --2- 172.21.15.31:0/758831238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3700102990 0x7f37001a79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.559+0000 7f3705a00640 1 -- 172.21.15.31:0/758831238 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3700103350 msgr2=0x7f37001a3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.559+0000 7f3705a00640 1 --2- 172.21.15.31:0/758831238 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3700103350 0x7f37001a3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.559+0000 7f3705a00640 1 -- 172.21.15.31:0/758831238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3700102990 msgr2=0x7f37001a79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.559+0000 7f3705a00640 1 --2- 172.21.15.31:0/758831238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3700102990 0x7f37001a79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.559+0000 7f3705a00640 1 -- 172.21.15.31:0/758831238 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37001a4890 con 0x7f37001017a0 2024-04-25T01:18:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.559+0000 7f3706400640 1 --2- 172.21.15.31:0/758831238 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3700103350 0x7f37001a3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:18:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.560+0000 7f3705a00640 1 --2- 172.21.15.31:0/758831238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37001017a0 0x7f37001a7460 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f36f800ebc0 tx=0x7f36f800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:18:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.560+0000 7f36fe000640 1 -- 172.21.15.31:0/758831238 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8003b70 con 0x7f37001017a0 2024-04-25T01:18:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.560+0000 7f3708400640 1 -- 172.21.15.31:0/758831238 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3700073c10 con 0x7f37001017a0 2024-04-25T01:18:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.560+0000 7f36fe000640 1 -- 172.21.15.31:0/758831238 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36f8003d10 con 0x7f37001017a0 2024-04-25T01:18:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.560+0000 7f36fe000640 1 -- 172.21.15.31:0/758831238 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8019980 con 0x7f37001017a0 2024-04-25T01:18:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.560+0000 7f3708400640 1 -- 172.21.15.31:0/758831238 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37000741b0 con 0x7f37001017a0 2024-04-25T01:18:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.564+0000 7f370b1f0740 1 Processor -- start 2024-04-25T01:18:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.564+0000 7f370b1f0740 1 -- start start 2024-04-25T01:18:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.565+0000 7f370b1f0740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22f9ca0 0x55ddc22fa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.565+0000 7f370b1f0740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ddc22fa690 0x55ddc22ff190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.565+0000 7f370b1f0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddc22ff800 0x55ddc2301c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.565+0000 7f3706400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22f9ca0 0x55ddc22fa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.565+0000 7f3706400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22f9ca0 0x55ddc22fa0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:55376/0 (socket says 172.21.15.31:55376) 2024-04-25T01:18:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.565+0000 7f3706400640 1 -- 172.21.15.31:0/2100965221 learned_addr learned my addr 172.21.15.31:0/2100965221 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:18:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.565+0000 7f370b1f0740 1 -- 172.21.15.31:0/2100965221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ddc225cab0 con 0x55ddc22ff800 2024-04-25T01:18:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.565+0000 7f370b1f0740 1 -- 172.21.15.31:0/2100965221 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ddc2205640 con 0x55ddc22fa690 2024-04-25T01:18:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.565+0000 7f370b1f0740 1 -- 172.21.15.31:0/2100965221 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ddc2258a70 con 0x55ddc22f9ca0 2024-04-25T01:18:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.565+0000 7f3705000640 1 --2- 172.21.15.31:0/2100965221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddc22ff800 0x55ddc2301c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.566+0000 7f3706400640 1 -- 172.21.15.31:0/2100965221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ddc22fa690 msgr2=0x55ddc22ff190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:18:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.566+0000 7f3706400640 1 --2- 172.21.15.31:0/2100965221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ddc22fa690 0x55ddc22ff190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.566+0000 7f3706400640 1 -- 172.21.15.31:0/2100965221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddc22ff800 msgr2=0x55ddc2301c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.566+0000 7f3706400640 1 --2- 172.21.15.31:0/2100965221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddc22ff800 0x55ddc2301c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.566+0000 7f3706400640 1 -- 172.21.15.31:0/2100965221 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddc1f71210 con 0x55ddc22f9ca0 2024-04-25T01:18:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.566+0000 7f3705000640 1 --2- 172.21.15.31:0/2100965221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddc22ff800 0x55ddc2301c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:18:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.566+0000 7f3706400640 1 --2- 172.21.15.31:0/2100965221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22f9ca0 0x55ddc22fa0c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f36ec02aa00 tx=0x7f36ec004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3b6b0768f1efeff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:18:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.566+0000 7f36f2a00640 1 -- 172.21.15.31:0/2100965221 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec035040 con 0x55ddc22f9ca0 2024-04-25T01:18:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.566+0000 7f36f2a00640 1 -- 172.21.15.31:0/2100965221 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36ec0053f0 con 0x55ddc22f9ca0 2024-04-25T01:18:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.566+0000 7f36f2a00640 1 -- 172.21.15.31:0/2100965221 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec031c40 con 0x55ddc22f9ca0 2024-04-25T01:18:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.567+0000 7f370b1f0740 1 -- 172.21.15.31:0/2100965221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22f9ca0 msgr2=0x55ddc22fa0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.567+0000 7f370b1f0740 1 --2- 172.21.15.31:0/2100965221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22f9ca0 0x55ddc22fa0c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f36ec02aa00 tx=0x7f36ec004610 comp rx=0 tx=0).stop 2024-04-25T01:18:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.567+0000 7f370b1f0740 1 -- 172.21.15.31:0/2100965221 shutdown_connections 2024-04-25T01:18:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.567+0000 7f370b1f0740 1 --2- 172.21.15.31:0/2100965221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddc22ff800 0x55ddc2301c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.567+0000 7f370b1f0740 1 --2- 172.21.15.31:0/2100965221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ddc22fa690 0x55ddc22ff190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.567+0000 7f370b1f0740 1 --2- 172.21.15.31:0/2100965221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22f9ca0 0x55ddc22fa0c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.567+0000 7f370b1f0740 1 -- 172.21.15.31:0/2100965221 >> 172.21.15.31:0/2100965221 conn(0x55ddc22f5310 msgr2=0x55ddc22f5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:18:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.567+0000 7f36fe000640 1 -- 172.21.15.31:0/758831238 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f36f8019b20 con 0x7f37001017a0 2024-04-25T01:18:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.567+0000 7f370b1f0740 1 -- 172.21.15.31:0/2100965221 shutdown_connections 2024-04-25T01:18:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.567+0000 7f370b1f0740 1 -- 172.21.15.31:0/2100965221 wait complete. 2024-04-25T01:18:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.568+0000 7f370b1f0740 1 -- start start 2024-04-25T01:18:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.568+0000 7f370b1f0740 1 Processor -- start 2024-04-25T01:18:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.569+0000 7f36fe000640 1 --2- 172.21.15.31:0/758831238 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f36e80a2c10 0x7f36e80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.569+0000 7f370b1f0740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22f9ca0 0x55ddc23040f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.569+0000 7f370b1f0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddc22ff800 0x55ddc2304630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.569+0000 7f370b1f0740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ddc2309660 0x55ddc230ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.569+0000 7f370b1f0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ddc2302750 con 0x55ddc22ff800 2024-04-25T01:18:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.569+0000 7f370b1f0740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55ddc2218990 con 0x55ddc2309660 2024-04-25T01:18:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.569+0000 7f370b1f0740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ddc23028d0 con 0x55ddc22f9ca0 2024-04-25T01:18:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.569+0000 7f36fe000640 1 -- 172.21.15.31:0/758831238 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f36f80cf7e0 con 0x7f37001017a0 2024-04-25T01:18:38.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.569+0000 7f370b1f0740 10 client.0 _ll_register_callbacks cb 0x55ddc221a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:18:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.570+0000 7f3706400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ddc2309660 0x55ddc230ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.570+0000 7f3705a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22f9ca0 0x55ddc23040f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.570+0000 7f3706400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ddc2309660 0x55ddc230ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:47562/0 (socket says 172.21.15.31:47562) 2024-04-25T01:18:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.570+0000 7f3706400640 1 -- 172.21.15.31:0/3055022363 learned_addr learned my addr 172.21.15.31:0/3055022363 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:18:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.570+0000 7f3705a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22f9ca0 0x55ddc23040f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:55386/0 (socket says 172.21.15.31:55386) 2024-04-25T01:18:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.570+0000 7f3705000640 1 --2- 172.21.15.31:0/3055022363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddc22ff800 0x55ddc2304630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.570+0000 7f3706400640 1 --2- 172.21.15.31:0/758831238 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f36e80a2c10 0x7f36e80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.570+0000 7f3705a00640 1 -- 172.21.15.31:0/3055022363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ddc2309660 msgr2=0x55ddc230ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.570+0000 7f3705a00640 1 --2- 172.21.15.31:0/3055022363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ddc2309660 0x55ddc230ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.570+0000 7f3705a00640 1 -- 172.21.15.31:0/3055022363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddc22ff800 msgr2=0x55ddc2304630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.570+0000 7f3705a00640 1 --2- 172.21.15.31:0/3055022363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddc22ff800 0x55ddc2304630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.570+0000 7f3705a00640 1 -- 172.21.15.31:0/3055022363 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddc203d620 con 0x55ddc22f9ca0 2024-04-25T01:18:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.570+0000 7f3705000640 1 --2- 172.21.15.31:0/3055022363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddc22ff800 0x55ddc2304630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:18:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.571+0000 7f3705a00640 1 --2- 172.21.15.31:0/3055022363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22f9ca0 0x55ddc23040f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55ddc22fb100 tx=0x7f36f809bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad9727be070a762c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:18:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.571+0000 7f36e7400640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:18:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.571+0000 7f36e7400640 1 -- 172.21.15.31:0/3055022363 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f809c030 con 0x55ddc22f9ca0 2024-04-25T01:18:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.571+0000 7f36e7400640 1 -- 172.21.15.31:0/3055022363 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f36f80a00d0 con 0x55ddc22f9ca0 2024-04-25T01:18:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.571+0000 7f370b1f0740 1 -- 172.21.15.31:0/3055022363 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36ec0053f0 con 0x55ddc22f9ca0 2024-04-25T01:18:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.571+0000 7f36e7400640 1 -- 172.21.15.31:0/3055022363 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f36f80a03e0 con 0x55ddc22f9ca0 2024-04-25T01:18:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.571+0000 7f370b1f0740 10 client.25429 fetch_fsmap learned FSMap version 27 2024-04-25T01:18:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.571+0000 7f370b1f0740 1 -- 172.21.15.31:0/3055022363 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ddc230ea10 con 0x55ddc22f9ca0 2024-04-25T01:18:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.572+0000 7f3706400640 1 --2- 172.21.15.31:0/758831238 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f36e80a2c10 0x7f36e80a50d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0314e0 tx=0x7f36ec031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:18:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.572+0000 7f36e7400640 1 -- 172.21.15.31:0/3055022363 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36ec0053f0 con 0x55ddc22f9ca0 2024-04-25T01:18:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.572+0000 7f36e7400640 1 -- 172.21.15.31:0/3055022363 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7f36f8096ba0 con 0x55ddc22f9ca0 2024-04-25T01:18:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.572+0000 7f370b1f0740 10 client.25429 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:18:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.572+0000 7f370b1f0740 10 client.25429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:18:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.572+0000 7f370b1f0740 20 client.25429 populate_metadata read hostname 'smithi031' 2024-04-25T01:18:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.572+0000 7f370b1f0740 1 --2- 172.21.15.31:0/3055022363 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55ddc23100c0 0x55ddc23124b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.572+0000 7f370b1f0740 4 client.25429 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:18:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.572+0000 7f370b1f0740 1 -- 172.21.15.31:0/3055022363 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddc2269ff0 con 0x55ddc23100c0 2024-04-25T01:18:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.572+0000 7f3705000640 1 --2- 172.21.15.31:0/3055022363 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55ddc23100c0 0x55ddc23124b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.573+0000 7f3705000640 1 --2- 172.21.15.31:0/3055022363 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55ddc23100c0 0x55ddc23124b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5afb3614d97342e server_cookie=dfbaa7112f75b58a in_seq=0 out_seq=0 2024-04-25T01:18:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.573+0000 7f36e7400640 10 client.25429 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:18:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.574+0000 7f36e7400640 1 -- 172.21.15.31:0/3055022363 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ddc2269ff0 con 0x55ddc23100c0 2024-04-25T01:18:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.574+0000 7f36e7400640 10 client.25429 handle_command_reply: tid=0 2024-04-25T01:18:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.574+0000 7f370b1f0740 1 client.25429 shutdown 2024-04-25T01:18:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.574+0000 7f370b1f0740 1 -- 172.21.15.31:0/3055022363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22f9ca0 msgr2=0x55ddc23040f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.574+0000 7f370b1f0740 1 --2- 172.21.15.31:0/3055022363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22f9ca0 0x55ddc23040f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55ddc22fb100 tx=0x7f36f809bed0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.574+0000 7f370b1f0740 1 -- 172.21.15.31:0/3055022363 shutdown_connections 2024-04-25T01:18:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.574+0000 7f370b1f0740 1 --2- 172.21.15.31:0/3055022363 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55ddc23100c0 0x55ddc23124b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.574+0000 7f370b1f0740 1 --2- 172.21.15.31:0/3055022363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ddc2309660 0x55ddc230ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.574+0000 7f370b1f0740 1 --2- 172.21.15.31:0/3055022363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddc22ff800 0x55ddc2304630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.574+0000 7f370b1f0740 1 --2- 172.21.15.31:0/3055022363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22f9ca0 0x55ddc23040f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.574+0000 7f370b1f0740 1 -- 172.21.15.31:0/3055022363 >> 172.21.15.31:0/3055022363 conn(0x55ddc22f5310 msgr2=0x55ddc22f4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:18:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.574+0000 7f36e7400640 0 client.25429 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:18:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.574+0000 7f370b1f0740 1 -- 172.21.15.31:0/3055022363 shutdown_connections 2024-04-25T01:18:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.574+0000 7f370b1f0740 1 -- 172.21.15.31:0/3055022363 wait complete. 2024-04-25T01:18:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.575+0000 7f370b1f0740 20 client.25429 trim_cache size 0 max 16384 2024-04-25T01:18:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.608+0000 7f370b1f0740 1 Processor -- start 2024-04-25T01:18:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.608+0000 7f370b1f0740 1 -- start start 2024-04-25T01:18:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.608+0000 7f370b1f0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddc22f9ca0 0x55ddc22fbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.608+0000 7f370b1f0740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22ff800 0x55ddc22fc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.608+0000 7f370b1f0740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ddc2309660 0x55ddc22f8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.608+0000 7f370b1f0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f37001c2890 con 0x55ddc22f9ca0 2024-04-25T01:18:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.608+0000 7f370b1f0740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f37001af0f0 con 0x55ddc2309660 2024-04-25T01:18:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.608+0000 7f370b1f0740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f37001acf40 con 0x55ddc22ff800 2024-04-25T01:18:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.608+0000 7f3706400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ddc2309660 0x55ddc22f8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.608+0000 7f3705a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddc22f9ca0 0x55ddc22fbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.608+0000 7f3706400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ddc2309660 0x55ddc22f8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:47576/0 (socket says 172.21.15.31:47576) 2024-04-25T01:18:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.608+0000 7f3706400640 1 -- 172.21.15.31:0/1901007227 learned_addr learned my addr 172.21.15.31:0/1901007227 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:18:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.609+0000 7f3705000640 1 --2- 172.21.15.31:0/1901007227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22ff800 0x55ddc22fc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.609+0000 7f3705a00640 1 -- 172.21.15.31:0/1901007227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22ff800 msgr2=0x55ddc22fc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.609+0000 7f3705a00640 1 --2- 172.21.15.31:0/1901007227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22ff800 0x55ddc22fc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.609+0000 7f3705a00640 1 -- 172.21.15.31:0/1901007227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ddc2309660 msgr2=0x55ddc22f8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.609+0000 7f3705a00640 1 --2- 172.21.15.31:0/1901007227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ddc2309660 0x55ddc22f8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.609+0000 7f3705a00640 1 -- 172.21.15.31:0/1901007227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddc22f42d0 con 0x55ddc22f9ca0 2024-04-25T01:18:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.609+0000 7f3706400640 1 --2- 172.21.15.31:0/1901007227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ddc2309660 0x55ddc22f8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:18:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.609+0000 7f3705a00640 1 --2- 172.21.15.31:0/1901007227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddc22f9ca0 0x55ddc22fbaf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f36f809a2e0 tx=0x7f36f8000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec17125a0d4fe70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:18:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.610+0000 7f36f2000640 1 -- 172.21.15.31:0/1901007227 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f807e070 con 0x55ddc22f9ca0 2024-04-25T01:18:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.610+0000 7f36f2000640 1 -- 172.21.15.31:0/1901007227 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36f8019e10 con 0x55ddc22f9ca0 2024-04-25T01:18:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.610+0000 7f36f2000640 1 -- 172.21.15.31:0/1901007227 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f809b750 con 0x55ddc22f9ca0 2024-04-25T01:18:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.610+0000 7f370b1f0740 1 -- 172.21.15.31:0/1901007227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddc22f9ca0 msgr2=0x55ddc22fbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.610+0000 7f370b1f0740 1 --2- 172.21.15.31:0/1901007227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddc22f9ca0 0x55ddc22fbaf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f36f809a2e0 tx=0x7f36f8000f40 comp rx=0 tx=0).stop 2024-04-25T01:18:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.610+0000 7f370b1f0740 1 -- 172.21.15.31:0/1901007227 shutdown_connections 2024-04-25T01:18:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.610+0000 7f370b1f0740 1 --2- 172.21.15.31:0/1901007227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ddc2309660 0x55ddc22f8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.610+0000 7f370b1f0740 1 --2- 172.21.15.31:0/1901007227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22ff800 0x55ddc22fc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.610+0000 7f370b1f0740 1 --2- 172.21.15.31:0/1901007227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddc22f9ca0 0x55ddc22fbaf0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.610+0000 7f370b1f0740 1 -- 172.21.15.31:0/1901007227 >> 172.21.15.31:0/1901007227 conn(0x55ddc22f5310 msgr2=0x55ddc22f56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:18:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.610+0000 7f370b1f0740 1 -- 172.21.15.31:0/1901007227 shutdown_connections 2024-04-25T01:18:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.611+0000 7f370b1f0740 1 -- 172.21.15.31:0/1901007227 wait complete. 2024-04-25T01:18:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.611+0000 7f370b1f0740 1 -- start start 2024-04-25T01:18:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.611+0000 7f370b1f0740 1 Processor -- start 2024-04-25T01:18:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.611+0000 7f370b1f0740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ddc22f9ca0 0x55ddc23218b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.611+0000 7f370b1f0740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22ff800 0x55ddc2321df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.612+0000 7f370b1f0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddc2309660 0x55ddc2322d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.612+0000 7f370b1f0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f37001ad5b0 con 0x55ddc2309660 2024-04-25T01:18:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.612+0000 7f370b1f0740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f37001aea20 con 0x55ddc22f9ca0 2024-04-25T01:18:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.612+0000 7f370b1f0740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f37001adc80 con 0x55ddc22ff800 2024-04-25T01:18:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.612+0000 7f370b1f0740 10 client.0 _ll_register_callbacks cb 0x7f36f80982f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:18:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.612+0000 7f3705000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22ff800 0x55ddc2321df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.612+0000 7f3705a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ddc22f9ca0 0x55ddc23218b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.612+0000 7f3705000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22ff800 0x55ddc2321df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:55412/0 (socket says 172.21.15.31:55412) 2024-04-25T01:18:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.612+0000 7f3705000640 1 -- 172.21.15.31:0/4167757079 learned_addr learned my addr 172.21.15.31:0/4167757079 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:18:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.612+0000 7f3705a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ddc22f9ca0 0x55ddc23218b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:47586/0 (socket says 172.21.15.31:47586) 2024-04-25T01:18:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.612+0000 7f3705000640 1 -- 172.21.15.31:0/4167757079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ddc22f9ca0 msgr2=0x55ddc23218b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.612+0000 7f3705000640 1 --2- 172.21.15.31:0/4167757079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ddc22f9ca0 0x55ddc23218b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.612+0000 7f3705000640 1 -- 172.21.15.31:0/4167757079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddc2309660 msgr2=0x55ddc2322d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.612+0000 7f3706400640 1 --2- 172.21.15.31:0/4167757079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddc2309660 0x55ddc2322d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.612+0000 7f3705000640 1 --2- 172.21.15.31:0/4167757079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddc2309660 0x55ddc2322d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.612+0000 7f3705000640 1 -- 172.21.15.31:0/4167757079 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddc230bdc0 con 0x55ddc22ff800 2024-04-25T01:18:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.612+0000 7f3705a00640 1 --2- 172.21.15.31:0/4167757079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ddc22f9ca0 0x55ddc23218b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:18:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.612+0000 7f3706400640 1 --2- 172.21.15.31:0/4167757079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddc2309660 0x55ddc2322d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:18:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.613+0000 7f3705000640 1 --2- 172.21.15.31:0/4167757079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22ff800 0x55ddc2321df0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f37001a53e0 tx=0x7f36f401af90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=720efe5a247f5141 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:18:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.613+0000 7f36e7400640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:18:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.613+0000 7f36e7400640 1 -- 172.21.15.31:0/4167757079 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f4004220 con 0x55ddc22ff800 2024-04-25T01:18:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.613+0000 7f36e7400640 1 -- 172.21.15.31:0/4167757079 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f36f40043c0 con 0x55ddc22ff800 2024-04-25T01:18:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.613+0000 7f370b1f0740 1 -- 172.21.15.31:0/4167757079 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36f8019e10 con 0x55ddc22ff800 2024-04-25T01:18:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.613+0000 7f36e7400640 1 -- 172.21.15.31:0/4167757079 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f36f4011650 con 0x55ddc22ff800 2024-04-25T01:18:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.613+0000 7f370b1f0740 10 client.25435 fetch_fsmap learned FSMap version 27 2024-04-25T01:18:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.613+0000 7f370b1f0740 1 -- 172.21.15.31:0/4167757079 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ddc2376810 con 0x55ddc22ff800 2024-04-25T01:18:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.613+0000 7f36e7400640 1 -- 172.21.15.31:0/4167757079 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36f8019e10 con 0x55ddc22ff800 2024-04-25T01:18:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.614+0000 7f36e7400640 1 -- 172.21.15.31:0/4167757079 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7f36f402b050 con 0x55ddc22ff800 2024-04-25T01:18:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.614+0000 7f370b1f0740 10 client.25435 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:18:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.614+0000 7f370b1f0740 10 client.25435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:18:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.614+0000 7f370b1f0740 20 client.25435 populate_metadata read hostname 'smithi031' 2024-04-25T01:18:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.614+0000 7f370b1f0740 1 --2- 172.21.15.31:0/4167757079 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55ddc23772a0 0x55ddc23796c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.614+0000 7f370b1f0740 4 client.25435 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:18:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.614+0000 7f370b1f0740 1 -- 172.21.15.31:0/4167757079 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ddc2264340 con 0x55ddc23772a0 2024-04-25T01:18:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.614+0000 7f3705a00640 1 --2- 172.21.15.31:0/4167757079 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55ddc23772a0 0x55ddc23796c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.615+0000 7f3705a00640 1 --2- 172.21.15.31:0/4167757079 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55ddc23772a0 0x55ddc23796c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a24c6a398474d4ca server_cookie=979d4858cf7edbca in_seq=0 out_seq=0 2024-04-25T01:18:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.615+0000 7f36e7400640 10 client.25435 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:18:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.674+0000 7f36e7400640 1 -- 172.21.15.31:0/4167757079 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ddc2264340 con 0x55ddc23772a0 2024-04-25T01:18:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.674+0000 7f36e7400640 10 client.25435 handle_command_reply: tid=0 2024-04-25T01:18:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.674+0000 7f370b1f0740 1 client.25435 shutdown 2024-04-25T01:18:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.675+0000 7f370b1f0740 1 -- 172.21.15.31:0/4167757079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22ff800 msgr2=0x55ddc2321df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.675+0000 7f370b1f0740 1 --2- 172.21.15.31:0/4167757079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22ff800 0x55ddc2321df0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f37001a53e0 tx=0x7f36f401af90 comp rx=0 tx=0).stop 2024-04-25T01:18:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.675+0000 7f370b1f0740 1 -- 172.21.15.31:0/4167757079 shutdown_connections 2024-04-25T01:18:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.675+0000 7f370b1f0740 1 --2- 172.21.15.31:0/4167757079 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55ddc23772a0 0x55ddc23796c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.675+0000 7f370b1f0740 1 --2- 172.21.15.31:0/4167757079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ddc2309660 0x55ddc2322d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.675+0000 7f370b1f0740 1 --2- 172.21.15.31:0/4167757079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddc22ff800 0x55ddc2321df0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.675+0000 7f370b1f0740 1 --2- 172.21.15.31:0/4167757079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ddc22f9ca0 0x55ddc23218b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.675+0000 7f370b1f0740 1 -- 172.21.15.31:0/4167757079 >> 172.21.15.31:0/4167757079 conn(0x55ddc22f5310 msgr2=0x55ddc22fbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:18:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.675+0000 7f36e7400640 0 client.25435 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:18:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.676+0000 7f370b1f0740 1 -- 172.21.15.31:0/4167757079 shutdown_connections 2024-04-25T01:18:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.676+0000 7f370b1f0740 1 -- 172.21.15.31:0/4167757079 wait complete. 2024-04-25T01:18:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.676+0000 7f370b1f0740 20 client.25435 trim_cache size 0 max 16384 2024-04-25T01:18:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:18:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:18:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:18:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:18:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.679+0000 7f3708400640 1 -- 172.21.15.31:0/758831238 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f36e80a2c10 msgr2=0x7f36e80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.679+0000 7f3708400640 1 --2- 172.21.15.31:0/758831238 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f36e80a2c10 0x7f36e80a50d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0314e0 tx=0x7f36ec031400 comp rx=0 tx=0).stop 2024-04-25T01:18:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.679+0000 7f3708400640 1 -- 172.21.15.31:0/758831238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37001017a0 msgr2=0x7f37001a7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.679+0000 7f3708400640 1 --2- 172.21.15.31:0/758831238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37001017a0 0x7f37001a7460 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f36f800ebc0 tx=0x7f36f800c6a0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.679+0000 7f3708400640 1 -- 172.21.15.31:0/758831238 shutdown_connections 2024-04-25T01:18:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.679+0000 7f3708400640 1 --2- 172.21.15.31:0/758831238 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3700103350 0x7f37001a3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.679+0000 7f3708400640 1 --2- 172.21.15.31:0/758831238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3700102990 0x7f37001a79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.679+0000 7f3708400640 1 --2- 172.21.15.31:0/758831238 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f36e80a2c10 0x7f36e80a50d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.679+0000 7f3708400640 1 --2- 172.21.15.31:0/758831238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37001017a0 0x7f37001a7460 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.679+0000 7f3708400640 1 -- 172.21.15.31:0/758831238 >> 172.21.15.31:0/758831238 conn(0x7f37000fcd10 msgr2=0x7f3700105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:18:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.680+0000 7f3708400640 1 -- 172.21.15.31:0/758831238 shutdown_connections 2024-04-25T01:18:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:38.680+0000 7f3708400640 1 -- 172.21.15.31:0/758831238 wait complete. 2024-04-25T01:18:39.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:38 smithi031 ceph-mon[28053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:18:39.192 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:38 smithi169 ceph-mon[32759]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:18:39.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:38 smithi040 ceph-mon[32518]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:18:39.295 INFO:tasks.workunit.client.0.smithi031.stdout:Syncing()...10 sec 2024-04-25T01:18:39.298 INFO:tasks.workunit.client.0.smithi031.stdout:Starting Actual Benchmark At: Thu Apr 25 01:18:39 2024 2024-04-25T01:18:39.298 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-04-25T01:18:40.008 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:39 smithi031 ceph-mon[28053]: pgmap v472: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 268 MiB/s wr, 91 op/s 2024-04-25T01:18:40.191 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:39 smithi169 ceph-mon[32759]: pgmap v472: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 268 MiB/s wr, 91 op/s 2024-04-25T01:18:40.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:39 smithi040 ceph-mon[32518]: pgmap v472: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 268 MiB/s wr, 91 op/s 2024-04-25T01:18:41.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:41 smithi040 ceph-mon[32518]: pgmap v473: 97 pgs: 97 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 289 MiB/s wr, 103 op/s 2024-04-25T01:18:42.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:41 smithi031 ceph-mon[28053]: pgmap v473: 97 pgs: 97 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 289 MiB/s wr, 103 op/s 2024-04-25T01:18:42.191 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:41 smithi169 ceph-mon[32759]: pgmap v473: 97 pgs: 97 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 289 MiB/s wr, 103 op/s 2024-04-25T01:18:44.191 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:43 smithi169 ceph-mon[32759]: pgmap v474: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 257 MiB/s wr, 142 op/s 2024-04-25T01:18:44.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:43 smithi040 ceph-mon[32518]: pgmap v474: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 257 MiB/s wr, 142 op/s 2024-04-25T01:18:44.252 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:43 smithi031 ceph-mon[28053]: pgmap v474: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 257 MiB/s wr, 142 op/s 2024-04-25T01:18:46.191 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:45 smithi169 ceph-mon[32759]: pgmap v475: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 230 MiB/s wr, 1.55k op/s 2024-04-25T01:18:46.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:45 smithi040 ceph-mon[32518]: pgmap v475: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 230 MiB/s wr, 1.55k op/s 2024-04-25T01:18:46.252 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:45 smithi031 ceph-mon[28053]: pgmap v475: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 230 MiB/s wr, 1.55k op/s 2024-04-25T01:18:48.191 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:47 smithi169 ceph-mon[32759]: pgmap v476: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 222 MiB/s wr, 4.53k op/s 2024-04-25T01:18:48.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:47 smithi040 ceph-mon[32518]: pgmap v476: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 222 MiB/s wr, 4.53k op/s 2024-04-25T01:18:48.252 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:47 smithi031 ceph-mon[28053]: pgmap v476: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 222 MiB/s wr, 4.53k op/s 2024-04-25T01:18:50.191 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:49 smithi169 ceph-mon[32759]: pgmap v477: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 198 MiB/s wr, 8.80k op/s 2024-04-25T01:18:50.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:49 smithi040 ceph-mon[32518]: pgmap v477: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 198 MiB/s wr, 8.80k op/s 2024-04-25T01:18:50.252 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:49 smithi031 ceph-mon[28053]: pgmap v477: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 198 MiB/s wr, 8.80k op/s 2024-04-25T01:18:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.882+0000 7ff506e00640 1 -- 172.21.15.31:0/1511135356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff500101110 msgr2=0x7ff5001019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:51.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.882+0000 7ff506e00640 1 --2- 172.21.15.31:0/1511135356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff500101110 0x7ff5001019b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0009900 tx=0x7ff4f002a3f0 comp rx=0 tx=0).stop 2024-04-25T01:18:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.882+0000 7ff506e00640 1 -- 172.21.15.31:0/1511135356 shutdown_connections 2024-04-25T01:18:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.882+0000 7ff506e00640 1 --2- 172.21.15.31:0/1511135356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff500101110 0x7ff5001019b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.882+0000 7ff506e00640 1 --2- 172.21.15.31:0/1511135356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff500100770 0x7ff500100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.882+0000 7ff506e00640 1 --2- 172.21.15.31:0/1511135356 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5000ff640 0x7ff5000ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.882+0000 7ff506e00640 1 -- 172.21.15.31:0/1511135356 >> 172.21.15.31:0/1511135356 conn(0x7ff5000faf60 msgr2=0x7ff5000fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:18:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.883+0000 7ff506e00640 1 -- 172.21.15.31:0/1511135356 shutdown_connections 2024-04-25T01:18:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.883+0000 7ff506e00640 1 -- 172.21.15.31:0/1511135356 wait complete. 2024-04-25T01:18:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.884+0000 7ff506e00640 1 Processor -- start 2024-04-25T01:18:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.884+0000 7ff506e00640 1 -- start start 2024-04-25T01:18:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.885+0000 7ff506e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5000ff640 0x7ff500199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.885+0000 7ff506e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff500100770 0x7ff50019a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.885+0000 7ff506e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff500101110 0x7ff5001a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.885+0000 7ff506e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff50010f6a0 con 0x7ff500101110 2024-04-25T01:18:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.885+0000 7ff506e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff50010f520 con 0x7ff500100770 2024-04-25T01:18:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.885+0000 7ff506e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff50010f820 con 0x7ff5000ff640 2024-04-25T01:18:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.885+0000 7ff4ffe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5000ff640 0x7ff500199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.885+0000 7ff4ff400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff500100770 0x7ff50019a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.885+0000 7ff4ff400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff500100770 0x7ff50019a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:35272/0 (socket says 172.21.15.31:35272) 2024-04-25T01:18:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.885+0000 7ff4ff400640 1 -- 172.21.15.31:0/1383629045 learned_addr learned my addr 172.21.15.31:0/1383629045 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:18:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.886+0000 7ff4ffe00640 1 -- 172.21.15.31:0/1383629045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff500100770 msgr2=0x7ff50019a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.886+0000 7ff4ffe00640 1 --2- 172.21.15.31:0/1383629045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff500100770 0x7ff50019a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.886+0000 7ff4ffe00640 1 -- 172.21.15.31:0/1383629045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff500101110 msgr2=0x7ff5001a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.886+0000 7ff4ffe00640 1 --2- 172.21.15.31:0/1383629045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff500101110 0x7ff5001a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.886+0000 7ff4ffe00640 1 -- 172.21.15.31:0/1383629045 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5001a1810 con 0x7ff5000ff640 2024-04-25T01:18:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.887+0000 7ff4ffe00640 1 --2- 172.21.15.31:0/1383629045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5000ff640 0x7ff500199bf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800e9f0 tx=0x7ff4f800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:18:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.887+0000 7ff4fcc00640 1 -- 172.21.15.31:0/1383629045 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f800cd80 con 0x7ff5000ff640 2024-04-25T01:18:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.887+0000 7ff4fcc00640 1 -- 172.21.15.31:0/1383629045 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4f8011950 con 0x7ff5000ff640 2024-04-25T01:18:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.887+0000 7ff4fcc00640 1 -- 172.21.15.31:0/1383629045 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8010a40 con 0x7ff5000ff640 2024-04-25T01:18:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.887+0000 7ff506e00640 1 -- 172.21.15.31:0/1383629045 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5001a1b00 con 0x7ff5000ff640 2024-04-25T01:18:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.887+0000 7ff506e00640 1 -- 172.21.15.31:0/1383629045 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5001a2000 con 0x7ff5000ff640 2024-04-25T01:18:51.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.900+0000 7ff509de9740 1 Processor -- start 2024-04-25T01:18:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.904+0000 7ff509de9740 1 -- start start 2024-04-25T01:18:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.905+0000 7ff509de9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dd408ee4b0 0x55dd408ee8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.906+0000 7ff4fcc00640 1 -- 172.21.15.31:0/1383629045 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff4f8011b00 con 0x7ff5000ff640 2024-04-25T01:18:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.908+0000 7ff4fcc00640 1 --2- 172.21.15.31:0/1383629045 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff4e40a2d30 0x7ff4e40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.908+0000 7ff4fcc00640 1 -- 172.21.15.31:0/1383629045 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7ff4f80ceb50 con 0x7ff5000ff640 2024-04-25T01:18:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.910+0000 7ff509de9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd408eeea0 0x55dd408f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.910+0000 7ff509de9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dd408f4010 0x55dd408f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.910+0000 7ff509de9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55dd40848ab0 con 0x55dd408eeea0 2024-04-25T01:18:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.910+0000 7ff509de9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55dd407f1640 con 0x55dd408ee4b0 2024-04-25T01:18:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.910+0000 7ff509de9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55dd40844a70 con 0x55dd408f4010 2024-04-25T01:18:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.911+0000 7ff504e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dd408f4010 0x55dd408f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.911+0000 7ff504e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dd408f4010 0x55dd408f64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:35872/0 (socket says 172.21.15.31:35872) 2024-04-25T01:18:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.911+0000 7ff504e00640 1 -- 172.21.15.31:0/1585903690 learned_addr learned my addr 172.21.15.31:0/1585903690 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:18:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.911+0000 7ff504e00640 1 --2- 172.21.15.31:0/1585903690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dd408ee4b0 0x55dd408ee8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.912+0000 7ff504e00640 1 -- 172.21.15.31:0/1585903690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dd408ee4b0 msgr2=0x55dd408ee8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.912+0000 7ff504e00640 1 --2- 172.21.15.31:0/1585903690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dd408ee4b0 0x55dd408ee8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.912+0000 7ff504e00640 1 -- 172.21.15.31:0/1585903690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd408eeea0 msgr2=0x55dd408f39a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:18:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.912+0000 7ff4ffe00640 1 --2- 172.21.15.31:0/1383629045 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff4e40a2d30 0x7ff4e40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.913+0000 7ff504e00640 1 --2- 172.21.15.31:0/1585903690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd408eeea0 0x55dd408f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.913+0000 7ff504e00640 1 -- 172.21.15.31:0/1585903690 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd4055d210 con 0x55dd408f4010 2024-04-25T01:18:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.914+0000 7ff504e00640 1 --2- 172.21.15.31:0/1585903690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dd408f4010 0x55dd408f64a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0009a20 tx=0x7ff4f0005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8de34919521d8af5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:18:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.914+0000 7ff4ffe00640 1 --2- 172.21.15.31:0/1383629045 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff4e40a2d30 0x7ff4e40a51f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800cfd0 tx=0x7ff4f8097040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:18:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.914+0000 7ff4f5600640 1 -- 172.21.15.31:0/1585903690 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0035070 con 0x55dd408f4010 2024-04-25T01:18:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.915+0000 7ff4f5600640 1 -- 172.21.15.31:0/1585903690 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4f0004230 con 0x55dd408f4010 2024-04-25T01:18:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.916+0000 7ff509de9740 1 -- 172.21.15.31:0/1585903690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dd408f4010 msgr2=0x55dd408f64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.916+0000 7ff509de9740 1 --2- 172.21.15.31:0/1585903690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dd408f4010 0x55dd408f64a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0009a20 tx=0x7ff4f0005b50 comp rx=0 tx=0).stop 2024-04-25T01:18:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.917+0000 7ff509de9740 1 -- 172.21.15.31:0/1585903690 shutdown_connections 2024-04-25T01:18:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.917+0000 7ff509de9740 1 --2- 172.21.15.31:0/1585903690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dd408f4010 0x55dd408f64a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.917+0000 7ff509de9740 1 --2- 172.21.15.31:0/1585903690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd408eeea0 0x55dd408f39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.917+0000 7ff509de9740 1 --2- 172.21.15.31:0/1585903690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dd408ee4b0 0x55dd408ee8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.917+0000 7ff509de9740 1 -- 172.21.15.31:0/1585903690 >> 172.21.15.31:0/1585903690 conn(0x55dd408e1310 msgr2=0x55dd408e1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:18:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.918+0000 7ff509de9740 1 -- 172.21.15.31:0/1585903690 shutdown_connections 2024-04-25T01:18:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.918+0000 7ff509de9740 1 -- 172.21.15.31:0/1585903690 wait complete. 2024-04-25T01:18:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.918+0000 7ff509de9740 1 -- start start 2024-04-25T01:18:51.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.920+0000 7ff509de9740 1 Processor -- start 2024-04-25T01:18:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.921+0000 7ff509de9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dd408ee4b0 0x55dd408f8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.921+0000 7ff509de9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd408eeea0 0x55dd408f8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.921+0000 7ff509de9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dd408fdbe0 0x55dd408fffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.921+0000 7ff509de9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55dd408f7000 con 0x55dd408eeea0 2024-04-25T01:18:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.921+0000 7ff509de9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55dd40804990 con 0x55dd408ee4b0 2024-04-25T01:18:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.921+0000 7ff509de9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55dd408f7180 con 0x55dd408fdbe0 2024-04-25T01:18:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.921+0000 7ff509de9740 10 client.0 _ll_register_callbacks cb 0x55dd40806950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:18:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.923+0000 7ff4ff400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd408eeea0 0x55dd408f8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.923+0000 7ff4ff400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd408eeea0 0x55dd408f8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51344/0 (socket says 172.21.15.31:51344) 2024-04-25T01:18:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.923+0000 7ff4ff400640 1 -- 172.21.15.31:0/828770055 learned_addr learned my addr 172.21.15.31:0/828770055 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:18:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.923+0000 7ff504e00640 1 --2- 172.21.15.31:0/828770055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dd408ee4b0 0x55dd408f8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.923+0000 7ff504e00640 1 --2- 172.21.15.31:0/828770055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dd408fdbe0 0x55dd408fffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.924+0000 7ff4ff400640 1 -- 172.21.15.31:0/828770055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dd408fdbe0 msgr2=0x55dd408fffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.924+0000 7ff4ff400640 1 --2- 172.21.15.31:0/828770055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dd408fdbe0 0x55dd408fffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.924+0000 7ff4ff400640 1 -- 172.21.15.31:0/828770055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dd408ee4b0 msgr2=0x55dd408f8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.924+0000 7ff4ff400640 1 --2- 172.21.15.31:0/828770055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dd408ee4b0 0x55dd408f8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.924+0000 7ff4ff400640 1 -- 172.21.15.31:0/828770055 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd40629620 con 0x55dd408eeea0 2024-04-25T01:18:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.925+0000 7ff4ff400640 1 --2- 172.21.15.31:0/828770055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd408eeea0 0x55dd408f8bb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00e480 tx=0x7ff4ec00e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611f38ce0d19816a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:18:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.926+0000 7ff4ea000640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T01:18:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.927+0000 7ff4ea000640 1 -- 172.21.15.31:0/828770055 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec017070 con 0x55dd408eeea0 2024-04-25T01:18:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.927+0000 7ff4ea000640 1 -- 172.21.15.31:0/828770055 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7ff4ec013680 con 0x55dd408eeea0 2024-04-25T01:18:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.928+0000 7ff509de9740 1 -- 172.21.15.31:0/828770055 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dd40855ff0 con 0x55dd408eeea0 2024-04-25T01:18:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.931+0000 7ff4ea000640 1 -- 172.21.15.31:0/828770055 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4ec021050 con 0x55dd408eeea0 2024-04-25T01:18:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.931+0000 7ff509de9740 10 client.15270 fetch_fsmap learned FSMap version 27 2024-04-25T01:18:51.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.931+0000 7ff509de9740 1 -- 172.21.15.31:0/828770055 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dd40903110 con 0x55dd408eeea0 2024-04-25T01:18:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.933+0000 7ff4ea000640 1 -- 172.21.15.31:0/828770055 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55dd40855ff0 con 0x55dd408eeea0 2024-04-25T01:18:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.934+0000 7ff4ea000640 1 -- 172.21.15.31:0/828770055 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7ff4ec01cb90 con 0x55dd408eeea0 2024-04-25T01:18:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.934+0000 7ff509de9740 10 client.15270 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:18:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.934+0000 7ff509de9740 10 client.15270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:18:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.934+0000 7ff509de9740 20 client.15270 populate_metadata read hostname 'smithi031' 2024-04-25T01:18:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.934+0000 7ff509de9740 1 --2- 172.21.15.31:0/828770055 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55dd40904ca0 0x55dd40907100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.934+0000 7ff509de9740 4 client.15270 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:18:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.934+0000 7ff509de9740 1 -- 172.21.15.31:0/828770055 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd40850340 con 0x55dd40904ca0 2024-04-25T01:18:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.936+0000 7ff504e00640 1 --2- 172.21.15.31:0/828770055 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55dd40904ca0 0x55dd40907100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.937+0000 7ff4ea000640 10 client.15270 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:18:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.937+0000 7ff504e00640 1 --2- 172.21.15.31:0/828770055 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55dd40904ca0 0x55dd40907100 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21d1bc969a222cfe server_cookie=a913d078036a654f in_seq=0 out_seq=0 2024-04-25T01:18:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.938+0000 7ff4ea000640 1 -- 172.21.15.31:0/828770055 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dd40850340 con 0x55dd40904ca0 2024-04-25T01:18:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.938+0000 7ff4ea000640 10 client.15270 handle_command_reply: tid=0 2024-04-25T01:18:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.938+0000 7ff509de9740 1 client.15270 shutdown 2024-04-25T01:18:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.939+0000 7ff509de9740 1 -- 172.21.15.31:0/828770055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd408eeea0 msgr2=0x55dd408f8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.939+0000 7ff509de9740 1 --2- 172.21.15.31:0/828770055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd408eeea0 0x55dd408f8bb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00e480 tx=0x7ff4ec00e940 comp rx=0 tx=0).stop 2024-04-25T01:18:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.939+0000 7ff509de9740 1 -- 172.21.15.31:0/828770055 shutdown_connections 2024-04-25T01:18:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.939+0000 7ff509de9740 1 --2- 172.21.15.31:0/828770055 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55dd40904ca0 0x55dd40907100 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.939+0000 7ff509de9740 1 --2- 172.21.15.31:0/828770055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dd408fdbe0 0x55dd408fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.939+0000 7ff4ea000640 0 client.15270 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:18:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.939+0000 7ff509de9740 1 --2- 172.21.15.31:0/828770055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd408eeea0 0x55dd408f8bb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.939+0000 7ff509de9740 1 --2- 172.21.15.31:0/828770055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dd408ee4b0 0x55dd408f8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.939+0000 7ff509de9740 1 -- 172.21.15.31:0/828770055 >> 172.21.15.31:0/828770055 conn(0x55dd408e1310 msgr2=0x55dd408e0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:18:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.940+0000 7ff509de9740 1 -- 172.21.15.31:0/828770055 shutdown_connections 2024-04-25T01:18:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.940+0000 7ff509de9740 1 -- 172.21.15.31:0/828770055 wait complete. 2024-04-25T01:18:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:51.942+0000 7ff509de9740 20 client.15270 trim_cache size 0 max 16384 2024-04-25T01:18:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.002+0000 7ff509de9740 1 Processor -- start 2024-04-25T01:18:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.002+0000 7ff509de9740 1 -- start start 2024-04-25T01:18:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.003+0000 7ff509de9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dd408ee4b0 0x55dd408f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.003+0000 7ff509de9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd408eeea0 0x55dd408dc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.003+0000 7ff509de9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dd408fdbe0 0x55dd408effc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.003+0000 7ff509de9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff5001a9280 con 0x55dd408eeea0 2024-04-25T01:18:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.003+0000 7ff509de9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff5001a6af0 con 0x55dd408ee4b0 2024-04-25T01:18:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.003+0000 7ff509de9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff5001a4960 con 0x55dd408fdbe0 2024-04-25T01:18:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.004+0000 7ff504e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dd408fdbe0 0x55dd408effc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.004+0000 7ff504e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dd408fdbe0 0x55dd408effc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:35888/0 (socket says 172.21.15.31:35888) 2024-04-25T01:18:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.004+0000 7ff4ff400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd408eeea0 0x55dd408dc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.005+0000 7ff504e00640 1 -- 172.21.15.31:0/1933247406 learned_addr learned my addr 172.21.15.31:0/1933247406 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:18:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.005+0000 7ff504e00640 1 --2- 172.21.15.31:0/1933247406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dd408ee4b0 0x55dd408f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.006+0000 7ff504e00640 1 -- 172.21.15.31:0/1933247406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dd408ee4b0 msgr2=0x55dd408f6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.006+0000 7ff504e00640 1 --2- 172.21.15.31:0/1933247406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dd408ee4b0 0x55dd408f6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.006+0000 7ff504e00640 1 -- 172.21.15.31:0/1933247406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd408eeea0 msgr2=0x55dd408dc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.007+0000 7ff504e00640 1 --2- 172.21.15.31:0/1933247406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd408eeea0 0x55dd408dc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.007+0000 7ff504e00640 1 -- 172.21.15.31:0/1933247406 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd40902f00 con 0x55dd408fdbe0 2024-04-25T01:18:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.007+0000 7ff4ff400640 1 --2- 172.21.15.31:0/1933247406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd408eeea0 0x55dd408dc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:18:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.010+0000 7ff504e00640 1 --2- 172.21.15.31:0/1933247406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dd408fdbe0 0x55dd408effc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00099f0 tx=0x7ff4f0009660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dff3080c1976f4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:18:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.010+0000 7ff4f4c00640 1 -- 172.21.15.31:0/1933247406 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0044040 con 0x55dd408fdbe0 2024-04-25T01:18:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.010+0000 7ff4f4c00640 1 -- 172.21.15.31:0/1933247406 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4f0003cd0 con 0x55dd408fdbe0 2024-04-25T01:18:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.010+0000 7ff4f4c00640 1 -- 172.21.15.31:0/1933247406 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f003cc90 con 0x55dd408fdbe0 2024-04-25T01:18:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.012+0000 7ff509de9740 1 -- 172.21.15.31:0/1933247406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dd408fdbe0 msgr2=0x55dd408effc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.013+0000 7ff509de9740 1 --2- 172.21.15.31:0/1933247406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dd408fdbe0 0x55dd408effc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00099f0 tx=0x7ff4f0009660 comp rx=0 tx=0).stop 2024-04-25T01:18:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.015+0000 7ff509de9740 1 -- 172.21.15.31:0/1933247406 shutdown_connections 2024-04-25T01:18:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.015+0000 7ff509de9740 1 --2- 172.21.15.31:0/1933247406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dd408fdbe0 0x55dd408effc0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.015+0000 7ff509de9740 1 --2- 172.21.15.31:0/1933247406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd408eeea0 0x55dd408dc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.015+0000 7ff509de9740 1 --2- 172.21.15.31:0/1933247406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dd408ee4b0 0x55dd408f6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.015+0000 7ff509de9740 1 -- 172.21.15.31:0/1933247406 >> 172.21.15.31:0/1933247406 conn(0x55dd408e1310 msgr2=0x55dd408e16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:18:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.015+0000 7ff509de9740 1 -- 172.21.15.31:0/1933247406 shutdown_connections 2024-04-25T01:18:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.015+0000 7ff509de9740 1 -- 172.21.15.31:0/1933247406 wait complete. 2024-04-25T01:18:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.016+0000 7ff509de9740 1 -- start start 2024-04-25T01:18:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.016+0000 7ff509de9740 1 Processor -- start 2024-04-25T01:18:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.019+0000 7ff509de9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dd408ee4b0 0x55dd40915980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.022+0000 7ff504e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dd408ee4b0 0x55dd40915980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.023+0000 7ff504e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dd408ee4b0 0x55dd40915980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:35902/0 (socket says 172.21.15.31:35902) 2024-04-25T01:18:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.023+0000 7ff504e00640 1 -- 172.21.15.31:0/362563124 learned_addr learned my addr 172.21.15.31:0/362563124 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:18:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.023+0000 7ff509de9740 1 --2- 172.21.15.31:0/362563124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd408eeea0 0x55dd40915ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.023+0000 7ff509de9740 1 --2- 172.21.15.31:0/362563124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dd408fdbe0 0x55dd40916e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.024+0000 7ff509de9740 1 -- 172.21.15.31:0/362563124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff5001a4fb0 con 0x55dd408eeea0 2024-04-25T01:18:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.024+0000 7ff509de9740 1 -- 172.21.15.31:0/362563124 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff5001a6420 con 0x55dd408fdbe0 2024-04-25T01:18:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.024+0000 7ff509de9740 1 -- 172.21.15.31:0/362563124 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff5001a5680 con 0x55dd408ee4b0 2024-04-25T01:18:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.026+0000 7ff4ff400640 1 --2- 172.21.15.31:0/362563124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd408eeea0 0x55dd40915ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:52.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.026+0000 7ff509de9740 10 client.0 _ll_register_callbacks cb 0x7ff4ec009450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:18:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.028+0000 7ff504e00640 1 --2- 172.21.15.31:0/362563124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dd408fdbe0 0x55dd40916e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.028+0000 7ff4ff400640 1 -- 172.21.15.31:0/362563124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dd408ee4b0 msgr2=0x55dd40915980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.028+0000 7ff4ff400640 1 --2- 172.21.15.31:0/362563124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dd408ee4b0 0x55dd40915980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.028+0000 7ff4ff400640 1 -- 172.21.15.31:0/362563124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dd408fdbe0 msgr2=0x55dd40916e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:18:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.028+0000 7ff4ff400640 1 --2- 172.21.15.31:0/362563124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dd408fdbe0 0x55dd40916e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.028+0000 7ff4ff400640 1 -- 172.21.15.31:0/362563124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd40917470 con 0x55dd408eeea0 2024-04-25T01:18:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.028+0000 7ff504e00640 1 --2- 172.21.15.31:0/362563124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dd408fdbe0 0x55dd40916e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:18:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.029+0000 7ff4ff400640 1 --2- 172.21.15.31:0/362563124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd408eeea0 0x55dd40915ec0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec001eb0 tx=0x7ff4ec005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eecf7c5d34bab43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:18:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.029+0000 7ff4ea000640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T01:18:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.031+0000 7ff4ea000640 1 -- 172.21.15.31:0/362563124 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec017050 con 0x55dd408eeea0 2024-04-25T01:18:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.031+0000 7ff4ea000640 1 -- 172.21.15.31:0/362563124 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7ff4ec012e20 con 0x55dd408eeea0 2024-04-25T01:18:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.031+0000 7ff509de9740 1 -- 172.21.15.31:0/362563124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4f0003cd0 con 0x55dd408eeea0 2024-04-25T01:18:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.032+0000 7ff4ea000640 1 -- 172.21.15.31:0/362563124 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4ec005e40 con 0x55dd408eeea0 2024-04-25T01:18:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.032+0000 7ff509de9740 10 client.15276 fetch_fsmap learned FSMap version 27 2024-04-25T01:18:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.032+0000 7ff509de9740 1 -- 172.21.15.31:0/362563124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dd40918760 con 0x55dd408eeea0 2024-04-25T01:18:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.033+0000 7ff4ea000640 1 -- 172.21.15.31:0/362563124 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4f0003cd0 con 0x55dd408eeea0 2024-04-25T01:18:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.033+0000 7ff4ea000640 1 -- 172.21.15.31:0/362563124 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7ff4ec006130 con 0x55dd408eeea0 2024-04-25T01:18:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.033+0000 7ff509de9740 10 client.15276 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:18:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.033+0000 7ff509de9740 10 client.15276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:18:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.033+0000 7ff509de9740 20 client.15276 populate_metadata read hostname 'smithi031' 2024-04-25T01:18:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.033+0000 7ff509de9740 1 --2- 172.21.15.31:0/362563124 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55dd40904ca0 0x55dd4096d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:18:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.033+0000 7ff509de9740 4 client.15276 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:18:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.034+0000 7ff509de9740 1 -- 172.21.15.31:0/362563124 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dd407ce7a0 con 0x55dd40904ca0 2024-04-25T01:18:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.034+0000 7ff504e00640 1 --2- 172.21.15.31:0/362563124 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55dd40904ca0 0x55dd4096d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:18:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.034+0000 7ff504e00640 1 --2- 172.21.15.31:0/362563124 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55dd40904ca0 0x55dd4096d410 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca544f8e21180913 server_cookie=2a44841e476a3953 in_seq=0 out_seq=0 2024-04-25T01:18:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.035+0000 7ff4ea000640 10 client.15276 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:18:52.191 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:51 smithi169 ceph-mon[32759]: pgmap v478: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 188 MiB/s wr, 14.74k op/s 2024-04-25T01:18:52.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:51 smithi040 ceph-mon[32518]: pgmap v478: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 188 MiB/s wr, 14.74k op/s 2024-04-25T01:18:52.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:51 smithi031 ceph-mon[28053]: pgmap v478: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 188 MiB/s wr, 14.74k op/s 2024-04-25T01:18:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.630+0000 7ff4ea000640 1 -- 172.21.15.31:0/362563124 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dd407ce7a0 con 0x55dd40904ca0 2024-04-25T01:18:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.630+0000 7ff4ea000640 10 client.15276 handle_command_reply: tid=0 2024-04-25T01:18:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.630+0000 7ff509de9740 1 client.15276 shutdown 2024-04-25T01:18:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.630+0000 7ff509de9740 1 -- 172.21.15.31:0/362563124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd408eeea0 msgr2=0x55dd40915ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.631+0000 7ff509de9740 1 --2- 172.21.15.31:0/362563124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd408eeea0 0x55dd40915ec0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec001eb0 tx=0x7ff4ec005c50 comp rx=0 tx=0).stop 2024-04-25T01:18:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.631+0000 7ff509de9740 1 -- 172.21.15.31:0/362563124 shutdown_connections 2024-04-25T01:18:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.631+0000 7ff509de9740 1 --2- 172.21.15.31:0/362563124 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55dd40904ca0 0x55dd4096d410 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.631+0000 7ff509de9740 1 --2- 172.21.15.31:0/362563124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dd408fdbe0 0x55dd40916e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.631+0000 7ff509de9740 1 --2- 172.21.15.31:0/362563124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd408eeea0 0x55dd40915ec0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.631+0000 7ff509de9740 1 --2- 172.21.15.31:0/362563124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dd408ee4b0 0x55dd40915980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.631+0000 7ff509de9740 1 -- 172.21.15.31:0/362563124 >> 172.21.15.31:0/362563124 conn(0x55dd408e1310 msgr2=0x55dd408f8670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:18:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.631+0000 7ff4ea000640 0 client.15276 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:18:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.633+0000 7ff509de9740 1 -- 172.21.15.31:0/362563124 shutdown_connections 2024-04-25T01:18:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.633+0000 7ff509de9740 1 -- 172.21.15.31:0/362563124 wait complete. 2024-04-25T01:18:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.634+0000 7ff509de9740 20 client.15276 trim_cache size 0 max 16384 2024-04-25T01:18:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:18:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:18:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:18:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:18:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.638+0000 7ff506e00640 1 -- 172.21.15.31:0/1383629045 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff4e40a2d30 msgr2=0x7ff4e40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.638+0000 7ff506e00640 1 --2- 172.21.15.31:0/1383629045 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff4e40a2d30 0x7ff4e40a51f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800cfd0 tx=0x7ff4f8097040 comp rx=0 tx=0).stop 2024-04-25T01:18:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.638+0000 7ff506e00640 1 -- 172.21.15.31:0/1383629045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5000ff640 msgr2=0x7ff500199bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:18:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.638+0000 7ff506e00640 1 --2- 172.21.15.31:0/1383629045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5000ff640 0x7ff500199bf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800e9f0 tx=0x7ff4f800eeb0 comp rx=0 tx=0).stop 2024-04-25T01:18:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.638+0000 7ff506e00640 1 -- 172.21.15.31:0/1383629045 shutdown_connections 2024-04-25T01:18:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.638+0000 7ff506e00640 1 --2- 172.21.15.31:0/1383629045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff500101110 0x7ff5001a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.638+0000 7ff506e00640 1 --2- 172.21.15.31:0/1383629045 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff4e40a2d30 0x7ff4e40a51f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.638+0000 7ff506e00640 1 --2- 172.21.15.31:0/1383629045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff500100770 0x7ff50019a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.638+0000 7ff506e00640 1 --2- 172.21.15.31:0/1383629045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5000ff640 0x7ff500199bf0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:18:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.638+0000 7ff506e00640 1 -- 172.21.15.31:0/1383629045 >> 172.21.15.31:0/1383629045 conn(0x7ff5000faf60 msgr2=0x7ff5000fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:18:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.639+0000 7ff506e00640 1 -- 172.21.15.31:0/1383629045 shutdown_connections 2024-04-25T01:18:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:18:52.639+0000 7ff506e00640 1 -- 172.21.15.31:0/1383629045 wait complete. 2024-04-25T01:18:54.191 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:53 smithi169 ceph-mon[32759]: pgmap v479: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 128 MiB/s wr, 15.18k op/s 2024-04-25T01:18:54.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:53 smithi040 ceph-mon[32518]: pgmap v479: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 128 MiB/s wr, 15.18k op/s 2024-04-25T01:18:54.253 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:53 smithi031 ceph-mon[28053]: pgmap v479: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 128 MiB/s wr, 15.18k op/s 2024-04-25T01:18:56.191 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:55 smithi169 ceph-mon[32759]: pgmap v480: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 154 MiB/s wr, 20.79k op/s 2024-04-25T01:18:56.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:55 smithi040 ceph-mon[32518]: pgmap v480: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 154 MiB/s wr, 20.79k op/s 2024-04-25T01:18:56.251 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:55 smithi031 ceph-mon[28053]: pgmap v480: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 154 MiB/s wr, 20.79k op/s 2024-04-25T01:18:58.191 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:57 smithi169 ceph-mon[32759]: pgmap v481: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 151 MiB/s wr, 22.82k op/s 2024-04-25T01:18:58.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:57 smithi040 ceph-mon[32518]: pgmap v481: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 151 MiB/s wr, 22.82k op/s 2024-04-25T01:18:58.253 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:57 smithi031 ceph-mon[28053]: pgmap v481: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 151 MiB/s wr, 22.82k op/s 2024-04-25T01:19:00.191 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:18:59 smithi169 ceph-mon[32759]: pgmap v482: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 149 MiB/s wr, 22.53k op/s 2024-04-25T01:19:00.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:18:59 smithi040 ceph-mon[32518]: pgmap v482: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 149 MiB/s wr, 22.53k op/s 2024-04-25T01:19:00.252 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:18:59 smithi031 ceph-mon[28053]: pgmap v482: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 149 MiB/s wr, 22.53k op/s 2024-04-25T01:19:02.191 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:01 smithi169 ceph-mon[32759]: pgmap v483: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 165 MiB/s wr, 24.24k op/s 2024-04-25T01:19:02.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:01 smithi040 ceph-mon[32518]: pgmap v483: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 165 MiB/s wr, 24.24k op/s 2024-04-25T01:19:02.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:01 smithi031 ceph-mon[28053]: pgmap v483: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 165 MiB/s wr, 24.24k op/s 2024-04-25T01:19:04.191 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:03 smithi169 ceph-mon[32759]: pgmap v484: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 128 MiB/s wr, 18.61k op/s 2024-04-25T01:19:04.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:03 smithi040 ceph-mon[32518]: pgmap v484: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 128 MiB/s wr, 18.61k op/s 2024-04-25T01:19:04.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:03 smithi031 ceph-mon[28053]: pgmap v484: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 128 MiB/s wr, 18.61k op/s 2024-04-25T01:19:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.881+0000 7fc394e00640 1 -- 172.21.15.31:0/3550199403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc388094c80 msgr2=0x7fc3880950a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.881+0000 7fc394e00640 1 --2- 172.21.15.31:0/3550199403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc388094c80 0x7fc3880950a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc380009900 tx=0x7fc38002a410 comp rx=0 tx=0).stop 2024-04-25T01:19:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.881+0000 7fc394e00640 1 -- 172.21.15.31:0/3550199403 shutdown_connections 2024-04-25T01:19:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.882+0000 7fc394e00640 1 --2- 172.21.15.31:0/3550199403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc388096830 0x7fc38809d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.882+0000 7fc394e00640 1 --2- 172.21.15.31:0/3550199403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc388095e70 0x7fc3880962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.882+0000 7fc394e00640 1 --2- 172.21.15.31:0/3550199403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc388094c80 0x7fc3880950a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.882+0000 7fc394e00640 1 -- 172.21.15.31:0/3550199403 >> 172.21.15.31:0/3550199403 conn(0x7fc3880901d0 msgr2=0x7fc388092630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:19:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.883+0000 7fc394e00640 1 -- 172.21.15.31:0/3550199403 shutdown_connections 2024-04-25T01:19:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.883+0000 7fc394e00640 1 -- 172.21.15.31:0/3550199403 wait complete. 2024-04-25T01:19:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.885+0000 7fc394e00640 1 Processor -- start 2024-04-25T01:19:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.885+0000 7fc394e00640 1 -- start start 2024-04-25T01:19:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.887+0000 7fc394e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc388094c80 0x7fc3880a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.887+0000 7fc394e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc388095e70 0x7fc3880a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:05.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.887+0000 7fc394e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc388096830 0x7fc3880ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.888+0000 7fc38e800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc388096830 0x7fc3880ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.888+0000 7fc38e800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc388096830 0x7fc3880ac6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:43944/0 (socket says 172.21.15.31:43944) 2024-04-25T01:19:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.888+0000 7fc38e800640 1 -- 172.21.15.31:0/3636045760 learned_addr learned my addr 172.21.15.31:0/3636045760 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:19:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.888+0000 7fc394e00640 1 -- 172.21.15.31:0/3636045760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc38809f860 con 0x7fc388094c80 2024-04-25T01:19:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.888+0000 7fc394e00640 1 -- 172.21.15.31:0/3636045760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc38809f6e0 con 0x7fc388096830 2024-04-25T01:19:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.888+0000 7fc394e00640 1 -- 172.21.15.31:0/3636045760 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc38809f9e0 con 0x7fc388095e70 2024-04-25T01:19:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.889+0000 7fc38e800640 1 -- 172.21.15.31:0/3636045760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc388095e70 msgr2=0x7fc3880a5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.889+0000 7fc38d400640 1 --2- 172.21.15.31:0/3636045760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc388095e70 0x7fc3880a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.889+0000 7fc38e800640 1 --2- 172.21.15.31:0/3636045760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc388095e70 0x7fc3880a5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.889+0000 7fc38e800640 1 -- 172.21.15.31:0/3636045760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc388094c80 msgr2=0x7fc3880a5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.889+0000 7fc38e800640 1 --2- 172.21.15.31:0/3636045760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc388094c80 0x7fc3880a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.889+0000 7fc38e800640 1 -- 172.21.15.31:0/3636045760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3880acd20 con 0x7fc388096830 2024-04-25T01:19:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.890+0000 7fc38d400640 1 --2- 172.21.15.31:0/3636045760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc388095e70 0x7fc3880a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:19:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.890+0000 7fc38e800640 1 --2- 172.21.15.31:0/3636045760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc388096830 0x7fc3880ac6b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc39004f7d0 tx=0x7fc39006d590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:19:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.892+0000 7fc386a00640 1 -- 172.21.15.31:0/3636045760 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390073440 con 0x7fc388096830 2024-04-25T01:19:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.892+0000 7fc386a00640 1 -- 172.21.15.31:0/3636045760 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc390073ac0 con 0x7fc388096830 2024-04-25T01:19:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.892+0000 7fc386a00640 1 -- 172.21.15.31:0/3636045760 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390079a80 con 0x7fc388096830 2024-04-25T01:19:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.892+0000 7fc394e00640 1 -- 172.21.15.31:0/3636045760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3880ad010 con 0x7fc388096830 2024-04-25T01:19:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.893+0000 7fc394e00640 1 -- 172.21.15.31:0/3636045760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3880ad300 con 0x7fc388096830 2024-04-25T01:19:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.899+0000 7fc386a00640 1 -- 172.21.15.31:0/3636045760 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc390078070 con 0x7fc388096830 2024-04-25T01:19:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.903+0000 7fc386a00640 1 --2- 172.21.15.31:0/3636045760 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc3740a2c90 0x7fc3740a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.904+0000 7fc386a00640 1 -- 172.21.15.31:0/3636045760 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fc390132780 con 0x7fc388096830 2024-04-25T01:19:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.904+0000 7fc38de00640 1 --2- 172.21.15.31:0/3636045760 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc3740a2c90 0x7fc3740a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.909+0000 7fc397c2f740 1 Processor -- start 2024-04-25T01:19:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.909+0000 7fc397c2f740 1 -- start start 2024-04-25T01:19:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.909+0000 7fc38de00640 1 --2- 172.21.15.31:0/3636045760 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc3740a2c90 0x7fc3740a5150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc3800057c0 tx=0x7fc380031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:19:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.909+0000 7fc397c2f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55671d268e50 0x55671d269270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.909+0000 7fc397c2f740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55671d269840 0x55671d26e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.909+0000 7fc397c2f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55671d26e9b0 0x55671d270e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.909+0000 7fc397c2f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55671d1cbab0 con 0x55671d26e9b0 2024-04-25T01:19:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.909+0000 7fc397c2f740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55671d174640 con 0x55671d269840 2024-04-25T01:19:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.909+0000 7fc397c2f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55671d1c7a70 con 0x55671d268e50 2024-04-25T01:19:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.910+0000 7fc38de00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55671d268e50 0x55671d269270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.910+0000 7fc38de00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55671d268e50 0x55671d269270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:40614/0 (socket says 172.21.15.31:40614) 2024-04-25T01:19:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.910+0000 7fc38de00640 1 -- 172.21.15.31:0/2430084862 learned_addr learned my addr 172.21.15.31:0/2430084862 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:19:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.910+0000 7fc38d400640 1 --2- 172.21.15.31:0/2430084862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55671d269840 0x55671d26e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.910+0000 7fc38de00640 1 -- 172.21.15.31:0/2430084862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55671d269840 msgr2=0x55671d26e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.910+0000 7fc38de00640 1 --2- 172.21.15.31:0/2430084862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55671d269840 0x55671d26e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.910+0000 7fc38de00640 1 -- 172.21.15.31:0/2430084862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55671d26e9b0 msgr2=0x55671d270e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.910+0000 7fc38e800640 1 --2- 172.21.15.31:0/2430084862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55671d26e9b0 0x55671d270e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.912+0000 7fc38de00640 1 --2- 172.21.15.31:0/2430084862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55671d26e9b0 0x55671d270e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.912+0000 7fc38de00640 1 -- 172.21.15.31:0/2430084862 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55671cee0210 con 0x55671d268e50 2024-04-25T01:19:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.912+0000 7fc38e800640 1 --2- 172.21.15.31:0/2430084862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55671d26e9b0 0x55671d270e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:19:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.912+0000 7fc38de00640 1 --2- 172.21.15.31:0/2430084862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55671d268e50 0x55671d269270 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc380033000 tx=0x7fc38003a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84964ddd3d4cec6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:19:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.913+0000 7fc37b400640 1 -- 172.21.15.31:0/2430084862 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380040070 con 0x55671d268e50 2024-04-25T01:19:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.913+0000 7fc37b400640 1 -- 172.21.15.31:0/2430084862 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc38006e070 con 0x55671d268e50 2024-04-25T01:19:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.913+0000 7fc37b400640 1 -- 172.21.15.31:0/2430084862 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380035040 con 0x55671d268e50 2024-04-25T01:19:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.915+0000 7fc397c2f740 1 -- 172.21.15.31:0/2430084862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55671d268e50 msgr2=0x55671d269270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.915+0000 7fc397c2f740 1 --2- 172.21.15.31:0/2430084862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55671d268e50 0x55671d269270 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc380033000 tx=0x7fc38003a000 comp rx=0 tx=0).stop 2024-04-25T01:19:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.916+0000 7fc397c2f740 1 -- 172.21.15.31:0/2430084862 shutdown_connections 2024-04-25T01:19:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.916+0000 7fc397c2f740 1 --2- 172.21.15.31:0/2430084862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55671d26e9b0 0x55671d270e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.916+0000 7fc397c2f740 1 --2- 172.21.15.31:0/2430084862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55671d269840 0x55671d26e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.916+0000 7fc397c2f740 1 --2- 172.21.15.31:0/2430084862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55671d268e50 0x55671d269270 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.916+0000 7fc397c2f740 1 -- 172.21.15.31:0/2430084862 >> 172.21.15.31:0/2430084862 conn(0x55671d2644c0 msgr2=0x55671d2648c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:19:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.916+0000 7fc397c2f740 1 -- 172.21.15.31:0/2430084862 shutdown_connections 2024-04-25T01:19:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.918+0000 7fc397c2f740 1 -- 172.21.15.31:0/2430084862 wait complete. 2024-04-25T01:19:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.921+0000 7fc397c2f740 1 -- start start 2024-04-25T01:19:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.922+0000 7fc397c2f740 1 Processor -- start 2024-04-25T01:19:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.922+0000 7fc397c2f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55671d268e50 0x55671d273070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.922+0000 7fc397c2f740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55671d269840 0x55671d2735b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.922+0000 7fc397c2f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55671d26e9b0 0x55671d27a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.922+0000 7fc397c2f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55671d271900 con 0x55671d268e50 2024-04-25T01:19:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.922+0000 7fc397c2f740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55671d187990 con 0x55671d269840 2024-04-25T01:19:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.922+0000 7fc397c2f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55671d271a80 con 0x55671d26e9b0 2024-04-25T01:19:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.923+0000 7fc397c2f740 10 client.0 _ll_register_callbacks cb 0x55671d189950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:19:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.923+0000 7fc38e800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55671d26e9b0 0x55671d27a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.923+0000 7fc38e800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55671d26e9b0 0x55671d27a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:40628/0 (socket says 172.21.15.31:40628) 2024-04-25T01:19:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.923+0000 7fc38e800640 1 -- 172.21.15.31:0/394990324 learned_addr learned my addr 172.21.15.31:0/394990324 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:19:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.924+0000 7fc38e800640 1 -- 172.21.15.31:0/394990324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55671d269840 msgr2=0x55671d2735b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T01:19:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.924+0000 7fc38e800640 1 --2- 172.21.15.31:0/394990324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55671d269840 0x55671d2735b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.924+0000 7fc38e800640 1 -- 172.21.15.31:0/394990324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55671d268e50 msgr2=0x55671d273070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:19:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.926+0000 7fc38de00640 1 --2- 172.21.15.31:0/394990324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55671d268e50 0x55671d273070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.926+0000 7fc38e800640 1 --2- 172.21.15.31:0/394990324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55671d268e50 0x55671d273070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.926+0000 7fc38e800640 1 -- 172.21.15.31:0/394990324 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55671cfac620 con 0x55671d26e9b0 2024-04-25T01:19:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.929+0000 7fc38e800640 1 --2- 172.21.15.31:0/394990324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55671d26e9b0 0x55671d27a5f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc390050e00 tx=0x7fc390079470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4938401af34fb97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:19:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.929+0000 7fc36fe00640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:19:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.929+0000 7fc36fe00640 1 -- 172.21.15.31:0/394990324 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390117070 con 0x55671d26e9b0 2024-04-25T01:19:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.929+0000 7fc36fe00640 1 -- 172.21.15.31:0/394990324 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fc390130380 con 0x55671d26e9b0 2024-04-25T01:19:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.930+0000 7fc397c2f740 1 -- 172.21.15.31:0/394990324 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc38006e070 con 0x55671d26e9b0 2024-04-25T01:19:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.930+0000 7fc36fe00640 1 -- 172.21.15.31:0/394990324 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc3900e2050 con 0x55671d26e9b0 2024-04-25T01:19:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.930+0000 7fc397c2f740 10 client.25462 fetch_fsmap learned FSMap version 27 2024-04-25T01:19:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.930+0000 7fc397c2f740 1 -- 172.21.15.31:0/394990324 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55671d27d660 con 0x55671d26e9b0 2024-04-25T01:19:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.931+0000 7fc36fe00640 1 -- 172.21.15.31:0/394990324 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc38006e070 con 0x55671d26e9b0 2024-04-25T01:19:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.932+0000 7fc36fe00640 1 -- 172.21.15.31:0/394990324 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7fc3901077b0 con 0x55671d26e9b0 2024-04-25T01:19:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.932+0000 7fc397c2f740 10 client.25462 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:19:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.932+0000 7fc397c2f740 10 client.25462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:19:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.932+0000 7fc397c2f740 20 client.25462 populate_metadata read hostname 'smithi031' 2024-04-25T01:19:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.932+0000 7fc397c2f740 1 --2- 172.21.15.31:0/394990324 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55671d27edf0 0x55671d281240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.932+0000 7fc397c2f740 4 client.25462 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:19:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.932+0000 7fc397c2f740 1 -- 172.21.15.31:0/394990324 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55671d1d8ff0 con 0x55671d27edf0 2024-04-25T01:19:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.935+0000 7fc38de00640 1 --2- 172.21.15.31:0/394990324 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55671d27edf0 0x55671d281240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.936+0000 7fc38de00640 1 --2- 172.21.15.31:0/394990324 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55671d27edf0 0x55671d281240 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f6555a0ee1d8bfd server_cookie=cdb111bb9145762d in_seq=0 out_seq=0 2024-04-25T01:19:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.936+0000 7fc36fe00640 10 client.25462 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:19:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.938+0000 7fc36fe00640 1 -- 172.21.15.31:0/394990324 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55671d1d8ff0 con 0x55671d27edf0 2024-04-25T01:19:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.938+0000 7fc36fe00640 10 client.25462 handle_command_reply: tid=0 2024-04-25T01:19:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.938+0000 7fc397c2f740 1 client.25462 shutdown 2024-04-25T01:19:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.939+0000 7fc397c2f740 1 -- 172.21.15.31:0/394990324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55671d26e9b0 msgr2=0x55671d27a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.939+0000 7fc397c2f740 1 --2- 172.21.15.31:0/394990324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55671d26e9b0 0x55671d27a5f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc390050e00 tx=0x7fc390079470 comp rx=0 tx=0).stop 2024-04-25T01:19:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.940+0000 7fc397c2f740 1 -- 172.21.15.31:0/394990324 shutdown_connections 2024-04-25T01:19:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.940+0000 7fc397c2f740 1 --2- 172.21.15.31:0/394990324 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55671d27edf0 0x55671d281240 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.940+0000 7fc397c2f740 1 --2- 172.21.15.31:0/394990324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55671d26e9b0 0x55671d27a5f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.940+0000 7fc397c2f740 1 --2- 172.21.15.31:0/394990324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55671d269840 0x55671d2735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.940+0000 7fc397c2f740 1 --2- 172.21.15.31:0/394990324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55671d268e50 0x55671d273070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.940+0000 7fc36fe00640 0 client.25462 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:19:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.940+0000 7fc397c2f740 1 -- 172.21.15.31:0/394990324 >> 172.21.15.31:0/394990324 conn(0x55671d2644c0 msgr2=0x55671d263620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:19:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.941+0000 7fc397c2f740 1 -- 172.21.15.31:0/394990324 shutdown_connections 2024-04-25T01:19:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.941+0000 7fc397c2f740 1 -- 172.21.15.31:0/394990324 wait complete. 2024-04-25T01:19:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.941+0000 7fc397c2f740 20 client.25462 trim_cache size 0 max 16384 2024-04-25T01:19:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.990+0000 7fc397c2f740 1 Processor -- start 2024-04-25T01:19:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.990+0000 7fc397c2f740 1 -- start start 2024-04-25T01:19:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.991+0000 7fc397c2f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55671d268e50 0x55671d26aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.991+0000 7fc397c2f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55671d269840 0x55671d27a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.991+0000 7fc397c2f740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55671d26e9b0 0x55671d26afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.991+0000 7fc397c2f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc3880b8790 con 0x55671d269840 2024-04-25T01:19:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.991+0000 7fc397c2f740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc3880b6000 con 0x55671d26e9b0 2024-04-25T01:19:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.991+0000 7fc397c2f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc3880b3e90 con 0x55671d268e50 2024-04-25T01:19:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.991+0000 7fc38de00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55671d268e50 0x55671d26aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.992+0000 7fc38de00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55671d268e50 0x55671d26aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:40638/0 (socket says 172.21.15.31:40638) 2024-04-25T01:19:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.992+0000 7fc38de00640 1 -- 172.21.15.31:0/3786281255 learned_addr learned my addr 172.21.15.31:0/3786281255 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:19:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.992+0000 7fc38e800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55671d26e9b0 0x55671d26afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.993+0000 7fc38e800640 1 -- 172.21.15.31:0/3786281255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55671d268e50 msgr2=0x55671d26aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.993+0000 7fc38e800640 1 --2- 172.21.15.31:0/3786281255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55671d268e50 0x55671d26aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.993+0000 7fc38e800640 1 -- 172.21.15.31:0/3786281255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55671d269840 msgr2=0x55671d27a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.993+0000 7fc38e800640 1 --2- 172.21.15.31:0/3786281255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55671d269840 0x55671d27a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.993+0000 7fc38e800640 1 -- 172.21.15.31:0/3786281255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55671d265190 con 0x55671d26e9b0 2024-04-25T01:19:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.993+0000 7fc38e800640 1 --2- 172.21.15.31:0/3786281255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55671d26e9b0 0x55671d26afa0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc3900fd280 tx=0x7fc3900fd4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d167e6ab331595b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:19:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.994+0000 7fc37aa00640 1 -- 172.21.15.31:0/3786281255 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390117070 con 0x55671d26e9b0 2024-04-25T01:19:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.994+0000 7fc37aa00640 1 -- 172.21.15.31:0/3786281255 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc390107c40 con 0x55671d26e9b0 2024-04-25T01:19:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.994+0000 7fc37aa00640 1 -- 172.21.15.31:0/3786281255 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390117070 con 0x55671d26e9b0 2024-04-25T01:19:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.995+0000 7fc397c2f740 1 -- 172.21.15.31:0/3786281255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55671d26e9b0 msgr2=0x55671d26afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.995+0000 7fc397c2f740 1 --2- 172.21.15.31:0/3786281255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55671d26e9b0 0x55671d26afa0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc3900fd280 tx=0x7fc3900fd4e0 comp rx=0 tx=0).stop 2024-04-25T01:19:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.996+0000 7fc397c2f740 1 -- 172.21.15.31:0/3786281255 shutdown_connections 2024-04-25T01:19:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.996+0000 7fc397c2f740 1 --2- 172.21.15.31:0/3786281255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55671d26e9b0 0x55671d26afa0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.996+0000 7fc397c2f740 1 --2- 172.21.15.31:0/3786281255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55671d269840 0x55671d27a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.996+0000 7fc397c2f740 1 --2- 172.21.15.31:0/3786281255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55671d268e50 0x55671d26aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.996+0000 7fc397c2f740 1 -- 172.21.15.31:0/3786281255 >> 172.21.15.31:0/3786281255 conn(0x55671d2644c0 msgr2=0x55671d25f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:19:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.996+0000 7fc397c2f740 1 -- 172.21.15.31:0/3786281255 shutdown_connections 2024-04-25T01:19:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.996+0000 7fc397c2f740 1 -- 172.21.15.31:0/3786281255 wait complete. 2024-04-25T01:19:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.997+0000 7fc397c2f740 1 -- start start 2024-04-25T01:19:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.997+0000 7fc397c2f740 1 Processor -- start 2024-04-25T01:19:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.998+0000 7fc397c2f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55671d268e50 0x55671d277800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.999+0000 7fc397c2f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55671d269840 0x55671d277d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:05.999+0000 7fc397c2f740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55671d26e9b0 0x55671d278d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.000+0000 7fc397c2f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc3880b44e0 con 0x55671d268e50 2024-04-25T01:19:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.000+0000 7fc38e800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55671d26e9b0 0x55671d278d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.000+0000 7fc38e800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55671d26e9b0 0x55671d278d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:43968/0 (socket says 172.21.15.31:43968) 2024-04-25T01:19:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.000+0000 7fc38e800640 1 -- 172.21.15.31:0/2778051894 learned_addr learned my addr 172.21.15.31:0/2778051894 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:19:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.000+0000 7fc397c2f740 1 -- 172.21.15.31:0/2778051894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc3880b5930 con 0x55671d26e9b0 2024-04-25T01:19:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.000+0000 7fc397c2f740 1 -- 172.21.15.31:0/2778051894 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc3880b4b90 con 0x55671d269840 2024-04-25T01:19:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.000+0000 7fc397c2f740 10 client.0 _ll_register_callbacks cb 0x7fc3900fb290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:19:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.000+0000 7fc38d400640 1 --2- 172.21.15.31:0/2778051894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55671d269840 0x55671d277d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.001+0000 7fc38de00640 1 --2- 172.21.15.31:0/2778051894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55671d268e50 0x55671d277800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.001+0000 7fc38e800640 1 -- 172.21.15.31:0/2778051894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55671d269840 msgr2=0x55671d277d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.001+0000 7fc38e800640 1 --2- 172.21.15.31:0/2778051894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55671d269840 0x55671d277d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.001+0000 7fc38e800640 1 -- 172.21.15.31:0/2778051894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55671d268e50 msgr2=0x55671d277800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.001+0000 7fc38e800640 1 --2- 172.21.15.31:0/2778051894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55671d268e50 0x55671d277800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.001+0000 7fc38e800640 1 -- 172.21.15.31:0/2778051894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55671d279340 con 0x55671d26e9b0 2024-04-25T01:19:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.002+0000 7fc38e800640 1 --2- 172.21.15.31:0/2778051894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55671d26e9b0 0x55671d278d50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc390107710 tx=0x7fc390130590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38560f9052d82e5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:19:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.002+0000 7fc36fe00640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T01:19:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.003+0000 7fc36fe00640 1 -- 172.21.15.31:0/2778051894 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390117070 con 0x55671d26e9b0 2024-04-25T01:19:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.003+0000 7fc397c2f740 1 -- 172.21.15.31:0/2778051894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc390107c40 con 0x55671d26e9b0 2024-04-25T01:19:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.003+0000 7fc36fe00640 1 -- 172.21.15.31:0/2778051894 <== mon.2 v2:172.21.15.40:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fc390073c70 con 0x55671d26e9b0 2024-04-25T01:19:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.004+0000 7fc36fe00640 1 -- 172.21.15.31:0/2778051894 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc3900e8250 con 0x55671d26e9b0 2024-04-25T01:19:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.004+0000 7fc397c2f740 10 client.25442 fetch_fsmap learned FSMap version 27 2024-04-25T01:19:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.004+0000 7fc397c2f740 1 -- 172.21.15.31:0/2778051894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55671d279340 con 0x55671d26e9b0 2024-04-25T01:19:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.004+0000 7fc36fe00640 1 -- 172.21.15.31:0/2778051894 <== mon.2 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc390107c40 con 0x55671d26e9b0 2024-04-25T01:19:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.008+0000 7fc36fe00640 1 -- 172.21.15.31:0/2778051894 <== mon.2 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7fc3900e9ce0 con 0x55671d26e9b0 2024-04-25T01:19:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.009+0000 7fc397c2f740 10 client.25442 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:19:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.009+0000 7fc397c2f740 10 client.25442 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:19:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.009+0000 7fc397c2f740 20 client.25442 populate_metadata read hostname 'smithi031' 2024-04-25T01:19:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.009+0000 7fc397c2f740 1 --2- 172.21.15.31:0/2778051894 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55671d2e54f0 0x55671d2e78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.009+0000 7fc397c2f740 4 client.25442 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:19:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.009+0000 7fc397c2f740 1 -- 172.21.15.31:0/2778051894 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55671d1d3340 con 0x55671d2e54f0 2024-04-25T01:19:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.009+0000 7fc38de00640 1 --2- 172.21.15.31:0/2778051894 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55671d2e54f0 0x55671d2e78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.010+0000 7fc38de00640 1 --2- 172.21.15.31:0/2778051894 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55671d2e54f0 0x55671d2e78e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cccfbdff825306a server_cookie=7d0a15594dd8c0e0 in_seq=0 out_seq=0 2024-04-25T01:19:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.010+0000 7fc36fe00640 10 client.25442 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:19:06.191 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:05 smithi169 ceph-mon[32759]: pgmap v485: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 161 MiB/s wr, 22.93k op/s 2024-04-25T01:19:06.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:05 smithi040 ceph-mon[32518]: pgmap v485: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 161 MiB/s wr, 22.93k op/s 2024-04-25T01:19:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.261+0000 7fc36fe00640 1 -- 172.21.15.31:0/2778051894 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55671d1d3340 con 0x55671d2e54f0 2024-04-25T01:19:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.261+0000 7fc36fe00640 10 client.25442 handle_command_reply: tid=0 2024-04-25T01:19:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.262+0000 7fc397c2f740 1 client.25442 shutdown 2024-04-25T01:19:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.262+0000 7fc397c2f740 1 -- 172.21.15.31:0/2778051894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55671d26e9b0 msgr2=0x55671d278d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.262+0000 7fc397c2f740 1 --2- 172.21.15.31:0/2778051894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55671d26e9b0 0x55671d278d50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc390107710 tx=0x7fc390130590 comp rx=0 tx=0).stop 2024-04-25T01:19:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.262+0000 7fc397c2f740 1 -- 172.21.15.31:0/2778051894 shutdown_connections 2024-04-25T01:19:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.263+0000 7fc397c2f740 1 --2- 172.21.15.31:0/2778051894 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55671d2e54f0 0x55671d2e78e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.263+0000 7fc397c2f740 1 --2- 172.21.15.31:0/2778051894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55671d26e9b0 0x55671d278d50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.263+0000 7fc397c2f740 1 --2- 172.21.15.31:0/2778051894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55671d269840 0x55671d277d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.263+0000 7fc397c2f740 1 --2- 172.21.15.31:0/2778051894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55671d268e50 0x55671d277800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.263+0000 7fc397c2f740 1 -- 172.21.15.31:0/2778051894 >> 172.21.15.31:0/2778051894 conn(0x55671d2644c0 msgr2=0x55671d273050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:19:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.263+0000 7fc36fe00640 0 client.25442 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:19:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.264+0000 7fc397c2f740 1 -- 172.21.15.31:0/2778051894 shutdown_connections 2024-04-25T01:19:06.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.265+0000 7fc397c2f740 1 -- 172.21.15.31:0/2778051894 wait complete. 2024-04-25T01:19:06.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.265+0000 7fc397c2f740 20 client.25442 trim_cache size 0 max 16384 2024-04-25T01:19:06.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:19:06.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:19:06.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:19:06.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:19:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.266+0000 7fc394e00640 1 -- 172.21.15.31:0/3636045760 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc3740a2c90 msgr2=0x7fc3740a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.267+0000 7fc394e00640 1 --2- 172.21.15.31:0/3636045760 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc3740a2c90 0x7fc3740a5150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc3800057c0 tx=0x7fc380031400 comp rx=0 tx=0).stop 2024-04-25T01:19:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.267+0000 7fc394e00640 1 -- 172.21.15.31:0/3636045760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc388096830 msgr2=0x7fc3880ac6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.267+0000 7fc394e00640 1 --2- 172.21.15.31:0/3636045760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc388096830 0x7fc3880ac6b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc39004f7d0 tx=0x7fc39006d590 comp rx=0 tx=0).stop 2024-04-25T01:19:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.267+0000 7fc394e00640 1 -- 172.21.15.31:0/3636045760 shutdown_connections 2024-04-25T01:19:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.268+0000 7fc394e00640 1 --2- 172.21.15.31:0/3636045760 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fc3740a2c90 0x7fc3740a5150 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.268+0000 7fc394e00640 1 --2- 172.21.15.31:0/3636045760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc388096830 0x7fc3880ac6b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.268+0000 7fc394e00640 1 --2- 172.21.15.31:0/3636045760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc388095e70 0x7fc3880a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.268+0000 7fc394e00640 1 --2- 172.21.15.31:0/3636045760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc388094c80 0x7fc3880a5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.268+0000 7fc394e00640 1 -- 172.21.15.31:0/3636045760 >> 172.21.15.31:0/3636045760 conn(0x7fc3880901d0 msgr2=0x7fc388099170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:19:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.268+0000 7fc394e00640 1 -- 172.21.15.31:0/3636045760 shutdown_connections 2024-04-25T01:19:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:06.268+0000 7fc394e00640 1 -- 172.21.15.31:0/3636045760 wait complete. 2024-04-25T01:19:06.253 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:05 smithi031 ceph-mon[28053]: pgmap v485: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 161 MiB/s wr, 22.93k op/s 2024-04-25T01:19:08.191 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:07 smithi169 ceph-mon[32759]: pgmap v486: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 149 MiB/s wr, 20.60k op/s 2024-04-25T01:19:08.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:07 smithi040 ceph-mon[32518]: pgmap v486: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 149 MiB/s wr, 20.60k op/s 2024-04-25T01:19:08.253 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:07 smithi031 ceph-mon[28053]: pgmap v486: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 149 MiB/s wr, 20.60k op/s 2024-04-25T01:19:10.191 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:09 smithi169 ceph-mon[32759]: pgmap v487: 97 pgs: 97 active+clean; 13 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 137 MiB/s wr, 18.55k op/s 2024-04-25T01:19:10.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:09 smithi040 ceph-mon[32518]: pgmap v487: 97 pgs: 97 active+clean; 13 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 137 MiB/s wr, 18.55k op/s 2024-04-25T01:19:10.253 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:09 smithi031 ceph-mon[28053]: pgmap v487: 97 pgs: 97 active+clean; 13 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 137 MiB/s wr, 18.55k op/s 2024-04-25T01:19:11.191 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:10 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:19:11.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:10 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:19:11.252 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:10 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:19:12.191 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:11 smithi169 ceph-mon[32759]: pgmap v488: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 177 MiB/s wr, 23.01k op/s 2024-04-25T01:19:12.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:11 smithi040 ceph-mon[32518]: pgmap v488: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 177 MiB/s wr, 23.01k op/s 2024-04-25T01:19:12.253 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:11 smithi031 ceph-mon[28053]: pgmap v488: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 177 MiB/s wr, 23.01k op/s 2024-04-25T01:19:14.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:13 smithi031 ceph-mon[28053]: pgmap v489: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 136 MiB/s wr, 17.32k op/s 2024-04-25T01:19:14.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:19:14.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:19:14.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:19:14.441 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:13 smithi169 ceph-mon[32759]: pgmap v489: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 136 MiB/s wr, 17.32k op/s 2024-04-25T01:19:14.441 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:19:14.441 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:19:14.441 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:19:14.444 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:13 smithi040 ceph-mon[32518]: pgmap v489: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 136 MiB/s wr, 17.32k op/s 2024-04-25T01:19:14.444 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:19:14.444 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:19:14.444 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:19:16.252 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:15 smithi031 ceph-mon[28053]: pgmap v490: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 151 MiB/s wr, 19.36k op/s 2024-04-25T01:19:16.441 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:15 smithi169 ceph-mon[32759]: pgmap v490: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 151 MiB/s wr, 19.36k op/s 2024-04-25T01:19:16.444 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:15 smithi040 ceph-mon[32518]: pgmap v490: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 151 MiB/s wr, 19.36k op/s 2024-04-25T01:19:18.253 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:17 smithi031 ceph-mon[28053]: pgmap v491: 97 pgs: 97 active+clean; 13 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 114 KiB/s rd, 159 MiB/s wr, 20.28k op/s 2024-04-25T01:19:18.441 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:17 smithi169 ceph-mon[32759]: pgmap v491: 97 pgs: 97 active+clean; 13 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 114 KiB/s rd, 159 MiB/s wr, 20.28k op/s 2024-04-25T01:19:18.444 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:18 smithi040 ceph-mon[32518]: pgmap v491: 97 pgs: 97 active+clean; 13 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 114 KiB/s rd, 159 MiB/s wr, 20.28k op/s 2024-04-25T01:19:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.507+0000 7f02c3000640 1 -- 172.21.15.31:0/3129940919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02bc1090a0 msgr2=0x7f02bc10fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.507+0000 7f02c3000640 1 --2- 172.21.15.31:0/3129940919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02bc1090a0 0x7f02bc10fa80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009900 tx=0x7f02a802a3f0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.508+0000 7f02c3000640 1 -- 172.21.15.31:0/3129940919 shutdown_connections 2024-04-25T01:19:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.508+0000 7f02c3000640 1 --2- 172.21.15.31:0/3129940919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02bc10ffc0 0x7f02bc1123b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.508+0000 7f02c3000640 1 --2- 172.21.15.31:0/3129940919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02bc1090a0 0x7f02bc10fa80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.508+0000 7f02c3000640 1 --2- 172.21.15.31:0/3129940919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02bc108740 0x7f02bc108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.508+0000 7f02c3000640 1 -- 172.21.15.31:0/3129940919 >> 172.21.15.31:0/3129940919 conn(0x7f02bc06e7b0 msgr2=0x7f02bc070bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:19:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.508+0000 7f02c3000640 1 -- 172.21.15.31:0/3129940919 shutdown_connections 2024-04-25T01:19:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.508+0000 7f02c3000640 1 -- 172.21.15.31:0/3129940919 wait complete. 2024-04-25T01:19:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.510+0000 7f02c3000640 1 Processor -- start 2024-04-25T01:19:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.510+0000 7f02c3000640 1 -- start start 2024-04-25T01:19:19.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.511+0000 7f02c3000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02bc108740 0x7f02bc19df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.512+0000 7f02c3000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02bc1090a0 0x7f02bc1a34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.512+0000 7f02c3000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02bc10ffc0 0x7f02bc19e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.512+0000 7f02c3000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc115c40 con 0x7f02bc10ffc0 2024-04-25T01:19:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.512+0000 7f02c3000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc115ac0 con 0x7f02bc1090a0 2024-04-25T01:19:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.512+0000 7f02c3000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc115dc0 con 0x7f02bc108740 2024-04-25T01:19:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.512+0000 7f02bbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02bc108740 0x7f02bc19df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.512+0000 7f02bbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02bc108740 0x7f02bc19df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:38046/0 (socket says 172.21.15.31:38046) 2024-04-25T01:19:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.512+0000 7f02bbe00640 1 -- 172.21.15.31:0/2616351407 learned_addr learned my addr 172.21.15.31:0/2616351407 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:19:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.513+0000 7f02bb400640 1 --2- 172.21.15.31:0/2616351407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02bc1090a0 0x7f02bc1a34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.513+0000 7f02c1000640 1 --2- 172.21.15.31:0/2616351407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02bc10ffc0 0x7f02bc19e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.513+0000 7f02bb400640 1 -- 172.21.15.31:0/2616351407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02bc108740 msgr2=0x7f02bc19df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.513+0000 7f02bb400640 1 --2- 172.21.15.31:0/2616351407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02bc108740 0x7f02bc19df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.513+0000 7f02bb400640 1 -- 172.21.15.31:0/2616351407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02bc10ffc0 msgr2=0x7f02bc19e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.513+0000 7f02bb400640 1 --2- 172.21.15.31:0/2616351407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02bc10ffc0 0x7f02bc19e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.513+0000 7f02bb400640 1 -- 172.21.15.31:0/2616351407 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02bc1b6b60 con 0x7f02bc1090a0 2024-04-25T01:19:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.514+0000 7f02bbe00640 1 --2- 172.21.15.31:0/2616351407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02bc108740 0x7f02bc19df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:19:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.514+0000 7f02bb400640 1 --2- 172.21.15.31:0/2616351407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02bc1090a0 0x7f02bc1a34c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009850 tx=0x7f02a8032da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:19:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.515+0000 7f02b8c00640 1 -- 172.21.15.31:0/2616351407 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a803f030 con 0x7f02bc1090a0 2024-04-25T01:19:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.515+0000 7f02c3000640 1 -- 172.21.15.31:0/2616351407 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02bc1b6df0 con 0x7f02bc1090a0 2024-04-25T01:19:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.515+0000 7f02b8c00640 1 -- 172.21.15.31:0/2616351407 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02a8003ce0 con 0x7f02bc1090a0 2024-04-25T01:19:19.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.517+0000 7f02b8c00640 1 -- 172.21.15.31:0/2616351407 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a803f030 con 0x7f02bc1090a0 2024-04-25T01:19:19.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.518+0000 7f02c3000640 1 -- 172.21.15.31:0/2616351407 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02bc1b7240 con 0x7f02bc1090a0 2024-04-25T01:19:19.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.522+0000 7f02c5d94740 1 Processor -- start 2024-04-25T01:19:19.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.524+0000 7f02b8c00640 1 -- 172.21.15.31:0/2616351407 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f02a8042030 con 0x7f02bc1090a0 2024-04-25T01:19:19.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.526+0000 7f02c5d94740 1 -- start start 2024-04-25T01:19:19.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.526+0000 7f02c5d94740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b1ca0 0x5616137b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:19.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.526+0000 7f02c5d94740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5616137b2690 0x5616137b7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:19.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.526+0000 7f02c5d94740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616137b7800 0x5616137b9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:19.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.526+0000 7f02c1000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b1ca0 0x5616137b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:19.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.526+0000 7f02c1000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b1ca0 0x5616137b20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:38060/0 (socket says 172.21.15.31:38060) 2024-04-25T01:19:19.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.527+0000 7f02c5d94740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561613714ab0 con 0x5616137b7800 2024-04-25T01:19:19.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.527+0000 7f02c5d94740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5616136bd640 con 0x5616137b2690 2024-04-25T01:19:19.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.527+0000 7f02c5d94740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561613710a70 con 0x5616137b1ca0 2024-04-25T01:19:19.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.527+0000 7f02c1000640 1 -- 172.21.15.31:0/245689190 learned_addr learned my addr 172.21.15.31:0/245689190 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:19:19.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.527+0000 7f02bbe00640 1 --2- 172.21.15.31:0/245689190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5616137b2690 0x5616137b7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.527+0000 7f02c1000640 1 -- 172.21.15.31:0/245689190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5616137b2690 msgr2=0x5616137b7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.527+0000 7f02c1000640 1 --2- 172.21.15.31:0/245689190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5616137b2690 0x5616137b7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.527+0000 7f02c1000640 1 -- 172.21.15.31:0/245689190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616137b7800 msgr2=0x5616137b9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.527+0000 7f02c1000640 1 --2- 172.21.15.31:0/245689190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616137b7800 0x5616137b9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.527+0000 7f02c1000640 1 -- 172.21.15.31:0/245689190 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561613429210 con 0x5616137b1ca0 2024-04-25T01:19:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.528+0000 7f02b8c00640 1 --2- 172.21.15.31:0/2616351407 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f02a00a2d10 0x7f02a00a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.528+0000 7f02b8c00640 1 -- 172.21.15.31:0/2616351407 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f02a80f2170 con 0x7f02bc1090a0 2024-04-25T01:19:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.528+0000 7f02c1000640 1 --2- 172.21.15.31:0/2616351407 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f02a00a2d10 0x7f02a00a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.528+0000 7f02c1000640 1 --2- 172.21.15.31:0/245689190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b1ca0 0x5616137b20c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009ea0 tx=0x7f02b4009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4f138719da02248 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:19:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.529+0000 7f02b1600640 1 -- 172.21.15.31:0/245689190 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b4021070 con 0x5616137b1ca0 2024-04-25T01:19:19.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.529+0000 7f02b1600640 1 -- 172.21.15.31:0/245689190 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02b40113e0 con 0x5616137b1ca0 2024-04-25T01:19:19.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.529+0000 7f02b1600640 1 -- 172.21.15.31:0/245689190 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b4016070 con 0x5616137b1ca0 2024-04-25T01:19:19.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.529+0000 7f02c1000640 1 --2- 172.21.15.31:0/2616351407 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f02a00a2d10 0x7f02a00a51d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f02b401e000 tx=0x7f02b4021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:19:19.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.530+0000 7f02c5d94740 1 -- 172.21.15.31:0/245689190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b1ca0 msgr2=0x5616137b20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:19.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.530+0000 7f02c5d94740 1 --2- 172.21.15.31:0/245689190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b1ca0 0x5616137b20c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009ea0 tx=0x7f02b4009f90 comp rx=0 tx=0).stop 2024-04-25T01:19:19.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.530+0000 7f02c5d94740 1 -- 172.21.15.31:0/245689190 shutdown_connections 2024-04-25T01:19:19.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.530+0000 7f02c5d94740 1 --2- 172.21.15.31:0/245689190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616137b7800 0x5616137b9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.531+0000 7f02c5d94740 1 --2- 172.21.15.31:0/245689190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5616137b2690 0x5616137b7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.531+0000 7f02c5d94740 1 --2- 172.21.15.31:0/245689190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b1ca0 0x5616137b20c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.531+0000 7f02c5d94740 1 -- 172.21.15.31:0/245689190 >> 172.21.15.31:0/245689190 conn(0x5616137ad310 msgr2=0x5616137ad710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:19:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.531+0000 7f02c5d94740 1 -- 172.21.15.31:0/245689190 shutdown_connections 2024-04-25T01:19:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.531+0000 7f02c5d94740 1 -- 172.21.15.31:0/245689190 wait complete. 2024-04-25T01:19:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.532+0000 7f02c5d94740 1 -- start start 2024-04-25T01:19:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.532+0000 7f02c5d94740 1 Processor -- start 2024-04-25T01:19:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.533+0000 7f02c5d94740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b1ca0 0x5616137bbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.534+0000 7f02c5d94740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616137b2690 0x5616137bc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:19.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.535+0000 7f02bbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b1ca0 0x5616137bbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:19.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.535+0000 7f02bbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b1ca0 0x5616137bbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:38072/0 (socket says 172.21.15.31:38072) 2024-04-25T01:19:19.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.535+0000 7f02c5d94740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5616137b7800 0x5616137c3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:19.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.535+0000 7f02c5d94740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5616137ba7b0 con 0x5616137b2690 2024-04-25T01:19:19.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.535+0000 7f02c5d94740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5616136d0990 con 0x5616137b7800 2024-04-25T01:19:19.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.535+0000 7f02c5d94740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5616137ba930 con 0x5616137b1ca0 2024-04-25T01:19:19.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.535+0000 7f02c5d94740 10 client.0 _ll_register_callbacks cb 0x5616136d2950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:19:19.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.535+0000 7f02bbe00640 1 -- 172.21.15.31:0/490048087 learned_addr learned my addr 172.21.15.31:0/490048087 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:19:19.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.535+0000 7f02c1000640 1 --2- 172.21.15.31:0/490048087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5616137b7800 0x5616137c3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:19.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.536+0000 7f02bb400640 1 --2- 172.21.15.31:0/490048087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616137b2690 0x5616137bc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:19.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.536+0000 7f02bbe00640 1 -- 172.21.15.31:0/490048087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5616137b7800 msgr2=0x5616137c3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:19.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.536+0000 7f02bbe00640 1 --2- 172.21.15.31:0/490048087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5616137b7800 0x5616137c3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.536+0000 7f02bbe00640 1 -- 172.21.15.31:0/490048087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616137b2690 msgr2=0x5616137bc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:19.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.536+0000 7f02bbe00640 1 --2- 172.21.15.31:0/490048087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616137b2690 0x5616137bc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.536+0000 7f02bbe00640 1 -- 172.21.15.31:0/490048087 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616134f5620 con 0x5616137b1ca0 2024-04-25T01:19:19.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.537+0000 7f02bbe00640 1 --2- 172.21.15.31:0/490048087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b1ca0 0x5616137bbf80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f02ac011710 tx=0x7f02ac011bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6915093323cb75c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:19:19.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.538+0000 7f02a6000640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:19:19.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.539+0000 7f02a6000640 1 -- 172.21.15.31:0/490048087 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ac00fae0 con 0x5616137b1ca0 2024-04-25T01:19:19.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.539+0000 7f02a6000640 1 -- 172.21.15.31:0/490048087 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f02ac00fc80 con 0x5616137b1ca0 2024-04-25T01:19:19.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.539+0000 7f02c5d94740 1 -- 172.21.15.31:0/490048087 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02b40113e0 con 0x5616137b1ca0 2024-04-25T01:19:19.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.539+0000 7f02a6000640 1 -- 172.21.15.31:0/490048087 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f02ac01d480 con 0x5616137b1ca0 2024-04-25T01:19:19.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.540+0000 7f02c5d94740 10 client.25480 fetch_fsmap learned FSMap version 27 2024-04-25T01:19:19.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.540+0000 7f02c5d94740 1 -- 172.21.15.31:0/490048087 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5616137c65b0 con 0x5616137b1ca0 2024-04-25T01:19:19.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.540+0000 7f02a6000640 1 -- 172.21.15.31:0/490048087 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02b40113e0 con 0x5616137b1ca0 2024-04-25T01:19:19.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.540+0000 7f02a6000640 1 -- 172.21.15.31:0/490048087 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7f02ac0126b0 con 0x5616137b1ca0 2024-04-25T01:19:19.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.541+0000 7f02c5d94740 10 client.25480 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:19:19.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.541+0000 7f02c5d94740 10 client.25480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:19:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.541+0000 7f02c5d94740 20 client.25480 populate_metadata read hostname 'smithi031' 2024-04-25T01:19:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.541+0000 7f02c5d94740 1 --2- 172.21.15.31:0/490048087 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5616137c7c30 0x5616137ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.542+0000 7f02c5d94740 4 client.25480 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:19:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.542+0000 7f02bb400640 1 --2- 172.21.15.31:0/490048087 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5616137c7c30 0x5616137ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.542+0000 7f02c5d94740 1 -- 172.21.15.31:0/490048087 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561613721ff0 con 0x5616137c7c30 2024-04-25T01:19:19.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.543+0000 7f02bb400640 1 --2- 172.21.15.31:0/490048087 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5616137c7c30 0x5616137ca0a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11d3f8039f2fcf73 server_cookie=e724e83eb386dccc in_seq=0 out_seq=0 2024-04-25T01:19:19.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.543+0000 7f02a6000640 10 client.25480 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:19:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.548+0000 7f02a6000640 1 -- 172.21.15.31:0/490048087 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561613721ff0 con 0x5616137c7c30 2024-04-25T01:19:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.548+0000 7f02a6000640 10 client.25480 handle_command_reply: tid=0 2024-04-25T01:19:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.548+0000 7f02c5d94740 1 client.25480 shutdown 2024-04-25T01:19:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.549+0000 7f02c5d94740 1 -- 172.21.15.31:0/490048087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b1ca0 msgr2=0x5616137bbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.549+0000 7f02c5d94740 1 --2- 172.21.15.31:0/490048087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b1ca0 0x5616137bbf80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f02ac011710 tx=0x7f02ac011bd0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.549+0000 7f02c5d94740 1 -- 172.21.15.31:0/490048087 shutdown_connections 2024-04-25T01:19:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.549+0000 7f02c5d94740 1 --2- 172.21.15.31:0/490048087 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5616137c7c30 0x5616137ca0a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.549+0000 7f02c5d94740 1 --2- 172.21.15.31:0/490048087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5616137b7800 0x5616137c3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.549+0000 7f02c5d94740 1 --2- 172.21.15.31:0/490048087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616137b2690 0x5616137bc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.549+0000 7f02c5d94740 1 --2- 172.21.15.31:0/490048087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b1ca0 0x5616137bbf80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.549+0000 7f02c5d94740 1 -- 172.21.15.31:0/490048087 >> 172.21.15.31:0/490048087 conn(0x5616137ad310 msgr2=0x5616137ac480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:19:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.549+0000 7f02a6000640 0 client.25480 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:19:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.549+0000 7f02c5d94740 1 -- 172.21.15.31:0/490048087 shutdown_connections 2024-04-25T01:19:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.550+0000 7f02c5d94740 1 -- 172.21.15.31:0/490048087 wait complete. 2024-04-25T01:19:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.550+0000 7f02c5d94740 20 client.25480 trim_cache size 0 max 16384 2024-04-25T01:19:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.582+0000 7f02c5d94740 1 Processor -- start 2024-04-25T01:19:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.584+0000 7f02c5d94740 1 -- start start 2024-04-25T01:19:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.584+0000 7f02c5d94740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616137b1ca0 0x5616137bb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.584+0000 7f02c5d94740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b2690 0x5616137bbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.584+0000 7f02c5d94740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5616137b7800 0x5616137bc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.584+0000 7f02c5d94740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc1c2920 con 0x5616137b1ca0 2024-04-25T01:19:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.584+0000 7f02c5d94740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc1c0190 con 0x5616137b7800 2024-04-25T01:19:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.584+0000 7f02c5d94740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc1bdfe0 con 0x5616137b2690 2024-04-25T01:19:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.587+0000 7f02bb400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b2690 0x5616137bbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.587+0000 7f02c1000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5616137b7800 0x5616137bc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.587+0000 7f02bb400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b2690 0x5616137bbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:38082/0 (socket says 172.21.15.31:38082) 2024-04-25T01:19:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.587+0000 7f02bb400640 1 -- 172.21.15.31:0/193746330 learned_addr learned my addr 172.21.15.31:0/193746330 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:19:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.587+0000 7f02bb400640 1 -- 172.21.15.31:0/193746330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5616137b7800 msgr2=0x5616137bc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.587+0000 7f02bb400640 1 --2- 172.21.15.31:0/193746330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5616137b7800 0x5616137bc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.587+0000 7f02bb400640 1 -- 172.21.15.31:0/193746330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616137b1ca0 msgr2=0x5616137bb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.588+0000 7f02bb400640 1 --2- 172.21.15.31:0/193746330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616137b1ca0 0x5616137bb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.588+0000 7f02bb400640 1 -- 172.21.15.31:0/193746330 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616137ad140 con 0x5616137b2690 2024-04-25T01:19:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.589+0000 7f02c1000640 1 --2- 172.21.15.31:0/193746330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5616137b7800 0x5616137bc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:19:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.591+0000 7f02bb400640 1 --2- 172.21.15.31:0/193746330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b2690 0x5616137bbae0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f02a8033000 tx=0x7f02a8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2cc7dc3d37a6b40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:19:19.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.594+0000 7f02b0c00640 1 -- 172.21.15.31:0/193746330 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a8035040 con 0x5616137b2690 2024-04-25T01:19:19.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.594+0000 7f02b0c00640 1 -- 172.21.15.31:0/193746330 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02a80bb520 con 0x5616137b2690 2024-04-25T01:19:19.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.594+0000 7f02b0c00640 1 -- 172.21.15.31:0/193746330 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a80f3060 con 0x5616137b2690 2024-04-25T01:19:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.595+0000 7f02c5d94740 1 -- 172.21.15.31:0/193746330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b2690 msgr2=0x5616137bbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.595+0000 7f02c5d94740 1 --2- 172.21.15.31:0/193746330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b2690 0x5616137bbae0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f02a8033000 tx=0x7f02a8000bc0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.599+0000 7f02c5d94740 1 -- 172.21.15.31:0/193746330 shutdown_connections 2024-04-25T01:19:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.599+0000 7f02c5d94740 1 --2- 172.21.15.31:0/193746330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5616137b7800 0x5616137bc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.599+0000 7f02c5d94740 1 --2- 172.21.15.31:0/193746330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b2690 0x5616137bbae0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.599+0000 7f02c5d94740 1 --2- 172.21.15.31:0/193746330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616137b1ca0 0x5616137bb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.599+0000 7f02c5d94740 1 -- 172.21.15.31:0/193746330 >> 172.21.15.31:0/193746330 conn(0x5616137ad310 msgr2=0x5616137b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:19:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.599+0000 7f02c5d94740 1 -- 172.21.15.31:0/193746330 shutdown_connections 2024-04-25T01:19:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.599+0000 7f02c5d94740 1 -- 172.21.15.31:0/193746330 wait complete. 2024-04-25T01:19:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.599+0000 7f02c5d94740 1 -- start start 2024-04-25T01:19:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.603+0000 7f02c5d94740 1 Processor -- start 2024-04-25T01:19:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.603+0000 7f02c5d94740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b1ca0 0x5616137c0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.605+0000 7f02bbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b1ca0 0x5616137c0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.605+0000 7f02bbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b1ca0 0x5616137c0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:38096/0 (socket says 172.21.15.31:38096) 2024-04-25T01:19:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.608+0000 7f02c5d94740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616137b2690 0x5616137c0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.608+0000 7f02c5d94740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5616137b7800 0x5616137c1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.608+0000 7f02c5d94740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc1be650 con 0x5616137b2690 2024-04-25T01:19:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.608+0000 7f02c5d94740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc1bfac0 con 0x5616137b7800 2024-04-25T01:19:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.608+0000 7f02c5d94740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc1bed20 con 0x5616137b1ca0 2024-04-25T01:19:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.608+0000 7f02c5d94740 10 client.0 _ll_register_callbacks cb 0x7f02ac00a720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:19:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.609+0000 7f02c1000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5616137b7800 0x5616137c1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.609+0000 7f02c1000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5616137b7800 0x5616137c1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:46190/0 (socket says 172.21.15.31:46190) 2024-04-25T01:19:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.609+0000 7f02c1000640 1 -- 172.21.15.31:0/2846980546 learned_addr learned my addr 172.21.15.31:0/2846980546 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:19:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.613+0000 7f02c1000640 1 -- 172.21.15.31:0/2846980546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b1ca0 msgr2=0x5616137c0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.613+0000 7f02c1000640 1 --2- 172.21.15.31:0/2846980546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b1ca0 0x5616137c0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.613+0000 7f02c1000640 1 -- 172.21.15.31:0/2846980546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616137b2690 msgr2=0x5616137c0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.613+0000 7f02c1000640 1 --2- 172.21.15.31:0/2846980546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616137b2690 0x5616137c0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.613+0000 7f02c1000640 1 -- 172.21.15.31:0/2846980546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616137c2240 con 0x5616137b7800 2024-04-25T01:19:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.614+0000 7f02c1000640 1 --2- 172.21.15.31:0/2846980546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5616137b7800 0x5616137c1c50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009ea0 tx=0x7f02b4012900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=315c7a46174de02a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:19:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.614+0000 7f02a6000640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T01:19:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.614+0000 7f02a6000640 1 -- 172.21.15.31:0/2846980546 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b4016050 con 0x5616137b7800 2024-04-25T01:19:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.615+0000 7f02a6000640 1 -- 172.21.15.31:0/2846980546 <== mon.2 v2:172.21.15.40:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f02b401b990 con 0x5616137b7800 2024-04-25T01:19:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.615+0000 7f02c5d94740 1 -- 172.21.15.31:0/2846980546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02a80bb520 con 0x5616137b7800 2024-04-25T01:19:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.615+0000 7f02a6000640 1 -- 172.21.15.31:0/2846980546 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f02b4009850 con 0x5616137b7800 2024-04-25T01:19:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.616+0000 7f02c5d94740 10 client.25460 fetch_fsmap learned FSMap version 27 2024-04-25T01:19:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.616+0000 7f02c5d94740 1 -- 172.21.15.31:0/2846980546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56161382e8c0 con 0x5616137b7800 2024-04-25T01:19:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.616+0000 7f02a6000640 1 -- 172.21.15.31:0/2846980546 <== mon.2 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02b401bb90 con 0x5616137b7800 2024-04-25T01:19:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.616+0000 7f02a6000640 1 -- 172.21.15.31:0/2846980546 <== mon.2 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7f02b400f040 con 0x5616137b7800 2024-04-25T01:19:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.617+0000 7f02c5d94740 10 client.25460 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:19:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.617+0000 7f02c5d94740 10 client.25460 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:19:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.617+0000 7f02c5d94740 20 client.25460 populate_metadata read hostname 'smithi031' 2024-04-25T01:19:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.617+0000 7f02c5d94740 1 --2- 172.21.15.31:0/2846980546 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x56161382f020 0x561613831440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.618+0000 7f02c5d94740 4 client.25460 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:19:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.618+0000 7f02c5d94740 1 -- 172.21.15.31:0/2846980546 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f02a80bb520 con 0x56161382f020 2024-04-25T01:19:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.618+0000 7f02bbe00640 1 --2- 172.21.15.31:0/2846980546 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x56161382f020 0x561613831440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.619+0000 7f02bbe00640 1 --2- 172.21.15.31:0/2846980546 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x56161382f020 0x561613831440 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29a9d13a71281872 server_cookie=c2eff16921e7dbcc in_seq=0 out_seq=0 2024-04-25T01:19:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:19.619+0000 7f02a6000640 10 client.25460 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:19:20.252 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:20 smithi031 ceph-mon[28053]: pgmap v492: 97 pgs: 97 active+clean; 13 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 77 KiB/s rd, 140 MiB/s wr, 17.64k op/s 2024-04-25T01:19:20.441 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:20 smithi169 ceph-mon[32759]: pgmap v492: 97 pgs: 97 active+clean; 13 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 77 KiB/s rd, 140 MiB/s wr, 17.64k op/s 2024-04-25T01:19:20.444 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:20 smithi040 ceph-mon[32518]: pgmap v492: 97 pgs: 97 active+clean; 13 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 77 KiB/s rd, 140 MiB/s wr, 17.64k op/s 2024-04-25T01:19:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.884+0000 7f02a6000640 1 -- 172.21.15.31:0/2846980546 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f02a80bb520 con 0x56161382f020 2024-04-25T01:19:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.884+0000 7f02a6000640 10 client.25460 handle_command_reply: tid=0 2024-04-25T01:19:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.884+0000 7f02c5d94740 1 client.25460 shutdown 2024-04-25T01:19:20.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.884+0000 7f02c5d94740 1 -- 172.21.15.31:0/2846980546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5616137b7800 msgr2=0x5616137c1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.884+0000 7f02c5d94740 1 --2- 172.21.15.31:0/2846980546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5616137b7800 0x5616137c1c50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009ea0 tx=0x7f02b4012900 comp rx=0 tx=0).stop 2024-04-25T01:19:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.885+0000 7f02c5d94740 1 -- 172.21.15.31:0/2846980546 shutdown_connections 2024-04-25T01:19:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.885+0000 7f02c5d94740 1 --2- 172.21.15.31:0/2846980546 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x56161382f020 0x561613831440 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.885+0000 7f02a6000640 0 client.25460 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:19:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.885+0000 7f02c5d94740 1 --2- 172.21.15.31:0/2846980546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5616137b7800 0x5616137c1c50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.885+0000 7f02c5d94740 1 --2- 172.21.15.31:0/2846980546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616137b2690 0x5616137c0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.885+0000 7f02c5d94740 1 --2- 172.21.15.31:0/2846980546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616137b1ca0 0x5616137c0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.885+0000 7f02c5d94740 1 -- 172.21.15.31:0/2846980546 >> 172.21.15.31:0/2846980546 conn(0x5616137ad310 msgr2=0x5616137afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:19:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.885+0000 7f02c5d94740 1 -- 172.21.15.31:0/2846980546 shutdown_connections 2024-04-25T01:19:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.885+0000 7f02c5d94740 1 -- 172.21.15.31:0/2846980546 wait complete. 2024-04-25T01:19:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.886+0000 7f02c5d94740 20 client.25460 trim_cache size 0 max 16384 2024-04-25T01:19:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:19:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:19:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:19:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:19:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.887+0000 7f02c3000640 1 -- 172.21.15.31:0/2616351407 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f02a00a2d10 msgr2=0x7f02a00a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.887+0000 7f02c3000640 1 --2- 172.21.15.31:0/2616351407 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f02a00a2d10 0x7f02a00a51d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f02b401e000 tx=0x7f02b4021000 comp rx=0 tx=0).stop 2024-04-25T01:19:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.887+0000 7f02c3000640 1 -- 172.21.15.31:0/2616351407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02bc1090a0 msgr2=0x7f02bc1a34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.887+0000 7f02c3000640 1 --2- 172.21.15.31:0/2616351407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02bc1090a0 0x7f02bc1a34c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009850 tx=0x7f02a8032da0 comp rx=0 tx=0).stop 2024-04-25T01:19:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.887+0000 7f02c3000640 1 -- 172.21.15.31:0/2616351407 shutdown_connections 2024-04-25T01:19:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.887+0000 7f02c3000640 1 --2- 172.21.15.31:0/2616351407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02bc10ffc0 0x7f02bc19e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.887+0000 7f02c3000640 1 --2- 172.21.15.31:0/2616351407 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f02a00a2d10 0x7f02a00a51d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.887+0000 7f02c3000640 1 --2- 172.21.15.31:0/2616351407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02bc1090a0 0x7f02bc1a34c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.887+0000 7f02c3000640 1 --2- 172.21.15.31:0/2616351407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02bc108740 0x7f02bc19df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.887+0000 7f02c3000640 1 -- 172.21.15.31:0/2616351407 >> 172.21.15.31:0/2616351407 conn(0x7f02bc06e7b0 msgr2=0x7f02bc109850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:19:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.887+0000 7f02c3000640 1 -- 172.21.15.31:0/2616351407 shutdown_connections 2024-04-25T01:19:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:20.888+0000 7f02c3000640 1 -- 172.21.15.31:0/2616351407 wait complete. 2024-04-25T01:19:22.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:22 smithi031 ceph-mon[28053]: pgmap v493: 97 pgs: 97 active+clean; 13 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 76 KiB/s rd, 174 MiB/s wr, 20.98k op/s 2024-04-25T01:19:22.441 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:22 smithi169 ceph-mon[32759]: pgmap v493: 97 pgs: 97 active+clean; 13 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 76 KiB/s rd, 174 MiB/s wr, 20.98k op/s 2024-04-25T01:19:22.444 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:22 smithi040 ceph-mon[32518]: pgmap v493: 97 pgs: 97 active+clean; 13 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 76 KiB/s rd, 174 MiB/s wr, 20.98k op/s 2024-04-25T01:19:23.441 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:23 smithi169 ceph-mon[32759]: pgmap v494: 97 pgs: 97 active+clean; 13 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 60 KiB/s rd, 146 MiB/s wr, 17.11k op/s 2024-04-25T01:19:23.444 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:23 smithi040 ceph-mon[32518]: pgmap v494: 97 pgs: 97 active+clean; 13 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 60 KiB/s rd, 146 MiB/s wr, 17.11k op/s 2024-04-25T01:19:23.502 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:23 smithi031 ceph-mon[28053]: pgmap v494: 97 pgs: 97 active+clean; 13 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 60 KiB/s rd, 146 MiB/s wr, 17.11k op/s 2024-04-25T01:19:25.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:25 smithi169 ceph-mon[32759]: pgmap v495: 97 pgs: 97 active+clean; 13 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 55 KiB/s rd, 161 MiB/s wr, 18.92k op/s 2024-04-25T01:19:25.694 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:25 smithi040 ceph-mon[32518]: pgmap v495: 97 pgs: 97 active+clean; 13 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 55 KiB/s rd, 161 MiB/s wr, 18.92k op/s 2024-04-25T01:19:25.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:25 smithi031 ceph-mon[28053]: pgmap v495: 97 pgs: 97 active+clean; 13 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 55 KiB/s rd, 161 MiB/s wr, 18.92k op/s 2024-04-25T01:19:27.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:27 smithi169 ceph-mon[32759]: pgmap v496: 97 pgs: 97 active+clean; 13 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 59 KiB/s rd, 181 MiB/s wr, 21.24k op/s 2024-04-25T01:19:27.694 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:27 smithi040 ceph-mon[32518]: pgmap v496: 97 pgs: 97 active+clean; 13 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 59 KiB/s rd, 181 MiB/s wr, 21.24k op/s 2024-04-25T01:19:27.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:27 smithi031 ceph-mon[28053]: pgmap v496: 97 pgs: 97 active+clean; 13 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 59 KiB/s rd, 181 MiB/s wr, 21.24k op/s 2024-04-25T01:19:29.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:29 smithi169 ceph-mon[32759]: pgmap v497: 97 pgs: 97 active+clean; 13 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 30 KiB/s rd, 142 MiB/s wr, 16.27k op/s 2024-04-25T01:19:29.694 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:29 smithi040 ceph-mon[32518]: pgmap v497: 97 pgs: 97 active+clean; 13 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 30 KiB/s rd, 142 MiB/s wr, 16.27k op/s 2024-04-25T01:19:29.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:29 smithi031 ceph-mon[28053]: pgmap v497: 97 pgs: 97 active+clean; 13 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 30 KiB/s rd, 142 MiB/s wr, 16.27k op/s 2024-04-25T01:19:31.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:31 smithi169 ceph-mon[32759]: pgmap v498: 97 pgs: 97 active+clean; 13 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 63 KiB/s rd, 174 MiB/s wr, 21.78k op/s 2024-04-25T01:19:31.694 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:31 smithi040 ceph-mon[32518]: pgmap v498: 97 pgs: 97 active+clean; 13 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 63 KiB/s rd, 174 MiB/s wr, 21.78k op/s 2024-04-25T01:19:31.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:31 smithi031 ceph-mon[28053]: pgmap v498: 97 pgs: 97 active+clean; 13 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 63 KiB/s rd, 174 MiB/s wr, 21.78k op/s 2024-04-25T01:19:33.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:33 smithi169 ceph-mon[32759]: pgmap v499: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 74 KiB/s rd, 143 MiB/s wr, 19.32k op/s 2024-04-25T01:19:33.694 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:33 smithi040 ceph-mon[32518]: pgmap v499: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 74 KiB/s rd, 143 MiB/s wr, 19.32k op/s 2024-04-25T01:19:33.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:33 smithi031 ceph-mon[28053]: pgmap v499: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 74 KiB/s rd, 143 MiB/s wr, 19.32k op/s 2024-04-25T01:19:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.114+0000 7f1039400640 1 -- 172.21.15.31:0/2164827306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1034075720 msgr2=0x7f1034073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.114+0000 7f1039400640 1 --2- 172.21.15.31:0/2164827306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1034075720 0x7f1034073b70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1024009900 tx=0x7f102402a4b0 comp rx=0 tx=0).stop 2024-04-25T01:19:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.115+0000 7f1039400640 1 -- 172.21.15.31:0/2164827306 shutdown_connections 2024-04-25T01:19:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.115+0000 7f1039400640 1 --2- 172.21.15.31:0/2164827306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1034109660 0x7f103410bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.115+0000 7f1039400640 1 --2- 172.21.15.31:0/2164827306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1034074140 0x7f10340745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.115+0000 7f1039400640 1 --2- 172.21.15.31:0/2164827306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1034075720 0x7f1034073b70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.115+0000 7f1039400640 1 -- 172.21.15.31:0/2164827306 >> 172.21.15.31:0/2164827306 conn(0x7f10340fcd60 msgr2=0x7f10340ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:19:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.116+0000 7f1039400640 1 -- 172.21.15.31:0/2164827306 shutdown_connections 2024-04-25T01:19:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.116+0000 7f1039400640 1 -- 172.21.15.31:0/2164827306 wait complete. 2024-04-25T01:19:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.118+0000 7f1039400640 1 Processor -- start 2024-04-25T01:19:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.120+0000 7f1039400640 1 -- start start 2024-04-25T01:19:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.121+0000 7f1039400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1034074140 0x7f1034199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.121+0000 7f1039400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1034075720 0x7f103419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.121+0000 7f1039400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1034109660 0x7f10341a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.121+0000 7f1031e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1034075720 0x7f103419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.122+0000 7f1031e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1034075720 0x7f103419a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:58766/0 (socket says 172.21.15.31:58766) 2024-04-25T01:19:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.123+0000 7f1033200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1034109660 0x7f10341a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.123+0000 7f1033200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1034109660 0x7f10341a1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:58334/0 (socket says 172.21.15.31:58334) 2024-04-25T01:19:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.123+0000 7f1031e00640 1 -- 172.21.15.31:0/3153067094 learned_addr learned my addr 172.21.15.31:0/3153067094 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:19:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.123+0000 7f1039400640 1 -- 172.21.15.31:0/3153067094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f103410f670 con 0x7f1034074140 2024-04-25T01:19:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.123+0000 7f1039400640 1 -- 172.21.15.31:0/3153067094 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f103410f4f0 con 0x7f1034075720 2024-04-25T01:19:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.123+0000 7f1039400640 1 -- 172.21.15.31:0/3153067094 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f103410f7f0 con 0x7f1034109660 2024-04-25T01:19:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.124+0000 7f1032800640 1 --2- 172.21.15.31:0/3153067094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1034074140 0x7f1034199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.124+0000 7f1031e00640 1 -- 172.21.15.31:0/3153067094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1034109660 msgr2=0x7f10341a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.124+0000 7f1031e00640 1 --2- 172.21.15.31:0/3153067094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1034109660 0x7f10341a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.124+0000 7f1031e00640 1 -- 172.21.15.31:0/3153067094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1034074140 msgr2=0x7f1034199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.124+0000 7f1031e00640 1 --2- 172.21.15.31:0/3153067094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1034074140 0x7f1034199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.124+0000 7f1031e00640 1 -- 172.21.15.31:0/3153067094 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10341a1830 con 0x7f1034075720 2024-04-25T01:19:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.124+0000 7f1032800640 1 --2- 172.21.15.31:0/3153067094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1034074140 0x7f1034199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:19:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.124+0000 7f1031e00640 1 --2- 172.21.15.31:0/3153067094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1034075720 0x7f103419a160 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f102000e9f0 tx=0x7f102000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:19:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.125+0000 7f102b400640 1 -- 172.21.15.31:0/3153067094 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102000cd80 con 0x7f1034075720 2024-04-25T01:19:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.125+0000 7f102b400640 1 -- 172.21.15.31:0/3153067094 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1020010910 con 0x7f1034075720 2024-04-25T01:19:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.125+0000 7f102b400640 1 -- 172.21.15.31:0/3153067094 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1020018990 con 0x7f1034075720 2024-04-25T01:19:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.126+0000 7f1039400640 1 -- 172.21.15.31:0/3153067094 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10341a1b20 con 0x7f1034075720 2024-04-25T01:19:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.126+0000 7f1039400640 1 -- 172.21.15.31:0/3153067094 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10341a2020 con 0x7f1034075720 2024-04-25T01:19:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.132+0000 7f103c220740 1 Processor -- start 2024-04-25T01:19:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.137+0000 7f102b400640 1 -- 172.21.15.31:0/3153067094 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1020018b30 con 0x7f1034075720 2024-04-25T01:19:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.137+0000 7f103c220740 1 -- start start 2024-04-25T01:19:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.137+0000 7f103c220740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b1b24ca0 0x5650b1b250c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.137+0000 7f103c220740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5650b1b25690 0x5650b1b2a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.137+0000 7f103c220740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5650b1b2a800 0x5650b1b2cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.137+0000 7f103c220740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5650b1a87ab0 con 0x5650b1b24ca0 2024-04-25T01:19:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.137+0000 7f103c220740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5650b1a30640 con 0x5650b1b25690 2024-04-25T01:19:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.137+0000 7f103c220740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5650b1a83a70 con 0x5650b1b2a800 2024-04-25T01:19:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.138+0000 7f1033200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b1b24ca0 0x5650b1b250c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.138+0000 7f1032800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5650b1b25690 0x5650b1b2a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.138+0000 7f1032800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5650b1b25690 0x5650b1b2a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:58774/0 (socket says 172.21.15.31:58774) 2024-04-25T01:19:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.138+0000 7f1032800640 1 -- 172.21.15.31:0/925247418 learned_addr learned my addr 172.21.15.31:0/925247418 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:19:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.140+0000 7f1033200640 1 -- 172.21.15.31:0/925247418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5650b1b2a800 msgr2=0x5650b1b2cc90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:19:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.140+0000 7f1033200640 1 --2- 172.21.15.31:0/925247418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5650b1b2a800 0x5650b1b2cc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.140+0000 7f1033200640 1 -- 172.21.15.31:0/925247418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5650b1b25690 msgr2=0x5650b1b2a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.140+0000 7f1033200640 1 --2- 172.21.15.31:0/925247418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5650b1b25690 0x5650b1b2a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.140+0000 7f1033200640 1 -- 172.21.15.31:0/925247418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b179c210 con 0x5650b1b24ca0 2024-04-25T01:19:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.144+0000 7f1033200640 1 --2- 172.21.15.31:0/925247418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b1b24ca0 0x5650b1b250c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f102c009e20 tx=0x7f102c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ccd8f1ec3a8fdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:19:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.145+0000 7f102b400640 1 --2- 172.21.15.31:0/3153067094 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f10180a2df0 0x7f10180a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.146+0000 7f102b400640 1 -- 172.21.15.31:0/3153067094 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f10200cf400 con 0x7f1034075720 2024-04-25T01:19:34.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.147+0000 7f1032800640 1 --2- 172.21.15.31:0/3153067094 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f10180a2df0 0x7f10180a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.149+0000 7f1032800640 1 --2- 172.21.15.31:0/3153067094 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f10180a2df0 0x7f10180a52b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1024006010 tx=0x7f1024009560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:19:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.151+0000 7f101f400640 1 -- 172.21.15.31:0/925247418 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c016070 con 0x5650b1b24ca0 2024-04-25T01:19:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.152+0000 7f101f400640 1 -- 172.21.15.31:0/925247418 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f102c004d20 con 0x5650b1b24ca0 2024-04-25T01:19:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.154+0000 7f101f400640 1 -- 172.21.15.31:0/925247418 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c0128b0 con 0x5650b1b24ca0 2024-04-25T01:19:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.154+0000 7f103c220740 1 -- 172.21.15.31:0/925247418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b1b24ca0 msgr2=0x5650b1b250c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.156+0000 7f103c220740 1 --2- 172.21.15.31:0/925247418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b1b24ca0 0x5650b1b250c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f102c009e20 tx=0x7f102c010740 comp rx=0 tx=0).stop 2024-04-25T01:19:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.156+0000 7f103c220740 1 -- 172.21.15.31:0/925247418 shutdown_connections 2024-04-25T01:19:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.156+0000 7f103c220740 1 --2- 172.21.15.31:0/925247418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5650b1b2a800 0x5650b1b2cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:34.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.156+0000 7f103c220740 1 --2- 172.21.15.31:0/925247418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5650b1b25690 0x5650b1b2a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.157+0000 7f103c220740 1 --2- 172.21.15.31:0/925247418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b1b24ca0 0x5650b1b250c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.157+0000 7f103c220740 1 -- 172.21.15.31:0/925247418 >> 172.21.15.31:0/925247418 conn(0x5650b1b20310 msgr2=0x5650b1b20710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:19:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.161+0000 7f103c220740 1 -- 172.21.15.31:0/925247418 shutdown_connections 2024-04-25T01:19:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.161+0000 7f103c220740 1 -- 172.21.15.31:0/925247418 wait complete. 2024-04-25T01:19:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.162+0000 7f103c220740 1 -- start start 2024-04-25T01:19:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.162+0000 7f103c220740 1 Processor -- start 2024-04-25T01:19:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.164+0000 7f103c220740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b1b24ca0 0x5650b1b2f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.164+0000 7f103c220740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5650b1b25690 0x5650b1b2f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.164+0000 7f103c220740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5650b1b2a800 0x5650b1b36550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.164+0000 7f103c220740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5650b1b2d800 con 0x5650b1b24ca0 2024-04-25T01:19:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.164+0000 7f103c220740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5650b1a43990 con 0x5650b1b25690 2024-04-25T01:19:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.164+0000 7f103c220740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5650b1b2d980 con 0x5650b1b2a800 2024-04-25T01:19:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.164+0000 7f103c220740 10 client.0 _ll_register_callbacks cb 0x5650b1a45950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:19:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.165+0000 7f1031e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5650b1b25690 0x5650b1b2f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.165+0000 7f1031e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5650b1b25690 0x5650b1b2f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:58782/0 (socket says 172.21.15.31:58782) 2024-04-25T01:19:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.165+0000 7f1031e00640 1 -- 172.21.15.31:0/1882917846 learned_addr learned my addr 172.21.15.31:0/1882917846 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:19:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.165+0000 7f1033200640 1 --2- 172.21.15.31:0/1882917846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5650b1b2a800 0x5650b1b36550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.165+0000 7f1031e00640 1 -- 172.21.15.31:0/1882917846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5650b1b2a800 msgr2=0x5650b1b36550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.165+0000 7f1031e00640 1 --2- 172.21.15.31:0/1882917846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5650b1b2a800 0x5650b1b36550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.165+0000 7f1031e00640 1 -- 172.21.15.31:0/1882917846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b1b24ca0 msgr2=0x5650b1b2f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.165+0000 7f1031e00640 1 --2- 172.21.15.31:0/1882917846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b1b24ca0 0x5650b1b2f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.165+0000 7f1031e00640 1 -- 172.21.15.31:0/1882917846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b1868620 con 0x5650b1b25690 2024-04-25T01:19:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.166+0000 7f1031e00640 1 --2- 172.21.15.31:0/1882917846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5650b1b25690 0x5650b1b2f580 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1020098630 tx=0x7f1020099e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25e90847b499a55a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:19:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.168+0000 7f1013e00640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T01:19:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.168+0000 7f1013e00640 1 -- 172.21.15.31:0/1882917846 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102009b340 con 0x5650b1b25690 2024-04-25T01:19:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.168+0000 7f1013e00640 1 -- 172.21.15.31:0/1882917846 <== mon.2 v2:172.21.15.40:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f1020010430 con 0x5650b1b25690 2024-04-25T01:19:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.168+0000 7f103c220740 1 -- 172.21.15.31:0/1882917846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f102c004d20 con 0x5650b1b25690 2024-04-25T01:19:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.168+0000 7f1013e00640 1 -- 172.21.15.31:0/1882917846 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f10200a00a0 con 0x5650b1b25690 2024-04-25T01:19:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.168+0000 7f103c220740 10 client.25472 fetch_fsmap learned FSMap version 27 2024-04-25T01:19:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.168+0000 7f103c220740 1 -- 172.21.15.31:0/1882917846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5650b1b395a0 con 0x5650b1b25690 2024-04-25T01:19:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.169+0000 7f1013e00640 1 -- 172.21.15.31:0/1882917846 <== mon.2 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10200a4960 con 0x5650b1b25690 2024-04-25T01:19:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.171+0000 7f1013e00640 1 -- 172.21.15.31:0/1882917846 <== mon.2 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7f102009abb0 con 0x5650b1b25690 2024-04-25T01:19:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.172+0000 7f103c220740 10 client.25472 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:19:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.172+0000 7f103c220740 10 client.25472 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:19:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.172+0000 7f103c220740 20 client.25472 populate_metadata read hostname 'smithi031' 2024-04-25T01:19:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.172+0000 7f103c220740 1 --2- 172.21.15.31:0/1882917846 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5650b1b3abf0 0x5650b1b3d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.173+0000 7f103c220740 4 client.25472 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:19:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.173+0000 7f103c220740 1 -- 172.21.15.31:0/1882917846 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f102c004d20 con 0x5650b1b3abf0 2024-04-25T01:19:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.173+0000 7f1032800640 1 --2- 172.21.15.31:0/1882917846 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5650b1b3abf0 0x5650b1b3d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.175+0000 7f1032800640 1 --2- 172.21.15.31:0/1882917846 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5650b1b3abf0 0x5650b1b3d020 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2493130f5544d826 server_cookie=dc873d92f39d7830 in_seq=0 out_seq=0 2024-04-25T01:19:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.175+0000 7f1013e00640 10 client.25472 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:19:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.176+0000 7f1013e00640 1 -- 172.21.15.31:0/1882917846 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f102c004d20 con 0x5650b1b3abf0 2024-04-25T01:19:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.176+0000 7f1013e00640 10 client.25472 handle_command_reply: tid=0 2024-04-25T01:19:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.176+0000 7f103c220740 1 client.25472 shutdown 2024-04-25T01:19:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.177+0000 7f103c220740 1 -- 172.21.15.31:0/1882917846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5650b1b25690 msgr2=0x5650b1b2f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.177+0000 7f103c220740 1 --2- 172.21.15.31:0/1882917846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5650b1b25690 0x5650b1b2f580 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1020098630 tx=0x7f1020099e30 comp rx=0 tx=0).stop 2024-04-25T01:19:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.178+0000 7f103c220740 1 -- 172.21.15.31:0/1882917846 shutdown_connections 2024-04-25T01:19:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.178+0000 7f103c220740 1 --2- 172.21.15.31:0/1882917846 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5650b1b3abf0 0x5650b1b3d020 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.178+0000 7f103c220740 1 --2- 172.21.15.31:0/1882917846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5650b1b2a800 0x5650b1b36550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.178+0000 7f103c220740 1 --2- 172.21.15.31:0/1882917846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5650b1b25690 0x5650b1b2f580 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.178+0000 7f103c220740 1 --2- 172.21.15.31:0/1882917846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b1b24ca0 0x5650b1b2f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.178+0000 7f103c220740 1 -- 172.21.15.31:0/1882917846 >> 172.21.15.31:0/1882917846 conn(0x5650b1b20310 msgr2=0x5650b1b1f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:19:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.178+0000 7f1013e00640 0 client.25472 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:19:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.178+0000 7f103c220740 1 -- 172.21.15.31:0/1882917846 shutdown_connections 2024-04-25T01:19:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.178+0000 7f103c220740 1 -- 172.21.15.31:0/1882917846 wait complete. 2024-04-25T01:19:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.179+0000 7f103c220740 20 client.25472 trim_cache size 0 max 16384 2024-04-25T01:19:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.231+0000 7f103c220740 1 Processor -- start 2024-04-25T01:19:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.231+0000 7f103c220740 1 -- start start 2024-04-25T01:19:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.232+0000 7f103c220740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5650b1b20310 0x5650b1b1fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.232+0000 7f103c220740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b1b24ca0 0x5650b1b36510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.232+0000 7f103c220740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5650b1b25690 0x5650b1b23700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.232+0000 7f103c220740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f10341ad700 con 0x5650b1b24ca0 2024-04-25T01:19:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.232+0000 7f103c220740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f10341aaf70 con 0x5650b1b25690 2024-04-25T01:19:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.232+0000 7f103c220740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f10341a8de0 con 0x5650b1b20310 2024-04-25T01:19:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.233+0000 7f1032800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5650b1b20310 0x5650b1b1fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.233+0000 7f1032800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5650b1b20310 0x5650b1b1fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:58342/0 (socket says 172.21.15.31:58342) 2024-04-25T01:19:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.233+0000 7f1032800640 1 -- 172.21.15.31:0/2919564427 learned_addr learned my addr 172.21.15.31:0/2919564427 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:19:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.234+0000 7f1031e00640 1 --2- 172.21.15.31:0/2919564427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b1b24ca0 0x5650b1b36510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.234+0000 7f1033200640 1 --2- 172.21.15.31:0/2919564427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5650b1b25690 0x5650b1b23700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.235+0000 7f1032800640 1 -- 172.21.15.31:0/2919564427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5650b1b25690 msgr2=0x5650b1b23700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.235+0000 7f1032800640 1 --2- 172.21.15.31:0/2919564427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5650b1b25690 0x5650b1b23700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.235+0000 7f1032800640 1 -- 172.21.15.31:0/2919564427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b1b24ca0 msgr2=0x5650b1b36510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.235+0000 7f1032800640 1 --2- 172.21.15.31:0/2919564427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b1b24ca0 0x5650b1b36510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.236+0000 7f1032800640 1 -- 172.21.15.31:0/2919564427 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b1b20140 con 0x5650b1b20310 2024-04-25T01:19:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.236+0000 7f1032800640 1 --2- 172.21.15.31:0/2919564427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5650b1b20310 0x5650b1b1fc00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1024009230 tx=0x7f102403f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbbcbca2591982d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:19:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.237+0000 7f1031e00640 1 --2- 172.21.15.31:0/2919564427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b1b24ca0 0x5650b1b36510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:19:34.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.238+0000 7f101ea00640 1 -- 172.21.15.31:0/2919564427 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1024057070 con 0x5650b1b20310 2024-04-25T01:19:34.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.238+0000 7f101ea00640 1 -- 172.21.15.31:0/2919564427 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1024051e20 con 0x5650b1b20310 2024-04-25T01:19:34.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.238+0000 7f101ea00640 1 -- 172.21.15.31:0/2919564427 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1024057070 con 0x5650b1b20310 2024-04-25T01:19:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.239+0000 7f103c220740 1 -- 172.21.15.31:0/2919564427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5650b1b20310 msgr2=0x5650b1b1fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.239+0000 7f103c220740 1 --2- 172.21.15.31:0/2919564427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5650b1b20310 0x5650b1b1fc00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1024009230 tx=0x7f102403f000 comp rx=0 tx=0).stop 2024-04-25T01:19:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.240+0000 7f103c220740 1 -- 172.21.15.31:0/2919564427 shutdown_connections 2024-04-25T01:19:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.240+0000 7f103c220740 1 --2- 172.21.15.31:0/2919564427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5650b1b25690 0x5650b1b23700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.240+0000 7f103c220740 1 --2- 172.21.15.31:0/2919564427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b1b24ca0 0x5650b1b36510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.240+0000 7f103c220740 1 --2- 172.21.15.31:0/2919564427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5650b1b20310 0x5650b1b1fc00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.240+0000 7f103c220740 1 -- 172.21.15.31:0/2919564427 >> 172.21.15.31:0/2919564427 conn(0x5650b1b2a340 msgr2=0x5650b1b22ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:19:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.242+0000 7f103c220740 1 -- 172.21.15.31:0/2919564427 shutdown_connections 2024-04-25T01:19:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.242+0000 7f103c220740 1 -- 172.21.15.31:0/2919564427 wait complete. 2024-04-25T01:19:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.243+0000 7f103c220740 1 -- start start 2024-04-25T01:19:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.244+0000 7f103c220740 1 Processor -- start 2024-04-25T01:19:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.252+0000 7f103c220740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5650b1b20310 0x5650b1b32810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.252+0000 7f103c220740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b1b24ca0 0x5650b1b32d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.252+0000 7f103c220740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5650b1b25690 0x5650b1b35c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.252+0000 7f103c220740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f10341a9430 con 0x5650b1b24ca0 2024-04-25T01:19:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.252+0000 7f103c220740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f10341aa8a0 con 0x5650b1b20310 2024-04-25T01:19:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.252+0000 7f103c220740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f10341a9b00 con 0x5650b1b25690 2024-04-25T01:19:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.252+0000 7f103c220740 10 client.0 _ll_register_callbacks cb 0x7f102000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:19:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.252+0000 7f1033200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5650b1b25690 0x5650b1b35c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.252+0000 7f1033200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5650b1b25690 0x5650b1b35c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:58346/0 (socket says 172.21.15.31:58346) 2024-04-25T01:19:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.252+0000 7f1033200640 1 -- 172.21.15.31:0/426088114 learned_addr learned my addr 172.21.15.31:0/426088114 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:19:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.257+0000 7f1033200640 1 -- 172.21.15.31:0/426088114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5650b1b20310 msgr2=0x5650b1b32810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.257+0000 7f1033200640 1 --2- 172.21.15.31:0/426088114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5650b1b20310 0x5650b1b32810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.257+0000 7f1033200640 1 -- 172.21.15.31:0/426088114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b1b24ca0 msgr2=0x5650b1b32d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.257+0000 7f1033200640 1 --2- 172.21.15.31:0/426088114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b1b24ca0 0x5650b1b32d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.261+0000 7f1033200640 1 -- 172.21.15.31:0/426088114 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b1b36270 con 0x5650b1b25690 2024-04-25T01:19:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.265+0000 7f1033200640 1 --2- 172.21.15.31:0/426088114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5650b1b25690 0x5650b1b35c80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x5650b1b25ef0 tx=0x7f102c004c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25a0f62c3e81e09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:19:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.265+0000 7f1013e00640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:19:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.267+0000 7f1013e00640 1 -- 172.21.15.31:0/426088114 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c016040 con 0x5650b1b25690 2024-04-25T01:19:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.269+0000 7f1013e00640 1 -- 172.21.15.31:0/426088114 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f102c005250 con 0x5650b1b25690 2024-04-25T01:19:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.270+0000 7f103c220740 1 -- 172.21.15.31:0/426088114 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1024051e20 con 0x5650b1b25690 2024-04-25T01:19:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.271+0000 7f1013e00640 1 -- 172.21.15.31:0/426088114 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f102c016040 con 0x5650b1b25690 2024-04-25T01:19:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.272+0000 7f103c220740 10 client.25501 fetch_fsmap learned FSMap version 27 2024-04-25T01:19:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.272+0000 7f103c220740 1 -- 172.21.15.31:0/426088114 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5650b1bbb500 con 0x5650b1b25690 2024-04-25T01:19:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.274+0000 7f1013e00640 1 -- 172.21.15.31:0/426088114 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f102c004e40 con 0x5650b1b25690 2024-04-25T01:19:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.274+0000 7f1013e00640 1 -- 172.21.15.31:0/426088114 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7f102c00f040 con 0x5650b1b25690 2024-04-25T01:19:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.275+0000 7f103c220740 10 client.25501 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:19:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.275+0000 7f103c220740 10 client.25501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:19:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.275+0000 7f103c220740 20 client.25501 populate_metadata read hostname 'smithi031' 2024-04-25T01:19:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.275+0000 7f103c220740 1 --2- 172.21.15.31:0/426088114 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5650b1b2a800 0x5650b1bbdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.276+0000 7f103c220740 4 client.25501 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:19:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.276+0000 7f103c220740 1 -- 172.21.15.31:0/426088114 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f1024051e20 con 0x5650b1b2a800 2024-04-25T01:19:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.277+0000 7f1032800640 1 --2- 172.21.15.31:0/426088114 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5650b1b2a800 0x5650b1bbdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.281+0000 7f1013e00640 10 client.25501 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:19:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:34.281+0000 7f1032800640 1 --2- 172.21.15.31:0/426088114 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5650b1b2a800 0x5650b1bbdd50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c0b93aae7ea8508 server_cookie=7a4f372d6424015b in_seq=0 out_seq=0 2024-04-25T01:19:35.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:35 smithi169 ceph-mon[32759]: pgmap v500: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 73 KiB/s rd, 124 MiB/s wr, 18.03k op/s 2024-04-25T01:19:35.694 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:35 smithi040 ceph-mon[32518]: pgmap v500: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 73 KiB/s rd, 124 MiB/s wr, 18.03k op/s 2024-04-25T01:19:35.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:35 smithi031 ceph-mon[28053]: pgmap v500: 97 pgs: 97 active+clean; 13 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 73 KiB/s rd, 124 MiB/s wr, 18.03k op/s 2024-04-25T01:19:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.667+0000 7f1013e00640 1 -- 172.21.15.31:0/426088114 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f1024051e20 con 0x5650b1b2a800 2024-04-25T01:19:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.667+0000 7f1013e00640 10 client.25501 handle_command_reply: tid=0 2024-04-25T01:19:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.667+0000 7f103c220740 1 client.25501 shutdown 2024-04-25T01:19:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.667+0000 7f103c220740 1 -- 172.21.15.31:0/426088114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5650b1b25690 msgr2=0x5650b1b35c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.667+0000 7f103c220740 1 --2- 172.21.15.31:0/426088114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5650b1b25690 0x5650b1b35c80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x5650b1b25ef0 tx=0x7f102c004c10 comp rx=0 tx=0).stop 2024-04-25T01:19:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.668+0000 7f103c220740 1 -- 172.21.15.31:0/426088114 shutdown_connections 2024-04-25T01:19:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.668+0000 7f103c220740 1 --2- 172.21.15.31:0/426088114 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5650b1b2a800 0x5650b1bbdd50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.668+0000 7f103c220740 1 --2- 172.21.15.31:0/426088114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5650b1b25690 0x5650b1b35c80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.668+0000 7f103c220740 1 --2- 172.21.15.31:0/426088114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b1b24ca0 0x5650b1b32d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.668+0000 7f103c220740 1 --2- 172.21.15.31:0/426088114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5650b1b20310 0x5650b1b32810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.668+0000 7f103c220740 1 -- 172.21.15.31:0/426088114 >> 172.21.15.31:0/426088114 conn(0x5650b1b2a340 msgr2=0x5650b1b23880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:19:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.668+0000 7f1013e00640 0 client.25501 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:19:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.668+0000 7f103c220740 1 -- 172.21.15.31:0/426088114 shutdown_connections 2024-04-25T01:19:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.668+0000 7f103c220740 1 -- 172.21.15.31:0/426088114 wait complete. 2024-04-25T01:19:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.669+0000 7f103c220740 20 client.25501 trim_cache size 0 max 16384 2024-04-25T01:19:37.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:19:37.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:19:37.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:19:37.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:19:37.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.670+0000 7f1039400640 1 -- 172.21.15.31:0/3153067094 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f10180a2df0 msgr2=0x7f10180a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:37.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.670+0000 7f1039400640 1 --2- 172.21.15.31:0/3153067094 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f10180a2df0 0x7f10180a52b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1024006010 tx=0x7f1024009560 comp rx=0 tx=0).stop 2024-04-25T01:19:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.670+0000 7f1039400640 1 -- 172.21.15.31:0/3153067094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1034075720 msgr2=0x7f103419a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.670+0000 7f1039400640 1 --2- 172.21.15.31:0/3153067094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1034075720 0x7f103419a160 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f102000e9f0 tx=0x7f102000eeb0 comp rx=0 tx=0).stop 2024-04-25T01:19:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.670+0000 7f1039400640 1 -- 172.21.15.31:0/3153067094 shutdown_connections 2024-04-25T01:19:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.670+0000 7f1039400640 1 --2- 172.21.15.31:0/3153067094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1034109660 0x7f10341a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.670+0000 7f1039400640 1 --2- 172.21.15.31:0/3153067094 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f10180a2df0 0x7f10180a52b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.670+0000 7f1039400640 1 --2- 172.21.15.31:0/3153067094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1034075720 0x7f103419a160 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.670+0000 7f1039400640 1 --2- 172.21.15.31:0/3153067094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1034074140 0x7f1034199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.670+0000 7f1039400640 1 -- 172.21.15.31:0/3153067094 >> 172.21.15.31:0/3153067094 conn(0x7f10340fcd60 msgr2=0x7f10341055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:19:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.670+0000 7f1039400640 1 -- 172.21.15.31:0/3153067094 shutdown_connections 2024-04-25T01:19:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:37.671+0000 7f1039400640 1 -- 172.21.15.31:0/3153067094 wait complete. 2024-04-25T01:19:37.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:37 smithi169 ceph-mon[32759]: pgmap v501: 97 pgs: 97 active+clean; 13 GiB data, 58 GiB used, 1014 GiB / 1.0 TiB avail; 121 KiB/s rd, 138 MiB/s wr, 21.36k op/s 2024-04-25T01:19:37.694 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:37 smithi040 ceph-mon[32518]: pgmap v501: 97 pgs: 97 active+clean; 13 GiB data, 58 GiB used, 1014 GiB / 1.0 TiB avail; 121 KiB/s rd, 138 MiB/s wr, 21.36k op/s 2024-04-25T01:19:37.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:37 smithi031 ceph-mon[28053]: pgmap v501: 97 pgs: 97 active+clean; 13 GiB data, 58 GiB used, 1014 GiB / 1.0 TiB avail; 121 KiB/s rd, 138 MiB/s wr, 21.36k op/s 2024-04-25T01:19:39.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:39 smithi169 ceph-mon[32759]: pgmap v502: 97 pgs: 97 active+clean; 13 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 123 KiB/s rd, 111 MiB/s wr, 18.52k op/s 2024-04-25T01:19:39.694 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:39 smithi040 ceph-mon[32518]: pgmap v502: 97 pgs: 97 active+clean; 13 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 123 KiB/s rd, 111 MiB/s wr, 18.52k op/s 2024-04-25T01:19:39.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:39 smithi031 ceph-mon[28053]: pgmap v502: 97 pgs: 97 active+clean; 13 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 123 KiB/s rd, 111 MiB/s wr, 18.52k op/s 2024-04-25T01:19:41.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:41 smithi169 ceph-mon[32759]: pgmap v503: 97 pgs: 97 active+clean; 13 GiB data, 59 GiB used, 1013 GiB / 1.0 TiB avail; 139 KiB/s rd, 147 MiB/s wr, 23.16k op/s 2024-04-25T01:19:41.694 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:41 smithi040 ceph-mon[32518]: pgmap v503: 97 pgs: 97 active+clean; 13 GiB data, 59 GiB used, 1013 GiB / 1.0 TiB avail; 139 KiB/s rd, 147 MiB/s wr, 23.16k op/s 2024-04-25T01:19:41.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:41 smithi031 ceph-mon[28053]: pgmap v503: 97 pgs: 97 active+clean; 13 GiB data, 59 GiB used, 1013 GiB / 1.0 TiB avail; 139 KiB/s rd, 147 MiB/s wr, 23.16k op/s 2024-04-25T01:19:43.694 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:43 smithi040 ceph-mon[32518]: pgmap v504: 97 pgs: 97 active+clean; 13 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 104 KiB/s rd, 120 MiB/s wr, 18.69k op/s 2024-04-25T01:19:43.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:43 smithi031 ceph-mon[28053]: pgmap v504: 97 pgs: 97 active+clean; 13 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 104 KiB/s rd, 120 MiB/s wr, 18.69k op/s 2024-04-25T01:19:43.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:43 smithi169 ceph-mon[32759]: pgmap v504: 97 pgs: 97 active+clean; 13 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 104 KiB/s rd, 120 MiB/s wr, 18.69k op/s 2024-04-25T01:19:45.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:45 smithi031 ceph-mon[28053]: pgmap v505: 97 pgs: 97 active+clean; 13 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 84 KiB/s rd, 127 MiB/s wr, 18.99k op/s 2024-04-25T01:19:45.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:45 smithi169 ceph-mon[32759]: pgmap v505: 97 pgs: 97 active+clean; 13 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 84 KiB/s rd, 127 MiB/s wr, 18.99k op/s 2024-04-25T01:19:45.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:45 smithi040 ceph-mon[32518]: pgmap v505: 97 pgs: 97 active+clean; 13 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 84 KiB/s rd, 127 MiB/s wr, 18.99k op/s 2024-04-25T01:19:47.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:47 smithi031 ceph-mon[28053]: pgmap v506: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 108 KiB/s rd, 145 MiB/s wr, 20.70k op/s 2024-04-25T01:19:47.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:47 smithi169 ceph-mon[32759]: pgmap v506: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 108 KiB/s rd, 145 MiB/s wr, 20.70k op/s 2024-04-25T01:19:47.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:47 smithi040 ceph-mon[32518]: pgmap v506: 97 pgs: 97 active+clean; 13 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 108 KiB/s rd, 145 MiB/s wr, 20.70k op/s 2024-04-25T01:19:49.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:49 smithi031 ceph-mon[28053]: pgmap v507: 97 pgs: 97 active+clean; 13 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 66 KiB/s rd, 133 MiB/s wr, 17.63k op/s 2024-04-25T01:19:49.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:49 smithi169 ceph-mon[32759]: pgmap v507: 97 pgs: 97 active+clean; 13 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 66 KiB/s rd, 133 MiB/s wr, 17.63k op/s 2024-04-25T01:19:49.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:49 smithi040 ceph-mon[32518]: pgmap v507: 97 pgs: 97 active+clean; 13 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 66 KiB/s rd, 133 MiB/s wr, 17.63k op/s 2024-04-25T01:19:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.061+0000 7f3a23400640 1 -- 172.21.15.31:0/3120856003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a1c108fc0 msgr2=0x7f3a1c109440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.061+0000 7f3a23400640 1 --2- 172.21.15.31:0/3120856003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a1c108fc0 0x7f3a1c109440 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009860 tx=0x7f3a0802a360 comp rx=0 tx=0).stop 2024-04-25T01:19:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.062+0000 7f3a1b400640 1 -- 172.21.15.31:0/3120856003 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a080309b0 con 0x7f3a1c108fc0 2024-04-25T01:19:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.063+0000 7f3a23400640 1 -- 172.21.15.31:0/3120856003 shutdown_connections 2024-04-25T01:19:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.063+0000 7f3a23400640 1 --2- 172.21.15.31:0/3120856003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a1c109980 0x7f3a1c110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.063+0000 7f3a23400640 1 --2- 172.21.15.31:0/3120856003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a1c108fc0 0x7f3a1c109440 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.063+0000 7f3a23400640 1 --2- 172.21.15.31:0/3120856003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a1c107dd0 0x7f3a1c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.063+0000 7f3a23400640 1 -- 172.21.15.31:0/3120856003 >> 172.21.15.31:0/3120856003 conn(0x7f3a1c076d90 msgr2=0x7f3a1c0791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:19:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.064+0000 7f3a23400640 1 -- 172.21.15.31:0/3120856003 shutdown_connections 2024-04-25T01:19:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.064+0000 7f3a23400640 1 -- 172.21.15.31:0/3120856003 wait complete. 2024-04-25T01:19:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.069+0000 7f3a23400640 1 Processor -- start 2024-04-25T01:19:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.069+0000 7f3a23400640 1 -- start start 2024-04-25T01:19:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.071+0000 7f3a23400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a1c107dd0 0x7f3a1c19de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.072+0000 7f3a23400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a1c108fc0 0x7f3a1c19e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.072+0000 7f3a1be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a1c108fc0 0x7f3a1c19e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.072+0000 7f3a1be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a1c108fc0 0x7f3a1c19e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:60150/0 (socket says 172.21.15.31:60150) 2024-04-25T01:19:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.072+0000 7f3a1be00640 1 -- 172.21.15.31:0/3540550491 learned_addr learned my addr 172.21.15.31:0/3540550491 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:19:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.072+0000 7f3a23400640 1 --2- 172.21.15.31:0/3540550491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a1c109980 0x7f3a1c1a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.073+0000 7f3a23400640 1 -- 172.21.15.31:0/3540550491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1c113a80 con 0x7f3a1c107dd0 2024-04-25T01:19:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.073+0000 7f3a23400640 1 -- 172.21.15.31:0/3540550491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1c113900 con 0x7f3a1c108fc0 2024-04-25T01:19:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.073+0000 7f3a23400640 1 -- 172.21.15.31:0/3540550491 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1c113c00 con 0x7f3a1c109980 2024-04-25T01:19:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.073+0000 7f3a21400640 1 --2- 172.21.15.31:0/3540550491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a1c109980 0x7f3a1c1a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.074+0000 7f3a21400640 1 -- 172.21.15.31:0/3540550491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a1c108fc0 msgr2=0x7f3a1c19e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.074+0000 7f3a21400640 1 --2- 172.21.15.31:0/3540550491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a1c108fc0 0x7f3a1c19e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.074+0000 7f3a21400640 1 -- 172.21.15.31:0/3540550491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a1c107dd0 msgr2=0x7f3a1c19de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.074+0000 7f3a20a00640 1 --2- 172.21.15.31:0/3540550491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a1c107dd0 0x7f3a1c19de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.075+0000 7f3a21400640 1 --2- 172.21.15.31:0/3540550491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a1c107dd0 0x7f3a1c19de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.076+0000 7f3a21400640 1 -- 172.21.15.31:0/3540550491 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1c1a59e0 con 0x7f3a1c109980 2024-04-25T01:19:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.076+0000 7f3a1be00640 1 --2- 172.21.15.31:0/3540550491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a1c108fc0 0x7f3a1c19e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:19:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.076+0000 7f3a20a00640 1 --2- 172.21.15.31:0/3540550491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a1c107dd0 0x7f3a1c19de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:19:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.076+0000 7f3a21400640 1 --2- 172.21.15.31:0/3540550491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a1c109980 0x7f3a1c1a5370 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3a14009fd0 tx=0x7f3a1400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:19:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.078+0000 7f3a19600640 1 -- 172.21.15.31:0/3540550491 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1400ce20 con 0x7f3a1c109980 2024-04-25T01:19:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.079+0000 7f3a19600640 1 -- 172.21.15.31:0/3540550491 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a14005020 con 0x7f3a1c109980 2024-04-25T01:19:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.079+0000 7f3a23400640 1 -- 172.21.15.31:0/3540550491 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a1c1a5cd0 con 0x7f3a1c109980 2024-04-25T01:19:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.079+0000 7f3a23400640 1 -- 172.21.15.31:0/3540550491 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a1c1a61d0 con 0x7f3a1c109980 2024-04-25T01:19:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.085+0000 7f3a19600640 1 -- 172.21.15.31:0/3540550491 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a14010720 con 0x7f3a1c109980 2024-04-25T01:19:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.095+0000 7f3a19600640 1 -- 172.21.15.31:0/3540550491 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3a140109a0 con 0x7f3a1c109980 2024-04-25T01:19:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.102+0000 7f3a19600640 1 --2- 172.21.15.31:0/3540550491 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f3a000a2d10 0x7f3a000a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.102+0000 7f3a19600640 1 -- 172.21.15.31:0/3540550491 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f3a14004b20 con 0x7f3a1c109980 2024-04-25T01:19:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.103+0000 7f3a26218740 1 Processor -- start 2024-04-25T01:19:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.104+0000 7f3a20a00640 1 --2- 172.21.15.31:0/3540550491 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f3a000a2d10 0x7f3a000a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.105+0000 7f3a26218740 1 -- start start 2024-04-25T01:19:51.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.106+0000 7f3a20a00640 1 --2- 172.21.15.31:0/3540550491 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f3a000a2d10 0x7f3a000a51d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c005fd0 tx=0x7f3a0c005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:19:51.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.106+0000 7f3a26218740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b138cc7ca0 0x55b138cc80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:51.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.108+0000 7f3a26218740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b138cc8690 0x55b138ccd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:51.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.108+0000 7f3a20a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b138cc7ca0 0x55b138cc80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:51.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.108+0000 7f3a20a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b138cc7ca0 0x55b138cc80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:60162/0 (socket says 172.21.15.31:60162) 2024-04-25T01:19:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.109+0000 7f3a20a00640 1 -- 172.21.15.31:0/194604621 learned_addr learned my addr 172.21.15.31:0/194604621 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:19:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.109+0000 7f3a26218740 1 --2- 172.21.15.31:0/194604621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b138ccd800 0x55b138ccfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.109+0000 7f3a26218740 1 -- 172.21.15.31:0/194604621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b138c2aab0 con 0x55b138cc8690 2024-04-25T01:19:51.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.109+0000 7f3a26218740 1 -- 172.21.15.31:0/194604621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b138bd3640 con 0x55b138cc7ca0 2024-04-25T01:19:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.109+0000 7f3a26218740 1 -- 172.21.15.31:0/194604621 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b138c26a70 con 0x55b138ccd800 2024-04-25T01:19:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.110+0000 7f3a20a00640 1 -- 172.21.15.31:0/194604621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b138ccd800 msgr2=0x55b138ccfc90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:19:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.110+0000 7f3a20a00640 1 --2- 172.21.15.31:0/194604621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b138ccd800 0x55b138ccfc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.110+0000 7f3a20a00640 1 -- 172.21.15.31:0/194604621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b138cc8690 msgr2=0x55b138ccd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.110+0000 7f3a20a00640 1 --2- 172.21.15.31:0/194604621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b138cc8690 0x55b138ccd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.110+0000 7f3a20a00640 1 -- 172.21.15.31:0/194604621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b13893f210 con 0x55b138cc7ca0 2024-04-25T01:19:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.111+0000 7f3a20a00640 1 --2- 172.21.15.31:0/194604621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b138cc7ca0 0x55b138cc80c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00de50 tx=0x7f3a0c010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17afc6174b2da33e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:19:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.112+0000 7f3a11c00640 1 -- 172.21.15.31:0/194604621 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c020070 con 0x55b138cc7ca0 2024-04-25T01:19:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.113+0000 7f3a11c00640 1 -- 172.21.15.31:0/194604621 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a0c00fe30 con 0x55b138cc7ca0 2024-04-25T01:19:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.113+0000 7f3a11c00640 1 -- 172.21.15.31:0/194604621 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c01bc00 con 0x55b138cc7ca0 2024-04-25T01:19:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.114+0000 7f3a26218740 1 -- 172.21.15.31:0/194604621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b138cc7ca0 msgr2=0x55b138cc80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.114+0000 7f3a26218740 1 --2- 172.21.15.31:0/194604621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b138cc7ca0 0x55b138cc80c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00de50 tx=0x7f3a0c010000 comp rx=0 tx=0).stop 2024-04-25T01:19:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.115+0000 7f3a26218740 1 -- 172.21.15.31:0/194604621 shutdown_connections 2024-04-25T01:19:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.116+0000 7f3a26218740 1 --2- 172.21.15.31:0/194604621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b138ccd800 0x55b138ccfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.116+0000 7f3a26218740 1 --2- 172.21.15.31:0/194604621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b138cc8690 0x55b138ccd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.116+0000 7f3a26218740 1 --2- 172.21.15.31:0/194604621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b138cc7ca0 0x55b138cc80c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.116+0000 7f3a26218740 1 -- 172.21.15.31:0/194604621 >> 172.21.15.31:0/194604621 conn(0x55b138cc3310 msgr2=0x55b138cc3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:19:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.117+0000 7f3a26218740 1 -- 172.21.15.31:0/194604621 shutdown_connections 2024-04-25T01:19:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.117+0000 7f3a26218740 1 -- 172.21.15.31:0/194604621 wait complete. 2024-04-25T01:19:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.118+0000 7f3a26218740 1 -- start start 2024-04-25T01:19:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.118+0000 7f3a26218740 1 Processor -- start 2024-04-25T01:19:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.120+0000 7f3a26218740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b138cc7ca0 0x55b138cd1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.121+0000 7f3a26218740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b138cc8690 0x55b138cd2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.122+0000 7f3a20a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b138cc7ca0 0x55b138cd1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.122+0000 7f3a26218740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b138ccd800 0x55b138cd9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.122+0000 7f3a20a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b138cc7ca0 0x55b138cd1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35068/0 (socket says 172.21.15.31:35068) 2024-04-25T01:19:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.123+0000 7f3a20a00640 1 -- 172.21.15.31:0/2184565316 learned_addr learned my addr 172.21.15.31:0/2184565316 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:19:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.123+0000 7f3a21400640 1 --2- 172.21.15.31:0/2184565316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b138ccd800 0x55b138cd9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.123+0000 7f3a1be00640 1 --2- 172.21.15.31:0/2184565316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b138cc8690 0x55b138cd2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.123+0000 7f3a26218740 1 -- 172.21.15.31:0/2184565316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b138cd07b0 con 0x55b138cc7ca0 2024-04-25T01:19:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.123+0000 7f3a26218740 1 -- 172.21.15.31:0/2184565316 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b138be6990 con 0x55b138cc8690 2024-04-25T01:19:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.123+0000 7f3a26218740 1 -- 172.21.15.31:0/2184565316 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b138cd0930 con 0x55b138ccd800 2024-04-25T01:19:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.124+0000 7f3a26218740 10 client.0 _ll_register_callbacks cb 0x55b138be8950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:19:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.125+0000 7f3a20a00640 1 -- 172.21.15.31:0/2184565316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b138ccd800 msgr2=0x55b138cd9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.125+0000 7f3a20a00640 1 --2- 172.21.15.31:0/2184565316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b138ccd800 0x55b138cd9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.125+0000 7f3a20a00640 1 -- 172.21.15.31:0/2184565316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b138cc8690 msgr2=0x55b138cd2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.125+0000 7f3a20a00640 1 --2- 172.21.15.31:0/2184565316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b138cc8690 0x55b138cd2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.125+0000 7f3a20a00640 1 -- 172.21.15.31:0/2184565316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b138a0b620 con 0x55b138cc7ca0 2024-04-25T01:19:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.127+0000 7f3a06000640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T01:19:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.127+0000 7f3a20a00640 1 --2- 172.21.15.31:0/2184565316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b138cc7ca0 0x55b138cd1f10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c01c890 tx=0x7f3a0c01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e9df33bad536c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:19:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.129+0000 7f3a06000640 1 -- 172.21.15.31:0/2184565316 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c034060 con 0x55b138cc7ca0 2024-04-25T01:19:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.129+0000 7f3a06000640 1 -- 172.21.15.31:0/2184565316 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f3a0c01ccf0 con 0x55b138cc7ca0 2024-04-25T01:19:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.130+0000 7f3a26218740 1 -- 172.21.15.31:0/2184565316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a0c00fe30 con 0x55b138cc7ca0 2024-04-25T01:19:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.132+0000 7f3a06000640 1 -- 172.21.15.31:0/2184565316 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3a0c020050 con 0x55b138cc7ca0 2024-04-25T01:19:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.133+0000 7f3a26218740 10 client.15294 fetch_fsmap learned FSMap version 27 2024-04-25T01:19:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.133+0000 7f3a26218740 1 -- 172.21.15.31:0/2184565316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b138cdc450 con 0x55b138cc7ca0 2024-04-25T01:19:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.134+0000 7f3a06000640 1 -- 172.21.15.31:0/2184565316 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a0c00fe30 con 0x55b138cc7ca0 2024-04-25T01:19:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.134+0000 7f3a06000640 1 -- 172.21.15.31:0/2184565316 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7f3a0c025c00 con 0x55b138cc7ca0 2024-04-25T01:19:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.137+0000 7f3a26218740 10 client.15294 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:19:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.137+0000 7f3a26218740 10 client.15294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:19:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.137+0000 7f3a26218740 20 client.15294 populate_metadata read hostname 'smithi031' 2024-04-25T01:19:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.137+0000 7f3a26218740 1 --2- 172.21.15.31:0/2184565316 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55b138cdd800 0x55b138cdfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.137+0000 7f3a26218740 4 client.15294 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:19:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.137+0000 7f3a26218740 1 -- 172.21.15.31:0/2184565316 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b138c37ff0 con 0x55b138cdd800 2024-04-25T01:19:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.138+0000 7f3a1be00640 1 --2- 172.21.15.31:0/2184565316 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55b138cdd800 0x55b138cdfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.139+0000 7f3a1be00640 1 --2- 172.21.15.31:0/2184565316 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55b138cdd800 0x55b138cdfbf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78bc74c1563a8886 server_cookie=35f4f37df1689711 in_seq=0 out_seq=0 2024-04-25T01:19:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.140+0000 7f3a06000640 10 client.15294 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:19:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.141+0000 7f3a06000640 1 -- 172.21.15.31:0/2184565316 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b138c37ff0 con 0x55b138cdd800 2024-04-25T01:19:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.141+0000 7f3a06000640 10 client.15294 handle_command_reply: tid=0 2024-04-25T01:19:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.141+0000 7f3a26218740 1 client.15294 shutdown 2024-04-25T01:19:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.142+0000 7f3a26218740 1 -- 172.21.15.31:0/2184565316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b138cc7ca0 msgr2=0x55b138cd1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.142+0000 7f3a26218740 1 --2- 172.21.15.31:0/2184565316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b138cc7ca0 0x55b138cd1f10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c01c890 tx=0x7f3a0c01c8c0 comp rx=0 tx=0).stop 2024-04-25T01:19:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.142+0000 7f3a26218740 1 -- 172.21.15.31:0/2184565316 shutdown_connections 2024-04-25T01:19:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.142+0000 7f3a26218740 1 --2- 172.21.15.31:0/2184565316 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55b138cdd800 0x55b138cdfbf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.142+0000 7f3a26218740 1 --2- 172.21.15.31:0/2184565316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b138ccd800 0x55b138cd9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.142+0000 7f3a26218740 1 --2- 172.21.15.31:0/2184565316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b138cc8690 0x55b138cd2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.142+0000 7f3a26218740 1 --2- 172.21.15.31:0/2184565316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b138cc7ca0 0x55b138cd1f10 secure :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c01c890 tx=0x7f3a0c01c8c0 comp rx=0 tx=0).stop 2024-04-25T01:19:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.142+0000 7f3a26218740 1 -- 172.21.15.31:0/2184565316 >> 172.21.15.31:0/2184565316 conn(0x55b138cc3310 msgr2=0x55b138cc2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:19:51.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.142+0000 7f3a06000640 0 client.15294 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:19:51.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.143+0000 7f3a26218740 1 -- 172.21.15.31:0/2184565316 shutdown_connections 2024-04-25T01:19:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.143+0000 7f3a26218740 1 -- 172.21.15.31:0/2184565316 wait complete. 2024-04-25T01:19:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.144+0000 7f3a26218740 20 client.15294 trim_cache size 0 max 16384 2024-04-25T01:19:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.188+0000 7f3a26218740 1 Processor -- start 2024-04-25T01:19:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.188+0000 7f3a26218740 1 -- start start 2024-04-25T01:19:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.189+0000 7f3a26218740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b138cc8690 0x55b138cd23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.189+0000 7f3a26218740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b138ccd800 0x55b138cd9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.189+0000 7f3a26218740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b138cbdfb0 0x55b138cccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.189+0000 7f3a26218740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc010ce0 con 0x55b138cbdfb0 2024-04-25T01:19:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.189+0000 7f3a26218740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc00e550 con 0x55b138ccd800 2024-04-25T01:19:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.189+0000 7f3a26218740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc00c3a0 con 0x55b138cc8690 2024-04-25T01:19:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.190+0000 7f3a20a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b138cc8690 0x55b138cd23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.191+0000 7f3a20a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b138cc8690 0x55b138cd23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:34652/0 (socket says 172.21.15.31:34652) 2024-04-25T01:19:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.191+0000 7f3a1be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b138ccd800 0x55b138cd9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.191+0000 7f3a1be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b138ccd800 0x55b138cd9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:60194/0 (socket says 172.21.15.31:60194) 2024-04-25T01:19:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.191+0000 7f3a1be00640 1 -- 172.21.15.31:0/3765528318 learned_addr learned my addr 172.21.15.31:0/3765528318 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:19:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.192+0000 7f3a1be00640 1 -- 172.21.15.31:0/3765528318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b138cc8690 msgr2=0x55b138cd23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.192+0000 7f3a1be00640 1 --2- 172.21.15.31:0/3765528318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b138cc8690 0x55b138cd23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.192+0000 7f3a1be00640 1 -- 172.21.15.31:0/3765528318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b138cbdfb0 msgr2=0x55b138cccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:19:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.192+0000 7f3a1be00640 1 --2- 172.21.15.31:0/3765528318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b138cbdfb0 0x55b138cccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.192+0000 7f3a1be00640 1 -- 172.21.15.31:0/3765528318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b138cd00b0 con 0x55b138ccd800 2024-04-25T01:19:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.193+0000 7f3a1be00640 1 --2- 172.21.15.31:0/3765528318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b138ccd800 0x55b138cd9450 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3a080095f0 tx=0x7f3a08005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5b1d1d743cb5a7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:19:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.195+0000 7f3a11200640 1 -- 172.21.15.31:0/3765528318 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a08037040 con 0x55b138ccd800 2024-04-25T01:19:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.195+0000 7f3a11200640 1 -- 172.21.15.31:0/3765528318 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a08041070 con 0x55b138ccd800 2024-04-25T01:19:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.195+0000 7f3a11200640 1 -- 172.21.15.31:0/3765528318 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a08032ae0 con 0x55b138ccd800 2024-04-25T01:19:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.196+0000 7f3a26218740 1 -- 172.21.15.31:0/3765528318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b138ccd800 msgr2=0x55b138cd9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.196+0000 7f3a26218740 1 --2- 172.21.15.31:0/3765528318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b138ccd800 0x55b138cd9450 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3a080095f0 tx=0x7f3a08005a90 comp rx=0 tx=0).stop 2024-04-25T01:19:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.197+0000 7f3a26218740 1 -- 172.21.15.31:0/3765528318 shutdown_connections 2024-04-25T01:19:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.197+0000 7f3a26218740 1 --2- 172.21.15.31:0/3765528318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b138cbdfb0 0x55b138cccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.197+0000 7f3a26218740 1 --2- 172.21.15.31:0/3765528318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b138ccd800 0x55b138cd9450 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.197+0000 7f3a26218740 1 --2- 172.21.15.31:0/3765528318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b138cc8690 0x55b138cd23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.197+0000 7f3a26218740 1 -- 172.21.15.31:0/3765528318 >> 172.21.15.31:0/3765528318 conn(0x55b138cc3310 msgr2=0x55b138cc36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:19:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.198+0000 7f3a26218740 1 -- 172.21.15.31:0/3765528318 shutdown_connections 2024-04-25T01:19:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.198+0000 7f3a26218740 1 -- 172.21.15.31:0/3765528318 wait complete. 2024-04-25T01:19:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.200+0000 7f3a26218740 1 -- start start 2024-04-25T01:19:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.200+0000 7f3a26218740 1 Processor -- start 2024-04-25T01:19:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.204+0000 7f3a26218740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b138cbdfb0 0x55b138cd6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.204+0000 7f3a26218740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b138cc8690 0x55b138cd7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.204+0000 7f3a26218740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b138ccd800 0x55b138cd8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.204+0000 7f3a26218740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc00ca10 con 0x55b138cbdfb0 2024-04-25T01:19:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.204+0000 7f3a26218740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc00de80 con 0x55b138ccd800 2024-04-25T01:19:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.204+0000 7f3a26218740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc00d0e0 con 0x55b138cc8690 2024-04-25T01:19:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.205+0000 7f3a26218740 10 client.0 _ll_register_callbacks cb 0x7f3a0c00dd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:19:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.205+0000 7f3a1be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b138cc8690 0x55b138cd7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.205+0000 7f3a1be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b138cc8690 0x55b138cd7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:34662/0 (socket says 172.21.15.31:34662) 2024-04-25T01:19:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.205+0000 7f3a1be00640 1 -- 172.21.15.31:0/1481968964 learned_addr learned my addr 172.21.15.31:0/1481968964 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:19:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.205+0000 7f3a1be00640 1 -- 172.21.15.31:0/1481968964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b138ccd800 msgr2=0x55b138cd8310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:19:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.205+0000 7f3a1be00640 1 --2- 172.21.15.31:0/1481968964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b138ccd800 0x55b138cd8310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.205+0000 7f3a1be00640 1 -- 172.21.15.31:0/1481968964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b138cbdfb0 msgr2=0x55b138cd6dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:19:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.205+0000 7f3a1be00640 1 --2- 172.21.15.31:0/1481968964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b138cbdfb0 0x55b138cd6dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.205+0000 7f3a1be00640 1 -- 172.21.15.31:0/1481968964 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b138cd8900 con 0x55b138cc8690 2024-04-25T01:19:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.207+0000 7f3a1be00640 1 --2- 172.21.15.31:0/1481968964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b138cc8690 0x55b138cd7300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3a0802d020 tx=0x7f3a08030e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=579710c09f3eb996 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:19:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.207+0000 7f3a06000640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:19:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.208+0000 7f3a06000640 1 -- 172.21.15.31:0/1481968964 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a08037040 con 0x55b138cc8690 2024-04-25T01:19:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.208+0000 7f3a26218740 1 -- 172.21.15.31:0/1481968964 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a08041070 con 0x55b138cc8690 2024-04-25T01:19:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.208+0000 7f3a06000640 1 -- 172.21.15.31:0/1481968964 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f3a08047070 con 0x55b138cc8690 2024-04-25T01:19:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.209+0000 7f3a06000640 1 -- 172.21.15.31:0/1481968964 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3a080309b0 con 0x55b138cc8690 2024-04-25T01:19:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.209+0000 7f3a26218740 10 client.25513 fetch_fsmap learned FSMap version 27 2024-04-25T01:19:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.209+0000 7f3a26218740 1 -- 172.21.15.31:0/1481968964 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b138d43ed0 con 0x55b138cc8690 2024-04-25T01:19:51.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.242+0000 7f3a06000640 1 -- 172.21.15.31:0/1481968964 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a08041070 con 0x55b138cc8690 2024-04-25T01:19:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.242+0000 7f3a06000640 1 -- 172.21.15.31:0/1481968964 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7f3a08049070 con 0x55b138cc8690 2024-04-25T01:19:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.245+0000 7f3a26218740 10 client.25513 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:19:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.245+0000 7f3a26218740 10 client.25513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:19:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.245+0000 7f3a26218740 20 client.25513 populate_metadata read hostname 'smithi031' 2024-04-25T01:19:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.245+0000 7f3a26218740 1 --2- 172.21.15.31:0/1481968964 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55b138d44850 0x55b138d46c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:19:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.245+0000 7f3a26218740 4 client.25513 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:19:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.245+0000 7f3a26218740 1 -- 172.21.15.31:0/1481968964 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b138c32340 con 0x55b138d44850 2024-04-25T01:19:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.245+0000 7f3a20a00640 1 --2- 172.21.15.31:0/1481968964 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55b138d44850 0x55b138d46c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:19:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.247+0000 7f3a20a00640 1 --2- 172.21.15.31:0/1481968964 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55b138d44850 0x55b138d46c70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a950c5eb46aefda server_cookie=68c9de541720f403 in_seq=0 out_seq=0 2024-04-25T01:19:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:51.247+0000 7f3a06000640 10 client.25513 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:19:51.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:51 smithi031 ceph-mon[28053]: pgmap v508: 97 pgs: 97 active+clean; 13 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 79 KiB/s rd, 165 MiB/s wr, 20.80k op/s 2024-04-25T01:19:51.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:51 smithi169 ceph-mon[32759]: pgmap v508: 97 pgs: 97 active+clean; 13 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 79 KiB/s rd, 165 MiB/s wr, 20.80k op/s 2024-04-25T01:19:51.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:51 smithi040 ceph-mon[32518]: pgmap v508: 97 pgs: 97 active+clean; 13 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 79 KiB/s rd, 165 MiB/s wr, 20.80k op/s 2024-04-25T01:19:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.070+0000 7f3a06000640 1 -- 172.21.15.31:0/1481968964 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b138c32340 con 0x55b138d44850 2024-04-25T01:19:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.070+0000 7f3a06000640 10 client.25513 handle_command_reply: tid=0 2024-04-25T01:19:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.071+0000 7f3a26218740 1 client.25513 shutdown 2024-04-25T01:19:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.074+0000 7f3a26218740 1 -- 172.21.15.31:0/1481968964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b138cc8690 msgr2=0x55b138cd7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.074+0000 7f3a26218740 1 --2- 172.21.15.31:0/1481968964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b138cc8690 0x55b138cd7300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3a0802d020 tx=0x7f3a08030e40 comp rx=0 tx=0).stop 2024-04-25T01:19:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.074+0000 7f3a26218740 1 -- 172.21.15.31:0/1481968964 shutdown_connections 2024-04-25T01:19:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.074+0000 7f3a26218740 1 --2- 172.21.15.31:0/1481968964 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55b138d44850 0x55b138d46c70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.074+0000 7f3a26218740 1 --2- 172.21.15.31:0/1481968964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b138ccd800 0x55b138cd8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.075+0000 7f3a06000640 0 client.25513 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:19:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.075+0000 7f3a26218740 1 --2- 172.21.15.31:0/1481968964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b138cc8690 0x55b138cd7300 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.075+0000 7f3a26218740 1 --2- 172.21.15.31:0/1481968964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b138cbdfb0 0x55b138cd6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.075+0000 7f3a26218740 1 -- 172.21.15.31:0/1481968964 >> 172.21.15.31:0/1481968964 conn(0x55b138cc3310 msgr2=0x55b138cd1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:19:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.076+0000 7f3a26218740 1 -- 172.21.15.31:0/1481968964 shutdown_connections 2024-04-25T01:19:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.076+0000 7f3a26218740 1 -- 172.21.15.31:0/1481968964 wait complete. 2024-04-25T01:19:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.076+0000 7f3a26218740 20 client.25513 trim_cache size 0 max 16384 2024-04-25T01:19:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:19:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:19:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:19:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:19:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.078+0000 7f3a23400640 1 -- 172.21.15.31:0/3540550491 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f3a000a2d10 msgr2=0x7f3a000a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:52.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.078+0000 7f3a23400640 1 --2- 172.21.15.31:0/3540550491 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f3a000a2d10 0x7f3a000a51d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c005fd0 tx=0x7f3a0c005ea0 comp rx=0 tx=0).stop 2024-04-25T01:19:52.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.078+0000 7f3a23400640 1 -- 172.21.15.31:0/3540550491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a1c109980 msgr2=0x7f3a1c1a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:19:52.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.078+0000 7f3a23400640 1 --2- 172.21.15.31:0/3540550491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a1c109980 0x7f3a1c1a5370 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3a14009fd0 tx=0x7f3a1400ef10 comp rx=0 tx=0).stop 2024-04-25T01:19:52.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.079+0000 7f3a23400640 1 -- 172.21.15.31:0/3540550491 shutdown_connections 2024-04-25T01:19:52.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.079+0000 7f3a23400640 1 --2- 172.21.15.31:0/3540550491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a1c109980 0x7f3a1c1a5370 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:52.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.079+0000 7f3a23400640 1 --2- 172.21.15.31:0/3540550491 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f3a000a2d10 0x7f3a000a51d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:52.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.079+0000 7f3a23400640 1 --2- 172.21.15.31:0/3540550491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a1c108fc0 0x7f3a1c19e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:52.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.079+0000 7f3a23400640 1 --2- 172.21.15.31:0/3540550491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a1c107dd0 0x7f3a1c19de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:19:52.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.079+0000 7f3a23400640 1 -- 172.21.15.31:0/3540550491 >> 172.21.15.31:0/3540550491 conn(0x7f3a1c076d90 msgr2=0x7f3a1c106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:19:52.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.079+0000 7f3a23400640 1 -- 172.21.15.31:0/3540550491 shutdown_connections 2024-04-25T01:19:52.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:19:52.080+0000 7f3a23400640 1 -- 172.21.15.31:0/3540550491 wait complete. 2024-04-25T01:19:53.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:53 smithi031 ceph-mon[28053]: pgmap v509: 97 pgs: 97 active+clean; 13 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 67 KiB/s rd, 131 MiB/s wr, 16.15k op/s 2024-04-25T01:19:53.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:53 smithi169 ceph-mon[32759]: pgmap v509: 97 pgs: 97 active+clean; 13 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 67 KiB/s rd, 131 MiB/s wr, 16.15k op/s 2024-04-25T01:19:53.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:53 smithi040 ceph-mon[32518]: pgmap v509: 97 pgs: 97 active+clean; 13 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 67 KiB/s rd, 131 MiB/s wr, 16.15k op/s 2024-04-25T01:19:55.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:55 smithi169 ceph-mon[32759]: pgmap v510: 97 pgs: 97 active+clean; 13 GiB data, 65 GiB used, 1008 GiB / 1.0 TiB avail; 63 KiB/s rd, 150 MiB/s wr, 18.04k op/s 2024-04-25T01:19:55.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:55 smithi040 ceph-mon[32518]: pgmap v510: 97 pgs: 97 active+clean; 13 GiB data, 65 GiB used, 1008 GiB / 1.0 TiB avail; 63 KiB/s rd, 150 MiB/s wr, 18.04k op/s 2024-04-25T01:19:56.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:55 smithi031 ceph-mon[28053]: pgmap v510: 97 pgs: 97 active+clean; 13 GiB data, 65 GiB used, 1008 GiB / 1.0 TiB avail; 63 KiB/s rd, 150 MiB/s wr, 18.04k op/s 2024-04-25T01:19:57.940 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:57 smithi169 ceph-mon[32759]: pgmap v511: 97 pgs: 97 active+clean; 13 GiB data, 66 GiB used, 1007 GiB / 1.0 TiB avail; 77 KiB/s rd, 156 MiB/s wr, 19.51k op/s 2024-04-25T01:19:57.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:57 smithi040 ceph-mon[32518]: pgmap v511: 97 pgs: 97 active+clean; 13 GiB data, 66 GiB used, 1007 GiB / 1.0 TiB avail; 77 KiB/s rd, 156 MiB/s wr, 19.51k op/s 2024-04-25T01:19:58.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:57 smithi031 ceph-mon[28053]: pgmap v511: 97 pgs: 97 active+clean; 13 GiB data, 66 GiB used, 1007 GiB / 1.0 TiB avail; 77 KiB/s rd, 156 MiB/s wr, 19.51k op/s 2024-04-25T01:19:59.804 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:19:59 smithi031 ceph-mon[28053]: pgmap v512: 97 pgs: 97 active+clean; 13 GiB data, 66 GiB used, 1007 GiB / 1.0 TiB avail; 64 KiB/s rd, 130 MiB/s wr, 16.43k op/s 2024-04-25T01:19:59.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:19:59 smithi169 ceph-mon[32759]: pgmap v512: 97 pgs: 97 active+clean; 13 GiB data, 66 GiB used, 1007 GiB / 1.0 TiB avail; 64 KiB/s rd, 130 MiB/s wr, 16.43k op/s 2024-04-25T01:19:59.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:19:59 smithi040 ceph-mon[32518]: pgmap v512: 97 pgs: 97 active+clean; 13 GiB data, 66 GiB used, 1007 GiB / 1.0 TiB avail; 64 KiB/s rd, 130 MiB/s wr, 16.43k op/s 2024-04-25T01:20:00.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:00 smithi169 ceph-mon[32759]: overall HEALTH_OK 2024-04-25T01:20:00.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:00 smithi040 ceph-mon[32518]: overall HEALTH_OK 2024-04-25T01:20:01.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:00 smithi031 ceph-mon[28053]: overall HEALTH_OK 2024-04-25T01:20:01.819 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:01 smithi040 ceph-mon[32518]: pgmap v513: 97 pgs: 97 active+clean; 13 GiB data, 66 GiB used, 1006 GiB / 1.0 TiB avail; 76 KiB/s rd, 153 MiB/s wr, 20.03k op/s 2024-04-25T01:20:01.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:01 smithi169 ceph-mon[32759]: pgmap v513: 97 pgs: 97 active+clean; 13 GiB data, 66 GiB used, 1006 GiB / 1.0 TiB avail; 76 KiB/s rd, 153 MiB/s wr, 20.03k op/s 2024-04-25T01:20:02.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:01 smithi031 ceph-mon[28053]: pgmap v513: 97 pgs: 97 active+clean; 13 GiB data, 66 GiB used, 1006 GiB / 1.0 TiB avail; 76 KiB/s rd, 153 MiB/s wr, 20.03k op/s 2024-04-25T01:20:03.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:03 smithi169 ceph-mon[32759]: pgmap v514: 97 pgs: 97 active+clean; 13 GiB data, 66 GiB used, 1007 GiB / 1.0 TiB avail; 57 KiB/s rd, 124 MiB/s wr, 16.79k op/s 2024-04-25T01:20:03.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:03 smithi040 ceph-mon[32518]: pgmap v514: 97 pgs: 97 active+clean; 13 GiB data, 66 GiB used, 1007 GiB / 1.0 TiB avail; 57 KiB/s rd, 124 MiB/s wr, 16.79k op/s 2024-04-25T01:20:04.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:03 smithi031 ceph-mon[28053]: pgmap v514: 97 pgs: 97 active+clean; 13 GiB data, 66 GiB used, 1007 GiB / 1.0 TiB avail; 57 KiB/s rd, 124 MiB/s wr, 16.79k op/s 2024-04-25T01:20:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.329+0000 7f5783e00640 1 -- 172.21.15.31:0/3575143560 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774003650 con 0x7f5784107dd0 2024-04-25T01:20:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.330+0000 7f578b800640 1 -- 172.21.15.31:0/3575143560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5784107dd0 msgr2=0x7f57841081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.330+0000 7f578b800640 1 --2- 172.21.15.31:0/3575143560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5784107dd0 0x7f57841081f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5774009900 tx=0x7f577402a410 comp rx=0 tx=0).stop 2024-04-25T01:20:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.331+0000 7f578b800640 1 -- 172.21.15.31:0/3575143560 shutdown_connections 2024-04-25T01:20:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.331+0000 7f578b800640 1 --2- 172.21.15.31:0/3575143560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5784109980 0x7f5784110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.331+0000 7f578b800640 1 --2- 172.21.15.31:0/3575143560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5784108fc0 0x7f5784109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.331+0000 7f578b800640 1 --2- 172.21.15.31:0/3575143560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5784107dd0 0x7f57841081f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.331+0000 7f578b800640 1 -- 172.21.15.31:0/3575143560 >> 172.21.15.31:0/3575143560 conn(0x7f5784076d90 msgr2=0x7f57840791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:20:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.331+0000 7f578b800640 1 -- 172.21.15.31:0/3575143560 shutdown_connections 2024-04-25T01:20:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.332+0000 7f578b800640 1 -- 172.21.15.31:0/3575143560 wait complete. 2024-04-25T01:20:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.333+0000 7f578b800640 1 Processor -- start 2024-04-25T01:20:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.333+0000 7f578b800640 1 -- start start 2024-04-25T01:20:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.334+0000 7f578b800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5784107dd0 0x7f578419de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.334+0000 7f578b800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5784108fc0 0x7f578419e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.334+0000 7f578b800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5784109980 0x7f57841a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.335+0000 7f5788e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5784107dd0 0x7f578419de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.335+0000 7f5788e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5784107dd0 0x7f578419de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:59116/0 (socket says 172.21.15.31:59116) 2024-04-25T01:20:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.335+0000 7f578b800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5784113a70 con 0x7f5784108fc0 2024-04-25T01:20:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.335+0000 7f578b800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f57841138f0 con 0x7f5784107dd0 2024-04-25T01:20:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.335+0000 7f578b800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5784113bf0 con 0x7f5784109980 2024-04-25T01:20:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.335+0000 7f5789800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5784109980 0x7f57841a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.335+0000 7f5789800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5784109980 0x7f57841a53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:40658/0 (socket says 172.21.15.31:40658) 2024-04-25T01:20:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.335+0000 7f5789800640 1 -- 172.21.15.31:0/2045839488 learned_addr learned my addr 172.21.15.31:0/2045839488 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:20:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.335+0000 7f577be00640 1 --2- 172.21.15.31:0/2045839488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5784108fc0 0x7f578419e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.336+0000 7f5789800640 1 -- 172.21.15.31:0/2045839488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5784107dd0 msgr2=0x7f578419de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.336+0000 7f5789800640 1 --2- 172.21.15.31:0/2045839488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5784107dd0 0x7f578419de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.336+0000 7f5789800640 1 -- 172.21.15.31:0/2045839488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5784108fc0 msgr2=0x7f578419e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.336+0000 7f5789800640 1 --2- 172.21.15.31:0/2045839488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5784108fc0 0x7f578419e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.336+0000 7f5789800640 1 -- 172.21.15.31:0/2045839488 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57841a5a20 con 0x7f5784109980 2024-04-25T01:20:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.336+0000 7f5789800640 1 --2- 172.21.15.31:0/2045839488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5784109980 0x7f57841a53b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f577c00eaa0 tx=0x7f577c00ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:20:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.337+0000 7f5782000640 1 -- 172.21.15.31:0/2045839488 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c00cdb0 con 0x7f5784109980 2024-04-25T01:20:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.337+0000 7f5782000640 1 -- 172.21.15.31:0/2045839488 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f577c005020 con 0x7f5784109980 2024-04-25T01:20:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.337+0000 7f5782000640 1 -- 172.21.15.31:0/2045839488 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c0116d0 con 0x7f5784109980 2024-04-25T01:20:05.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.337+0000 7f578b800640 1 -- 172.21.15.31:0/2045839488 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57841a5d10 con 0x7f5784109980 2024-04-25T01:20:05.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.337+0000 7f578b800640 1 -- 172.21.15.31:0/2045839488 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57841a61b0 con 0x7f5784109980 2024-04-25T01:20:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.344+0000 7f578e7d9740 1 Processor -- start 2024-04-25T01:20:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.358+0000 7f5782000640 1 -- 172.21.15.31:0/2045839488 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f577c004a40 con 0x7f5784109980 2024-04-25T01:20:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.358+0000 7f578e7d9740 1 -- start start 2024-04-25T01:20:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.359+0000 7f578e7d9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9376feca0 0x55f9376ff0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.359+0000 7f578e7d9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9376ff690 0x55f937704190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.359+0000 7f578e7d9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f937704800 0x55f937706c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.359+0000 7f578e7d9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f937661ab0 con 0x55f937704800 2024-04-25T01:20:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.359+0000 7f578e7d9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f93760a640 con 0x55f9376ff690 2024-04-25T01:20:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.359+0000 7f578e7d9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f93765da70 con 0x55f9376feca0 2024-04-25T01:20:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.360+0000 7f5782000640 1 --2- 172.21.15.31:0/2045839488 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f57680a2e20 0x7f57680a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.360+0000 7f5788e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9376ff690 0x55f937704190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.361+0000 7f5788e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9376ff690 0x55f937704190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:59130/0 (socket says 172.21.15.31:59130) 2024-04-25T01:20:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.361+0000 7f5788e00640 1 -- 172.21.15.31:0/3686166342 learned_addr learned my addr 172.21.15.31:0/3686166342 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:20:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.361+0000 7f5782000640 1 -- 172.21.15.31:0/2045839488 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f577c0cf160 con 0x7f5784109980 2024-04-25T01:20:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.361+0000 7f577be00640 1 --2- 172.21.15.31:0/3686166342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9376feca0 0x55f9376ff0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.362+0000 7f5789800640 1 --2- 172.21.15.31:0/2045839488 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f57680a2e20 0x7f57680a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.362+0000 7f5788e00640 1 -- 172.21.15.31:0/3686166342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9376feca0 msgr2=0x55f9376ff0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.362+0000 7f5788e00640 1 --2- 172.21.15.31:0/3686166342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9376feca0 0x55f9376ff0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.362+0000 7f5788e00640 1 -- 172.21.15.31:0/3686166342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f937704800 msgr2=0x55f937706c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.362+0000 7f5788e00640 1 --2- 172.21.15.31:0/3686166342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f937704800 0x55f937706c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.362+0000 7f5788e00640 1 -- 172.21.15.31:0/3686166342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f937376210 con 0x55f9376ff690 2024-04-25T01:20:05.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.363+0000 7f5788e00640 1 --2- 172.21.15.31:0/3686166342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9376ff690 0x55f937704190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5774004fd0 tx=0x7f57740035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e364482e0c02db1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:20:05.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.364+0000 7f577a000640 1 -- 172.21.15.31:0/3686166342 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774041070 con 0x55f9376ff690 2024-04-25T01:20:05.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.364+0000 7f5789800640 1 --2- 172.21.15.31:0/2045839488 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f57680a2e20 0x7f57680a52e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f577c097e10 tx=0x7f577c097c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:20:05.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.364+0000 7f577a000640 1 -- 172.21.15.31:0/3686166342 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57740048c0 con 0x55f9376ff690 2024-04-25T01:20:05.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.365+0000 7f577a000640 1 -- 172.21.15.31:0/3686166342 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774041070 con 0x55f9376ff690 2024-04-25T01:20:05.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.366+0000 7f578e7d9740 1 -- 172.21.15.31:0/3686166342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9376ff690 msgr2=0x55f937704190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.366+0000 7f578e7d9740 1 --2- 172.21.15.31:0/3686166342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9376ff690 0x55f937704190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5774004fd0 tx=0x7f57740035f0 comp rx=0 tx=0).stop 2024-04-25T01:20:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.366+0000 7f578e7d9740 1 -- 172.21.15.31:0/3686166342 shutdown_connections 2024-04-25T01:20:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.366+0000 7f578e7d9740 1 --2- 172.21.15.31:0/3686166342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f937704800 0x55f937706c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.366+0000 7f578e7d9740 1 --2- 172.21.15.31:0/3686166342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9376ff690 0x55f937704190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.366+0000 7f578e7d9740 1 --2- 172.21.15.31:0/3686166342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9376feca0 0x55f9376ff0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.366+0000 7f578e7d9740 1 -- 172.21.15.31:0/3686166342 >> 172.21.15.31:0/3686166342 conn(0x55f9376fa310 msgr2=0x55f9376fa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:20:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.366+0000 7f578e7d9740 1 -- 172.21.15.31:0/3686166342 shutdown_connections 2024-04-25T01:20:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.366+0000 7f578e7d9740 1 -- 172.21.15.31:0/3686166342 wait complete. 2024-04-25T01:20:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.369+0000 7f578e7d9740 1 -- start start 2024-04-25T01:20:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.369+0000 7f578e7d9740 1 Processor -- start 2024-04-25T01:20:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.370+0000 7f578e7d9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9376feca0 0x55f937708e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.370+0000 7f578e7d9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9376ff690 0x55f9377093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.370+0000 7f578e7d9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f937704800 0x55f937710410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.370+0000 7f578e7d9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f9377077f0 con 0x55f9376feca0 2024-04-25T01:20:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.370+0000 7f578e7d9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55f93761d990 con 0x55f9376ff690 2024-04-25T01:20:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.370+0000 7f578e7d9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f937707970 con 0x55f937704800 2024-04-25T01:20:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.371+0000 7f578e7d9740 10 client.0 _ll_register_callbacks cb 0x55f93761f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:20:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.371+0000 7f577be00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9376feca0 0x55f937708e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.371+0000 7f577be00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9376feca0 0x55f937708e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46626/0 (socket says 172.21.15.31:46626) 2024-04-25T01:20:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.371+0000 7f577be00640 1 -- 172.21.15.31:0/1144252090 learned_addr learned my addr 172.21.15.31:0/1144252090 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:20:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.371+0000 7f5788e00640 1 --2- 172.21.15.31:0/1144252090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9376ff690 0x55f9377093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.372+0000 7f577be00640 1 --2- 172.21.15.31:0/1144252090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f937704800 0x55f937710410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.372+0000 7f577be00640 1 -- 172.21.15.31:0/1144252090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f937704800 msgr2=0x55f937710410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.372+0000 7f577be00640 1 --2- 172.21.15.31:0/1144252090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f937704800 0x55f937710410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.372+0000 7f577be00640 1 -- 172.21.15.31:0/1144252090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9376ff690 msgr2=0x55f9377093d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.373+0000 7f577be00640 1 --2- 172.21.15.31:0/1144252090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9376ff690 0x55f9377093d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.373+0000 7f577be00640 1 -- 172.21.15.31:0/1144252090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f937442620 con 0x55f9376feca0 2024-04-25T01:20:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.375+0000 7f577be00640 1 --2- 172.21.15.31:0/1144252090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9376feca0 0x55f937708e90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f576c009dc0 tx=0x7f576c0096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463196b82a7707c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:20:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.375+0000 7f5772a00640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T01:20:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.377+0000 7f5772a00640 1 -- 172.21.15.31:0/1144252090 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c01b040 con 0x55f9376feca0 2024-04-25T01:20:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.377+0000 7f5772a00640 1 -- 172.21.15.31:0/1144252090 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f576c010e40 con 0x55f9376feca0 2024-04-25T01:20:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.377+0000 7f578e7d9740 1 -- 172.21.15.31:0/1144252090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57740048c0 con 0x55f9376feca0 2024-04-25T01:20:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.378+0000 7f5772a00640 1 -- 172.21.15.31:0/1144252090 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f576c026050 con 0x55f9376feca0 2024-04-25T01:20:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.378+0000 7f578e7d9740 10 client.15303 fetch_fsmap learned FSMap version 27 2024-04-25T01:20:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.378+0000 7f578e7d9740 1 -- 172.21.15.31:0/1144252090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f9377134c0 con 0x55f9376feca0 2024-04-25T01:20:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.379+0000 7f5772a00640 1 -- 172.21.15.31:0/1144252090 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57740048c0 con 0x55f9376feca0 2024-04-25T01:20:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.382+0000 7f5772a00640 1 -- 172.21.15.31:0/1144252090 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7f576c020420 con 0x55f9376feca0 2024-04-25T01:20:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.383+0000 7f578e7d9740 10 client.15303 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:20:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.383+0000 7f578e7d9740 10 client.15303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:20:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.383+0000 7f578e7d9740 20 client.15303 populate_metadata read hostname 'smithi031' 2024-04-25T01:20:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.383+0000 7f578e7d9740 1 --2- 172.21.15.31:0/1144252090 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55f937714bb0 0x55f937717040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.383+0000 7f578e7d9740 4 client.15303 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:20:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.383+0000 7f578e7d9740 1 -- 172.21.15.31:0/1144252090 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f93766eff0 con 0x55f937714bb0 2024-04-25T01:20:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.384+0000 7f5788e00640 1 --2- 172.21.15.31:0/1144252090 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55f937714bb0 0x55f937717040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.385+0000 7f5788e00640 1 --2- 172.21.15.31:0/1144252090 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55f937714bb0 0x55f937717040 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58355f51cf35e4c8 server_cookie=84d52f08fcb38766 in_seq=0 out_seq=0 2024-04-25T01:20:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.386+0000 7f5772a00640 10 client.15303 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:20:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.387+0000 7f5772a00640 1 -- 172.21.15.31:0/1144252090 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f93766eff0 con 0x55f937714bb0 2024-04-25T01:20:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.387+0000 7f5772a00640 10 client.15303 handle_command_reply: tid=0 2024-04-25T01:20:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.388+0000 7f578e7d9740 1 client.15303 shutdown 2024-04-25T01:20:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.388+0000 7f578e7d9740 1 -- 172.21.15.31:0/1144252090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9376feca0 msgr2=0x55f937708e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.388+0000 7f578e7d9740 1 --2- 172.21.15.31:0/1144252090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9376feca0 0x55f937708e90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f576c009dc0 tx=0x7f576c0096f0 comp rx=0 tx=0).stop 2024-04-25T01:20:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.389+0000 7f578e7d9740 1 -- 172.21.15.31:0/1144252090 shutdown_connections 2024-04-25T01:20:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.389+0000 7f578e7d9740 1 --2- 172.21.15.31:0/1144252090 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55f937714bb0 0x55f937717040 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.389+0000 7f578e7d9740 1 --2- 172.21.15.31:0/1144252090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f937704800 0x55f937710410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.389+0000 7f578e7d9740 1 --2- 172.21.15.31:0/1144252090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f9376ff690 0x55f9377093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.389+0000 7f578e7d9740 1 --2- 172.21.15.31:0/1144252090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9376feca0 0x55f937708e90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.389+0000 7f578e7d9740 1 -- 172.21.15.31:0/1144252090 >> 172.21.15.31:0/1144252090 conn(0x55f9376fa310 msgr2=0x55f9376f9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:20:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.389+0000 7f5772a00640 0 client.15303 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:20:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.390+0000 7f578e7d9740 1 -- 172.21.15.31:0/1144252090 shutdown_connections 2024-04-25T01:20:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.390+0000 7f578e7d9740 1 -- 172.21.15.31:0/1144252090 wait complete. 2024-04-25T01:20:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.391+0000 7f578e7d9740 20 client.15303 trim_cache size 0 max 16384 2024-04-25T01:20:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.443+0000 7f578e7d9740 1 Processor -- start 2024-04-25T01:20:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.444+0000 7f578e7d9740 1 -- start start 2024-04-25T01:20:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.445+0000 7f578e7d9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9376ff690 0x55f9376f9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.445+0000 7f578e7d9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f937704800 0x55f9377103d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.445+0000 7f578e7d9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f93771b940 0x55f93771bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.445+0000 7f578e7d9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5760010d10 con 0x55f9376ff690 2024-04-25T01:20:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.445+0000 7f578e7d9740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f576000e580 con 0x55f937704800 2024-04-25T01:20:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.445+0000 7f578e7d9740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f576000c3d0 con 0x55f93771b940 2024-04-25T01:20:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.446+0000 7f5788e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f937704800 0x55f9377103d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.447+0000 7f5788e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f937704800 0x55f9377103d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:59152/0 (socket says 172.21.15.31:59152) 2024-04-25T01:20:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.447+0000 7f5788e00640 1 -- 172.21.15.31:0/3394606722 learned_addr learned my addr 172.21.15.31:0/3394606722 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:20:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.448+0000 7f577be00640 1 --2- 172.21.15.31:0/3394606722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f93771b940 0x55f93771bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.448+0000 7f577be00640 1 --2- 172.21.15.31:0/3394606722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9376ff690 0x55f9376f9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.448+0000 7f577be00640 1 -- 172.21.15.31:0/3394606722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f937704800 msgr2=0x55f9377103d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.448+0000 7f577be00640 1 --2- 172.21.15.31:0/3394606722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f937704800 0x55f9377103d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.448+0000 7f577be00640 1 -- 172.21.15.31:0/3394606722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9376ff690 msgr2=0x55f9376f9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.448+0000 7f577be00640 1 --2- 172.21.15.31:0/3394606722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9376ff690 0x55f9376f9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.448+0000 7f577be00640 1 -- 172.21.15.31:0/3394606722 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9376fe8e0 con 0x55f93771b940 2024-04-25T01:20:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.449+0000 7f577be00640 1 --2- 172.21.15.31:0/3394606722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f93771b940 0x55f93771bda0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f576c001eb0 tx=0x7f576c005e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7d74e50f15373e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:20:05.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.451+0000 7f5779600640 1 -- 172.21.15.31:0/3394606722 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c010df0 con 0x55f93771b940 2024-04-25T01:20:05.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.451+0000 7f5779600640 1 -- 172.21.15.31:0/3394606722 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f576c005590 con 0x55f93771b940 2024-04-25T01:20:05.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.451+0000 7f5779600640 1 -- 172.21.15.31:0/3394606722 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c0035f0 con 0x55f93771b940 2024-04-25T01:20:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.452+0000 7f578e7d9740 1 -- 172.21.15.31:0/3394606722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f93771b940 msgr2=0x55f93771bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.452+0000 7f578e7d9740 1 --2- 172.21.15.31:0/3394606722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f93771b940 0x55f93771bda0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f576c001eb0 tx=0x7f576c005e30 comp rx=0 tx=0).stop 2024-04-25T01:20:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.453+0000 7f578e7d9740 1 -- 172.21.15.31:0/3394606722 shutdown_connections 2024-04-25T01:20:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.453+0000 7f578e7d9740 1 --2- 172.21.15.31:0/3394606722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f93771b940 0x55f93771bda0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.453+0000 7f578e7d9740 1 --2- 172.21.15.31:0/3394606722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f937704800 0x55f9377103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.453+0000 7f578e7d9740 1 --2- 172.21.15.31:0/3394606722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9376ff690 0x55f9376f9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.453+0000 7f578e7d9740 1 -- 172.21.15.31:0/3394606722 >> 172.21.15.31:0/3394606722 conn(0x55f9376fa310 msgr2=0x55f937709110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:20:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.454+0000 7f578e7d9740 1 -- 172.21.15.31:0/3394606722 shutdown_connections 2024-04-25T01:20:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.454+0000 7f578e7d9740 1 -- 172.21.15.31:0/3394606722 wait complete. 2024-04-25T01:20:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.455+0000 7f578e7d9740 1 -- start start 2024-04-25T01:20:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.455+0000 7f578e7d9740 1 Processor -- start 2024-04-25T01:20:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.459+0000 7f578e7d9740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9376ff690 0x55f937710360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.459+0000 7f578e7d9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f937704800 0x55f93770e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.460+0000 7f578e7d9740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f93771b940 0x55f93770ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.460+0000 7f577be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9376ff690 0x55f937710360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.460+0000 7f578e7d9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f576000ca40 con 0x55f937704800 2024-04-25T01:20:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.460+0000 7f577be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9376ff690 0x55f937710360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:40682/0 (socket says 172.21.15.31:40682) 2024-04-25T01:20:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.460+0000 7f577be00640 1 -- 172.21.15.31:0/839719685 learned_addr learned my addr 172.21.15.31:0/839719685 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:20:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.461+0000 7f578e7d9740 1 -- 172.21.15.31:0/839719685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f576000deb0 con 0x55f93771b940 2024-04-25T01:20:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.461+0000 7f578e7d9740 1 -- 172.21.15.31:0/839719685 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f576000d110 con 0x55f9376ff690 2024-04-25T01:20:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.461+0000 7f578e7d9740 10 client.0 _ll_register_callbacks cb 0x7f576c0098a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:20:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.461+0000 7f5788e00640 1 --2- 172.21.15.31:0/839719685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f937704800 0x55f93770e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.462+0000 7f577be00640 1 --2- 172.21.15.31:0/839719685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f93771b940 0x55f93770ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.462+0000 7f577be00640 1 -- 172.21.15.31:0/839719685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f93771b940 msgr2=0x55f93770ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.462+0000 7f577be00640 1 --2- 172.21.15.31:0/839719685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f93771b940 0x55f93770ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.462+0000 7f577be00640 1 -- 172.21.15.31:0/839719685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f937704800 msgr2=0x55f93770e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.462+0000 7f577be00640 1 --2- 172.21.15.31:0/839719685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f937704800 0x55f93770e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.462+0000 7f577be00640 1 -- 172.21.15.31:0/839719685 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f93777b1c0 con 0x55f9376ff690 2024-04-25T01:20:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.463+0000 7f5788e00640 1 --2- 172.21.15.31:0/839719685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f937704800 0x55f93770e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:20:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.463+0000 7f577be00640 1 --2- 172.21.15.31:0/839719685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9376ff690 0x55f937710360 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f576c009730 tx=0x7f576c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2b03bf8b5a43366 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:20:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.463+0000 7f5772a00640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:20:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.463+0000 7f5772a00640 1 -- 172.21.15.31:0/839719685 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c02f070 con 0x55f9376ff690 2024-04-25T01:20:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.463+0000 7f5772a00640 1 -- 172.21.15.31:0/839719685 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f576c010df0 con 0x55f9376ff690 2024-04-25T01:20:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.464+0000 7f578e7d9740 1 -- 172.21.15.31:0/839719685 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f576c005590 con 0x55f9376ff690 2024-04-25T01:20:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.464+0000 7f5772a00640 1 -- 172.21.15.31:0/839719685 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f576c020b70 con 0x55f9376ff690 2024-04-25T01:20:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.466+0000 7f578e7d9740 10 client.25531 fetch_fsmap learned FSMap version 27 2024-04-25T01:20:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.466+0000 7f578e7d9740 1 -- 172.21.15.31:0/839719685 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f93777c2c0 con 0x55f9376ff690 2024-04-25T01:20:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.466+0000 7f5772a00640 1 -- 172.21.15.31:0/839719685 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f576c005590 con 0x55f9376ff690 2024-04-25T01:20:05.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.467+0000 7f5772a00640 1 -- 172.21.15.31:0/839719685 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7f576c02bc50 con 0x55f9376ff690 2024-04-25T01:20:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.468+0000 7f578e7d9740 10 client.25531 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:20:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.468+0000 7f578e7d9740 10 client.25531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:20:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.468+0000 7f578e7d9740 20 client.25531 populate_metadata read hostname 'smithi031' 2024-04-25T01:20:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.468+0000 7f578e7d9740 1 --2- 172.21.15.31:0/839719685 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55f93777cc00 0x55f93777f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.468+0000 7f578e7d9740 4 client.25531 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:20:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.468+0000 7f578e7d9740 1 -- 172.21.15.31:0/839719685 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f937669340 con 0x55f93777cc00 2024-04-25T01:20:05.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.471+0000 7f5788e00640 1 --2- 172.21.15.31:0/839719685 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55f93777cc00 0x55f93777f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.472+0000 7f5788e00640 1 --2- 172.21.15.31:0/839719685 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55f93777cc00 0x55f93777f020 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc2867b8d0c518cb server_cookie=a9f470b136576ba in_seq=0 out_seq=0 2024-04-25T01:20:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:05.472+0000 7f5772a00640 10 client.25531 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:20:05.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:05 smithi169 ceph-mon[32759]: pgmap v515: 97 pgs: 97 active+clean; 13 GiB data, 67 GiB used, 1006 GiB / 1.0 TiB avail; 49 KiB/s rd, 140 MiB/s wr, 18.78k op/s 2024-04-25T01:20:05.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:05 smithi040 ceph-mon[32518]: pgmap v515: 97 pgs: 97 active+clean; 13 GiB data, 67 GiB used, 1006 GiB / 1.0 TiB avail; 49 KiB/s rd, 140 MiB/s wr, 18.78k op/s 2024-04-25T01:20:06.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:05 smithi031 ceph-mon[28053]: pgmap v515: 97 pgs: 97 active+clean; 13 GiB data, 67 GiB used, 1006 GiB / 1.0 TiB avail; 49 KiB/s rd, 140 MiB/s wr, 18.78k op/s 2024-04-25T01:20:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.673+0000 7f5772a00640 1 -- 172.21.15.31:0/839719685 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f937669340 con 0x55f93777cc00 2024-04-25T01:20:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.673+0000 7f5772a00640 10 client.25531 handle_command_reply: tid=0 2024-04-25T01:20:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.673+0000 7f578e7d9740 1 client.25531 shutdown 2024-04-25T01:20:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.674+0000 7f578e7d9740 1 -- 172.21.15.31:0/839719685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9376ff690 msgr2=0x55f937710360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.674+0000 7f578e7d9740 1 --2- 172.21.15.31:0/839719685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9376ff690 0x55f937710360 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f576c009730 tx=0x7f576c009260 comp rx=0 tx=0).stop 2024-04-25T01:20:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.675+0000 7f578e7d9740 1 -- 172.21.15.31:0/839719685 shutdown_connections 2024-04-25T01:20:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.675+0000 7f578e7d9740 1 --2- 172.21.15.31:0/839719685 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55f93777cc00 0x55f93777f020 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.675+0000 7f5772a00640 0 client.25531 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:20:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.675+0000 7f578e7d9740 1 --2- 172.21.15.31:0/839719685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f93771b940 0x55f93770ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.675+0000 7f578e7d9740 1 --2- 172.21.15.31:0/839719685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f937704800 0x55f93770e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.675+0000 7f578e7d9740 1 --2- 172.21.15.31:0/839719685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9376ff690 0x55f937710360 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.675+0000 7f578e7d9740 1 -- 172.21.15.31:0/839719685 >> 172.21.15.31:0/839719685 conn(0x55f9376fa310 msgr2=0x55f9376fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:20:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.675+0000 7f578e7d9740 1 -- 172.21.15.31:0/839719685 shutdown_connections 2024-04-25T01:20:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.675+0000 7f578e7d9740 1 -- 172.21.15.31:0/839719685 wait complete. 2024-04-25T01:20:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.676+0000 7f578e7d9740 20 client.25531 trim_cache size 0 max 16384 2024-04-25T01:20:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:20:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:20:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:20:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:20:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.677+0000 7f578b800640 1 -- 172.21.15.31:0/2045839488 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f57680a2e20 msgr2=0x7f57680a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.677+0000 7f578b800640 1 --2- 172.21.15.31:0/2045839488 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f57680a2e20 0x7f57680a52e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f577c097e10 tx=0x7f577c097c50 comp rx=0 tx=0).stop 2024-04-25T01:20:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.677+0000 7f578b800640 1 -- 172.21.15.31:0/2045839488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5784109980 msgr2=0x7f57841a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.677+0000 7f578b800640 1 --2- 172.21.15.31:0/2045839488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5784109980 0x7f57841a53b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f577c00eaa0 tx=0x7f577c00ef60 comp rx=0 tx=0).stop 2024-04-25T01:20:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.677+0000 7f578b800640 1 -- 172.21.15.31:0/2045839488 shutdown_connections 2024-04-25T01:20:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.677+0000 7f578b800640 1 --2- 172.21.15.31:0/2045839488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5784109980 0x7f57841a53b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.677+0000 7f578b800640 1 --2- 172.21.15.31:0/2045839488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5784108fc0 0x7f578419e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.677+0000 7f578b800640 1 --2- 172.21.15.31:0/2045839488 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f57680a2e20 0x7f57680a52e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.677+0000 7f578b800640 1 --2- 172.21.15.31:0/2045839488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5784107dd0 0x7f578419de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.677+0000 7f578b800640 1 -- 172.21.15.31:0/2045839488 >> 172.21.15.31:0/2045839488 conn(0x7f5784076d90 msgr2=0x7f578410c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:20:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.680+0000 7f578b800640 1 -- 172.21.15.31:0/2045839488 shutdown_connections 2024-04-25T01:20:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:06.680+0000 7f578b800640 1 -- 172.21.15.31:0/2045839488 wait complete. 2024-04-25T01:20:07.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:07 smithi169 ceph-mon[32759]: pgmap v516: 97 pgs: 97 active+clean; 13 GiB data, 68 GiB used, 1004 GiB / 1.0 TiB avail; 68 KiB/s rd, 145 MiB/s wr, 19.63k op/s 2024-04-25T01:20:07.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:07 smithi040 ceph-mon[32518]: pgmap v516: 97 pgs: 97 active+clean; 13 GiB data, 68 GiB used, 1004 GiB / 1.0 TiB avail; 68 KiB/s rd, 145 MiB/s wr, 19.63k op/s 2024-04-25T01:20:08.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:07 smithi031 ceph-mon[28053]: pgmap v516: 97 pgs: 97 active+clean; 13 GiB data, 68 GiB used, 1004 GiB / 1.0 TiB avail; 68 KiB/s rd, 145 MiB/s wr, 19.63k op/s 2024-04-25T01:20:09.940 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:09 smithi169 ceph-mon[32759]: pgmap v517: 97 pgs: 97 active+clean; 13 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 55 KiB/s rd, 125 MiB/s wr, 16.48k op/s 2024-04-25T01:20:09.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:09 smithi040 ceph-mon[32518]: pgmap v517: 97 pgs: 97 active+clean; 13 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 55 KiB/s rd, 125 MiB/s wr, 16.48k op/s 2024-04-25T01:20:10.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:09 smithi031 ceph-mon[28053]: pgmap v517: 97 pgs: 97 active+clean; 13 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 55 KiB/s rd, 125 MiB/s wr, 16.48k op/s 2024-04-25T01:20:11.940 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:11 smithi169 ceph-mon[32759]: pgmap v518: 97 pgs: 97 active+clean; 13 GiB data, 68 GiB used, 1004 GiB / 1.0 TiB avail; 60 KiB/s rd, 147 MiB/s wr, 19.61k op/s 2024-04-25T01:20:11.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:11 smithi040 ceph-mon[32518]: pgmap v518: 97 pgs: 97 active+clean; 13 GiB data, 68 GiB used, 1004 GiB / 1.0 TiB avail; 60 KiB/s rd, 147 MiB/s wr, 19.61k op/s 2024-04-25T01:20:12.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:11 smithi031 ceph-mon[28053]: pgmap v518: 97 pgs: 97 active+clean; 13 GiB data, 68 GiB used, 1004 GiB / 1.0 TiB avail; 60 KiB/s rd, 147 MiB/s wr, 19.61k op/s 2024-04-25T01:20:13.940 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:13 smithi169 ceph-mon[32759]: pgmap v519: 97 pgs: 97 active+clean; 13 GiB data, 69 GiB used, 1004 GiB / 1.0 TiB avail; 41 KiB/s rd, 132 MiB/s wr, 17.10k op/s 2024-04-25T01:20:13.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:13 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:20:13.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:13 smithi040 ceph-mon[32518]: pgmap v519: 97 pgs: 97 active+clean; 13 GiB data, 69 GiB used, 1004 GiB / 1.0 TiB avail; 41 KiB/s rd, 132 MiB/s wr, 17.10k op/s 2024-04-25T01:20:13.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:13 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:20:14.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:13 smithi031 ceph-mon[28053]: pgmap v519: 97 pgs: 97 active+clean; 13 GiB data, 69 GiB used, 1004 GiB / 1.0 TiB avail; 41 KiB/s rd, 132 MiB/s wr, 17.10k op/s 2024-04-25T01:20:14.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:13 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:20:15.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:15 smithi169 ceph-mon[32759]: pgmap v520: 97 pgs: 97 active+clean; 13 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 40 KiB/s rd, 128 MiB/s wr, 16.50k op/s 2024-04-25T01:20:15.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:15 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:20:15.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:15 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:20:15.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:15 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:20:15.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:15 smithi040 ceph-mon[32518]: pgmap v520: 97 pgs: 97 active+clean; 13 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 40 KiB/s rd, 128 MiB/s wr, 16.50k op/s 2024-04-25T01:20:15.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:15 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:20:15.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:15 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:20:15.944 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:15 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:20:16.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:15 smithi031 ceph-mon[28053]: pgmap v520: 97 pgs: 97 active+clean; 13 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 40 KiB/s rd, 128 MiB/s wr, 16.50k op/s 2024-04-25T01:20:16.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:15 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:20:16.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:15 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:20:16.003 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:15 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:20:17.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:17 smithi169 ceph-mon[32759]: pgmap v521: 97 pgs: 97 active+clean; 13 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 61 KiB/s rd, 139 MiB/s wr, 19.15k op/s 2024-04-25T01:20:17.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:17 smithi040 ceph-mon[32518]: pgmap v521: 97 pgs: 97 active+clean; 13 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 61 KiB/s rd, 139 MiB/s wr, 19.15k op/s 2024-04-25T01:20:18.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:17 smithi031 ceph-mon[28053]: pgmap v521: 97 pgs: 97 active+clean; 13 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 61 KiB/s rd, 139 MiB/s wr, 19.15k op/s 2024-04-25T01:20:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.930+0000 7fd8d8200640 1 -- 172.21.15.31:0/3155936501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8d0108740 msgr2=0x7fd8d0108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.930+0000 7fd8cfe00640 1 -- 172.21.15.31:0/3155936501 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc037070 con 0x7fd8d0108740 2024-04-25T01:20:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.930+0000 7fd8d8200640 1 --2- 172.21.15.31:0/3155936501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8d0108740 0x7fd8d0108b60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009860 tx=0x7fd8bc02a340 comp rx=0 tx=0).stop 2024-04-25T01:20:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.930+0000 7fd8d8200640 1 -- 172.21.15.31:0/3155936501 shutdown_connections 2024-04-25T01:20:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.930+0000 7fd8d8200640 1 --2- 172.21.15.31:0/3155936501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8d010ffc0 0x7fd8d01123b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.930+0000 7fd8d8200640 1 --2- 172.21.15.31:0/3155936501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8d01090a0 0x7fd8d010fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.930+0000 7fd8d8200640 1 --2- 172.21.15.31:0/3155936501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8d0108740 0x7fd8d0108b60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.931+0000 7fd8d8200640 1 -- 172.21.15.31:0/3155936501 >> 172.21.15.31:0/3155936501 conn(0x7fd8d006e7b0 msgr2=0x7fd8d0070bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:20:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.931+0000 7fd8d8200640 1 -- 172.21.15.31:0/3155936501 shutdown_connections 2024-04-25T01:20:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.931+0000 7fd8d8200640 1 -- 172.21.15.31:0/3155936501 wait complete. 2024-04-25T01:20:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.933+0000 7fd8d8200640 1 Processor -- start 2024-04-25T01:20:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.933+0000 7fd8d8200640 1 -- start start 2024-04-25T01:20:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.934+0000 7fd8d8200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8d0108740 0x7fd8d019ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.934+0000 7fd8d8200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8d01090a0 0x7fd8d01a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.934+0000 7fd8d5800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8d0108740 0x7fd8d019ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.934+0000 7fd8d5800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8d0108740 0x7fd8d019ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:48358/0 (socket says 172.21.15.31:48358) 2024-04-25T01:20:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.935+0000 7fd8d5800640 1 -- 172.21.15.31:0/2954794794 learned_addr learned my addr 172.21.15.31:0/2954794794 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:20:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.935+0000 7fd8d8200640 1 --2- 172.21.15.31:0/2954794794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8d010ffc0 0x7fd8d01a74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.936+0000 7fd8d8200640 1 -- 172.21.15.31:0/2954794794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d0115c80 con 0x7fd8d010ffc0 2024-04-25T01:20:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.936+0000 7fd8d8200640 1 -- 172.21.15.31:0/2954794794 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d0115b00 con 0x7fd8d01090a0 2024-04-25T01:20:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.936+0000 7fd8d8200640 1 -- 172.21.15.31:0/2954794794 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d0115e00 con 0x7fd8d0108740 2024-04-25T01:20:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.936+0000 7fd8d4e00640 1 --2- 172.21.15.31:0/2954794794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8d01090a0 0x7fd8d01a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.936+0000 7fd8d5800640 1 -- 172.21.15.31:0/2954794794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8d01090a0 msgr2=0x7fd8d01a0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.936+0000 7fd8d5800640 1 --2- 172.21.15.31:0/2954794794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8d01090a0 0x7fd8d01a0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.937+0000 7fd8d6200640 1 --2- 172.21.15.31:0/2954794794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8d010ffc0 0x7fd8d01a74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.937+0000 7fd8d5800640 1 -- 172.21.15.31:0/2954794794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8d010ffc0 msgr2=0x7fd8d01a74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.937+0000 7fd8d5800640 1 --2- 172.21.15.31:0/2954794794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8d010ffc0 0x7fd8d01a74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.937+0000 7fd8d5800640 1 -- 172.21.15.31:0/2954794794 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d01a7b60 con 0x7fd8d0108740 2024-04-25T01:20:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.937+0000 7fd8d6200640 1 --2- 172.21.15.31:0/2954794794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8d010ffc0 0x7fd8d01a74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:20:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.937+0000 7fd8d4e00640 1 --2- 172.21.15.31:0/2954794794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8d01090a0 0x7fd8d01a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:20:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.938+0000 7fd8d5800640 1 --2- 172.21.15.31:0/2954794794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8d0108740 0x7fd8d019ff40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc0042f0 tx=0x7fd8bc004320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:20:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.941+0000 7fd8ce000640 1 -- 172.21.15.31:0/2954794794 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc037040 con 0x7fd8d0108740 2024-04-25T01:20:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.941+0000 7fd8d8200640 1 -- 172.21.15.31:0/2954794794 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8d01a7df0 con 0x7fd8d0108740 2024-04-25T01:20:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.942+0000 7fd8ce000640 1 -- 172.21.15.31:0/2954794794 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8bc042070 con 0x7fd8d0108740 2024-04-25T01:20:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.942+0000 7fd8ce000640 1 -- 172.21.15.31:0/2954794794 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc037040 con 0x7fd8d0108740 2024-04-25T01:20:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.943+0000 7fd8d8200640 1 -- 172.21.15.31:0/2954794794 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8d01a8250 con 0x7fd8d0108740 2024-04-25T01:20:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.953+0000 7fd8ce000640 1 -- 172.21.15.31:0/2954794794 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd8bc06ab30 con 0x7fd8d0108740 2024-04-25T01:20:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.957+0000 7fd8ce000640 1 --2- 172.21.15.31:0/2954794794 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd8b80a2f80 0x7fd8b80a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.957+0000 7fd8ce000640 1 -- 172.21.15.31:0/2954794794 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fd8bc0eef60 con 0x7fd8d0108740 2024-04-25T01:20:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.958+0000 7fd8db039740 1 Processor -- start 2024-04-25T01:20:19.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.958+0000 7fd8db039740 1 -- start start 2024-04-25T01:20:19.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.959+0000 7fd8d4e00640 1 --2- 172.21.15.31:0/2954794794 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd8b80a2f80 0x7fd8b80a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:19.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.959+0000 7fd8db039740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557581bb2ca0 0x557581bb30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:19.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.959+0000 7fd8db039740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557581bb3690 0x557581bb8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:19.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.959+0000 7fd8db039740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557581bb8800 0x557581bbac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:19.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.959+0000 7fd8db039740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557581b15ab0 con 0x557581bb2ca0 2024-04-25T01:20:19.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.959+0000 7fd8db039740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x557581abe640 con 0x557581bb8800 2024-04-25T01:20:19.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.959+0000 7fd8db039740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557581b11a70 con 0x557581bb3690 2024-04-25T01:20:19.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.959+0000 7fd8d4e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557581bb3690 0x557581bb8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.960+0000 7fd8d4e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557581bb3690 0x557581bb8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:48360/0 (socket says 172.21.15.31:48360) 2024-04-25T01:20:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.960+0000 7fd8d4e00640 1 -- 172.21.15.31:0/4063359723 learned_addr learned my addr 172.21.15.31:0/4063359723 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:20:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.960+0000 7fd8d6200640 1 --2- 172.21.15.31:0/4063359723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557581bb8800 0x557581bbac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.960+0000 7fd8d4e00640 1 -- 172.21.15.31:0/4063359723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557581bb8800 msgr2=0x557581bbac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.960+0000 7fd8d4e00640 1 --2- 172.21.15.31:0/4063359723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557581bb8800 0x557581bbac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.960+0000 7fd8d4e00640 1 -- 172.21.15.31:0/4063359723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557581bb2ca0 msgr2=0x557581bb30c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:20:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.960+0000 7fd8d4e00640 1 --2- 172.21.15.31:0/4063359723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557581bb2ca0 0x557581bb30c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.960+0000 7fd8d4e00640 1 -- 172.21.15.31:0/4063359723 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55758182a210 con 0x557581bb3690 2024-04-25T01:20:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.961+0000 7fd8d4e00640 1 --2- 172.21.15.31:0/4063359723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557581bb3690 0x557581bb8190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80139c0 tx=0x7fd8c8013e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87f66447628bea2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:20:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.962+0000 7fd8c2a00640 1 -- 172.21.15.31:0/4063359723 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8022070 con 0x557581bb3690 2024-04-25T01:20:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.962+0000 7fd8c2a00640 1 -- 172.21.15.31:0/4063359723 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8c8010c80 con 0x557581bb3690 2024-04-25T01:20:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.962+0000 7fd8c2a00640 1 -- 172.21.15.31:0/4063359723 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c800d6c0 con 0x557581bb3690 2024-04-25T01:20:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.965+0000 7fd8db039740 1 -- 172.21.15.31:0/4063359723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557581bb3690 msgr2=0x557581bb8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.965+0000 7fd8db039740 1 --2- 172.21.15.31:0/4063359723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557581bb3690 0x557581bb8190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80139c0 tx=0x7fd8c8013e80 comp rx=0 tx=0).stop 2024-04-25T01:20:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.965+0000 7fd8d4e00640 1 --2- 172.21.15.31:0/2954794794 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd8b80a2f80 0x7fd8b80a5440 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8013fd0 tx=0x7fd8c800d900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:20:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.968+0000 7fd8db039740 1 -- 172.21.15.31:0/4063359723 shutdown_connections 2024-04-25T01:20:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.968+0000 7fd8db039740 1 --2- 172.21.15.31:0/4063359723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557581bb8800 0x557581bbac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.968+0000 7fd8db039740 1 --2- 172.21.15.31:0/4063359723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557581bb3690 0x557581bb8190 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.968+0000 7fd8db039740 1 --2- 172.21.15.31:0/4063359723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557581bb2ca0 0x557581bb30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.968+0000 7fd8db039740 1 -- 172.21.15.31:0/4063359723 >> 172.21.15.31:0/4063359723 conn(0x557581bae310 msgr2=0x557581bae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:20:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.969+0000 7fd8db039740 1 -- 172.21.15.31:0/4063359723 shutdown_connections 2024-04-25T01:20:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.969+0000 7fd8db039740 1 -- 172.21.15.31:0/4063359723 wait complete. 2024-04-25T01:20:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.970+0000 7fd8db039740 1 -- start start 2024-04-25T01:20:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.971+0000 7fd8db039740 1 Processor -- start 2024-04-25T01:20:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.972+0000 7fd8db039740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557581bb2ca0 0x557581bbcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.972+0000 7fd8db039740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557581bb3690 0x557581bbd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.973+0000 7fd8db039740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557581bb8800 0x557581bc4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.973+0000 7fd8db039740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557581bbb7b0 con 0x557581bb2ca0 2024-04-25T01:20:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.973+0000 7fd8db039740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x557581ad1990 con 0x557581bb8800 2024-04-25T01:20:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.973+0000 7fd8d6200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557581bb8800 0x557581bc4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.973+0000 7fd8d4e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557581bb3690 0x557581bbd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.973+0000 7fd8d6200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557581bb8800 0x557581bc4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:60026/0 (socket says 172.21.15.31:60026) 2024-04-25T01:20:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.973+0000 7fd8d6200640 1 -- 172.21.15.31:0/3059992412 learned_addr learned my addr 172.21.15.31:0/3059992412 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:20:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.974+0000 7fd8db039740 1 -- 172.21.15.31:0/3059992412 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557581bbb930 con 0x557581bb3690 2024-04-25T01:20:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.974+0000 7fd8db039740 10 client.0 _ll_register_callbacks cb 0x557581ad3950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:20:19.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.974+0000 7fd8d5800640 1 --2- 172.21.15.31:0/3059992412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557581bb2ca0 0x557581bbcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.977+0000 7fd8d4e00640 1 -- 172.21.15.31:0/3059992412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557581bb8800 msgr2=0x557581bc4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.977+0000 7fd8d4e00640 1 --2- 172.21.15.31:0/3059992412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557581bb8800 0x557581bc4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.977+0000 7fd8d4e00640 1 -- 172.21.15.31:0/3059992412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557581bb2ca0 msgr2=0x557581bbcf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.977+0000 7fd8d4e00640 1 --2- 172.21.15.31:0/3059992412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557581bb2ca0 0x557581bbcf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.977+0000 7fd8d4e00640 1 -- 172.21.15.31:0/3059992412 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575818f6620 con 0x557581bb3690 2024-04-25T01:20:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.977+0000 7fd8d4e00640 1 --2- 172.21.15.31:0/3059992412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557581bb3690 0x557581bbd4b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80139c0 tx=0x7fd8c8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=756d6675ed0fc0ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:20:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.979+0000 7fd8b7400640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:20:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.979+0000 7fd8b7400640 1 -- 172.21.15.31:0/3059992412 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8034060 con 0x557581bb3690 2024-04-25T01:20:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.979+0000 7fd8b7400640 1 -- 172.21.15.31:0/3059992412 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fd8c802c070 con 0x557581bb3690 2024-04-25T01:20:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.980+0000 7fd8db039740 1 -- 172.21.15.31:0/3059992412 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8c8010c80 con 0x557581bb3690 2024-04-25T01:20:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.981+0000 7fd8b7400640 1 -- 172.21.15.31:0/3059992412 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd8c8022040 con 0x557581bb3690 2024-04-25T01:20:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.981+0000 7fd8db039740 10 client.25543 fetch_fsmap learned FSMap version 27 2024-04-25T01:20:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.981+0000 7fd8db039740 1 -- 172.21.15.31:0/3059992412 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557581bc7520 con 0x557581bb3690 2024-04-25T01:20:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.984+0000 7fd8b7400640 1 -- 172.21.15.31:0/3059992412 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8c8010c80 con 0x557581bb3690 2024-04-25T01:20:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.984+0000 7fd8b7400640 1 -- 172.21.15.31:0/3059992412 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7fd8c8005050 con 0x557581bb3690 2024-04-25T01:20:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.984+0000 7fd8db039740 10 client.25543 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:20:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.984+0000 7fd8db039740 10 client.25543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:20:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.984+0000 7fd8db039740 20 client.25543 populate_metadata read hostname 'smithi031' 2024-04-25T01:20:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.984+0000 7fd8db039740 1 --2- 172.21.15.31:0/3059992412 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x557581bc8980 0x557581bcad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.984+0000 7fd8db039740 4 client.25543 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:20:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.984+0000 7fd8db039740 1 -- 172.21.15.31:0/3059992412 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557581b22ff0 con 0x557581bc8980 2024-04-25T01:20:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.984+0000 7fd8d5800640 1 --2- 172.21.15.31:0/3059992412 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x557581bc8980 0x557581bcad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.986+0000 7fd8d5800640 1 --2- 172.21.15.31:0/3059992412 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x557581bc8980 0x557581bcad70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30dfc8d72476c894 server_cookie=e5fe261088f53052 in_seq=0 out_seq=0 2024-04-25T01:20:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.986+0000 7fd8b7400640 10 client.25543 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:20:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.988+0000 7fd8b7400640 1 -- 172.21.15.31:0/3059992412 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557581b22ff0 con 0x557581bc8980 2024-04-25T01:20:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.988+0000 7fd8b7400640 10 client.25543 handle_command_reply: tid=0 2024-04-25T01:20:19.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.988+0000 7fd8db039740 1 client.25543 shutdown 2024-04-25T01:20:19.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.988+0000 7fd8db039740 1 -- 172.21.15.31:0/3059992412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557581bb3690 msgr2=0x557581bbd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:19.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.989+0000 7fd8db039740 1 --2- 172.21.15.31:0/3059992412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557581bb3690 0x557581bbd4b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80139c0 tx=0x7fd8c8004ab0 comp rx=0 tx=0).stop 2024-04-25T01:20:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.991+0000 7fd8db039740 1 -- 172.21.15.31:0/3059992412 shutdown_connections 2024-04-25T01:20:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.991+0000 7fd8db039740 1 --2- 172.21.15.31:0/3059992412 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x557581bc8980 0x557581bcad70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.991+0000 7fd8db039740 1 --2- 172.21.15.31:0/3059992412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557581bb8800 0x557581bc4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.991+0000 7fd8db039740 1 --2- 172.21.15.31:0/3059992412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557581bb3690 0x557581bbd4b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.991+0000 7fd8db039740 1 --2- 172.21.15.31:0/3059992412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557581bb2ca0 0x557581bbcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.991+0000 7fd8db039740 1 -- 172.21.15.31:0/3059992412 >> 172.21.15.31:0/3059992412 conn(0x557581bae310 msgr2=0x557581bad480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:20:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.991+0000 7fd8b7400640 0 client.25543 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:20:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.992+0000 7fd8db039740 1 -- 172.21.15.31:0/3059992412 shutdown_connections 2024-04-25T01:20:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.992+0000 7fd8db039740 1 -- 172.21.15.31:0/3059992412 wait complete. 2024-04-25T01:20:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:19.992+0000 7fd8db039740 20 client.25543 trim_cache size 0 max 16384 2024-04-25T01:20:20.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:19 smithi031 ceph-mon[28053]: pgmap v522: 97 pgs: 97 active+clean; 13 GiB data, 71 GiB used, 1002 GiB / 1.0 TiB avail; 51 KiB/s rd, 117 MiB/s wr, 16.90k op/s 2024-04-25T01:20:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.035+0000 7fd8db039740 1 Processor -- start 2024-04-25T01:20:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.035+0000 7fd8db039740 1 -- start start 2024-04-25T01:20:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.036+0000 7fd8db039740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557581bb2ca0 0x557581bb1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.036+0000 7fd8db039740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557581bb3690 0x557581bc4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.036+0000 7fd8db039740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557581bb8800 0x557581bcf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.036+0000 7fd8db039740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d01b1720 con 0x557581bb2ca0 2024-04-25T01:20:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.036+0000 7fd8db039740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d01aef90 con 0x557581bb8800 2024-04-25T01:20:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.036+0000 7fd8db039740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d01ace00 con 0x557581bb3690 2024-04-25T01:20:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.037+0000 7fd8d5800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557581bb2ca0 0x557581bb1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.037+0000 7fd8d5800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557581bb2ca0 0x557581bb1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42786/0 (socket says 172.21.15.31:42786) 2024-04-25T01:20:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.037+0000 7fd8d5800640 1 -- 172.21.15.31:0/4157765572 learned_addr learned my addr 172.21.15.31:0/4157765572 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:20:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.037+0000 7fd8d4e00640 1 --2- 172.21.15.31:0/4157765572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557581bb3690 0x557581bc4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.037+0000 7fd8d6200640 1 --2- 172.21.15.31:0/4157765572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557581bb8800 0x557581bcf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.037+0000 7fd8d5800640 1 -- 172.21.15.31:0/4157765572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557581bb3690 msgr2=0x557581bc4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.037+0000 7fd8d5800640 1 --2- 172.21.15.31:0/4157765572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557581bb3690 0x557581bc4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.037+0000 7fd8d5800640 1 -- 172.21.15.31:0/4157765572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557581bb8800 msgr2=0x557581bcf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.037+0000 7fd8d5800640 1 --2- 172.21.15.31:0/4157765572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557581bb8800 0x557581bcf5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.037+0000 7fd8d5800640 1 -- 172.21.15.31:0/4157765572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557581bae140 con 0x557581bb2ca0 2024-04-25T01:20:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.038+0000 7fd8d5800640 1 --2- 172.21.15.31:0/4157765572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557581bb2ca0 0x557581bb1700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009830 tx=0x7fd8bc0ec920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67ead6a135e2706 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:20:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.038+0000 7fd8c2000640 1 -- 172.21.15.31:0/4157765572 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc0bb110 con 0x557581bb2ca0 2024-04-25T01:20:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.039+0000 7fd8c2000640 1 -- 172.21.15.31:0/4157765572 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8bc0bbeb0 con 0x557581bb2ca0 2024-04-25T01:20:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.039+0000 7fd8c2000640 1 -- 172.21.15.31:0/4157765572 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc0bb110 con 0x557581bb2ca0 2024-04-25T01:20:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.040+0000 7fd8db039740 1 -- 172.21.15.31:0/4157765572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557581bb2ca0 msgr2=0x557581bb1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.040+0000 7fd8db039740 1 --2- 172.21.15.31:0/4157765572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557581bb2ca0 0x557581bb1700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009830 tx=0x7fd8bc0ec920 comp rx=0 tx=0).stop 2024-04-25T01:20:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.040+0000 7fd8db039740 1 -- 172.21.15.31:0/4157765572 shutdown_connections 2024-04-25T01:20:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.040+0000 7fd8db039740 1 --2- 172.21.15.31:0/4157765572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557581bb8800 0x557581bcf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.040+0000 7fd8db039740 1 --2- 172.21.15.31:0/4157765572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557581bb3690 0x557581bc4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.040+0000 7fd8db039740 1 --2- 172.21.15.31:0/4157765572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557581bb2ca0 0x557581bb1700 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.040+0000 7fd8db039740 1 -- 172.21.15.31:0/4157765572 >> 172.21.15.31:0/4157765572 conn(0x557581bae310 msgr2=0x557581bb0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:20:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.041+0000 7fd8db039740 1 -- 172.21.15.31:0/4157765572 shutdown_connections 2024-04-25T01:20:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.041+0000 7fd8db039740 1 -- 172.21.15.31:0/4157765572 wait complete. 2024-04-25T01:20:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.042+0000 7fd8db039740 1 -- start start 2024-04-25T01:20:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.043+0000 7fd8db039740 1 Processor -- start 2024-04-25T01:20:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.043+0000 7fd8db039740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557581bb2ca0 0x557581bc2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.043+0000 7fd8db039740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557581bb3690 0x557581bc2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:20.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.043+0000 7fd8db039740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557581bb8800 0x557581bc3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:20.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.044+0000 7fd8db039740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d01ad450 con 0x557581bb8800 2024-04-25T01:20:20.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.044+0000 7fd8db039740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d01ae8c0 con 0x557581bb3690 2024-04-25T01:20:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.044+0000 7fd8db039740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d01adb20 con 0x557581bb2ca0 2024-04-25T01:20:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.044+0000 7fd8db039740 10 client.0 _ll_register_callbacks cb 0x7fd8c8020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:20:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.044+0000 7fd8d4e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557581bb3690 0x557581bc2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.044+0000 7fd8d4e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557581bb3690 0x557581bc2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:60048/0 (socket says 172.21.15.31:60048) 2024-04-25T01:20:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.045+0000 7fd8d4e00640 1 -- 172.21.15.31:0/233742190 learned_addr learned my addr 172.21.15.31:0/233742190 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:20:20.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.044+0000 7fd8d5800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557581bb2ca0 0x557581bc2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:20.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.045+0000 7fd8d6200640 1 --2- 172.21.15.31:0/233742190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557581bb8800 0x557581bc3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.047+0000 7fd8d4e00640 1 -- 172.21.15.31:0/233742190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557581bb2ca0 msgr2=0x557581bc2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.047+0000 7fd8d4e00640 1 --2- 172.21.15.31:0/233742190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557581bb2ca0 0x557581bc2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.047+0000 7fd8d4e00640 1 -- 172.21.15.31:0/233742190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557581bb8800 msgr2=0x557581bc3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.047+0000 7fd8d4e00640 1 --2- 172.21.15.31:0/233742190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557581bb8800 0x557581bc3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.047+0000 7fd8d4e00640 1 -- 172.21.15.31:0/233742190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557581bbd820 con 0x557581bb3690 2024-04-25T01:20:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.050+0000 7fd8d5800640 1 --2- 172.21.15.31:0/233742190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557581bb2ca0 0x557581bc2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:20:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.050+0000 7fd8d6200640 1 --2- 172.21.15.31:0/233742190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557581bb8800 0x557581bc3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:20:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.051+0000 7fd8d4e00640 1 --2- 172.21.15.31:0/233742190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557581bb3690 0x557581bc2a40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8007bb0 tx=0x7fd8c8005f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fa807c8028ff6b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:20:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.051+0000 7fd8b7400640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T01:20:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.051+0000 7fd8b7400640 1 -- 172.21.15.31:0/233742190 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8034070 con 0x557581bb3690 2024-04-25T01:20:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.052+0000 7fd8b7400640 1 -- 172.21.15.31:0/233742190 <== mon.2 v2:172.21.15.40:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fd8c802c070 con 0x557581bb3690 2024-04-25T01:20:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.052+0000 7fd8db039740 1 -- 172.21.15.31:0/233742190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8bc0bbeb0 con 0x557581bb3690 2024-04-25T01:20:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.053+0000 7fd8b7400640 1 -- 172.21.15.31:0/233742190 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd8c8010c80 con 0x557581bb3690 2024-04-25T01:20:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.053+0000 7fd8db039740 10 client.25523 fetch_fsmap learned FSMap version 27 2024-04-25T01:20:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.053+0000 7fd8db039740 1 -- 172.21.15.31:0/233742190 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557581c37d30 con 0x557581bb3690 2024-04-25T01:20:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.054+0000 7fd8b7400640 1 -- 172.21.15.31:0/233742190 <== mon.2 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8bc0bbeb0 con 0x557581bb3690 2024-04-25T01:20:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.056+0000 7fd8b7400640 1 -- 172.21.15.31:0/233742190 <== mon.2 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7fd8c8013240 con 0x557581bb3690 2024-04-25T01:20:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.056+0000 7fd8db039740 10 client.25523 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:20:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.056+0000 7fd8db039740 10 client.25523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:20:20.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.056+0000 7fd8db039740 20 client.25523 populate_metadata read hostname 'smithi031' 2024-04-25T01:20:20.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.056+0000 7fd8db039740 1 --2- 172.21.15.31:0/233742190 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x557581c385d0 0x557581c3a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:20.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.056+0000 7fd8db039740 4 client.25523 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:20:20.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.056+0000 7fd8db039740 1 -- 172.21.15.31:0/233742190 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557581bb8530 con 0x557581c385d0 2024-04-25T01:20:20.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.056+0000 7fd8d5800640 1 --2- 172.21.15.31:0/233742190 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x557581c385d0 0x557581c3a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.058+0000 7fd8d5800640 1 --2- 172.21.15.31:0/233742190 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x557581c385d0 0x557581c3a9f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3330a5bc30dfd426 server_cookie=dd87b51efaecabaf in_seq=0 out_seq=0 2024-04-25T01:20:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.058+0000 7fd8b7400640 10 client.25523 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:20:20.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:19 smithi169 ceph-mon[32759]: pgmap v522: 97 pgs: 97 active+clean; 13 GiB data, 71 GiB used, 1002 GiB / 1.0 TiB avail; 51 KiB/s rd, 117 MiB/s wr, 16.90k op/s 2024-04-25T01:20:20.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:19 smithi040 ceph-mon[32518]: pgmap v522: 97 pgs: 97 active+clean; 13 GiB data, 71 GiB used, 1002 GiB / 1.0 TiB avail; 51 KiB/s rd, 117 MiB/s wr, 16.90k op/s 2024-04-25T01:20:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.943+0000 7fd8b7400640 1 -- 172.21.15.31:0/233742190 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557581bb8530 con 0x557581c385d0 2024-04-25T01:20:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.943+0000 7fd8b7400640 10 client.25523 handle_command_reply: tid=0 2024-04-25T01:20:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.943+0000 7fd8db039740 1 client.25523 shutdown 2024-04-25T01:20:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.944+0000 7fd8db039740 1 -- 172.21.15.31:0/233742190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557581bb3690 msgr2=0x557581bc2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.944+0000 7fd8db039740 1 --2- 172.21.15.31:0/233742190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557581bb3690 0x557581bc2a40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8007bb0 tx=0x7fd8c8005f60 comp rx=0 tx=0).stop 2024-04-25T01:20:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.944+0000 7fd8db039740 1 -- 172.21.15.31:0/233742190 shutdown_connections 2024-04-25T01:20:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.944+0000 7fd8db039740 1 --2- 172.21.15.31:0/233742190 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x557581c385d0 0x557581c3a9f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.944+0000 7fd8db039740 1 --2- 172.21.15.31:0/233742190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557581bb8800 0x557581bc3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.944+0000 7fd8db039740 1 --2- 172.21.15.31:0/233742190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557581bb3690 0x557581bc2a40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.944+0000 7fd8db039740 1 --2- 172.21.15.31:0/233742190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557581bb2ca0 0x557581bc2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.945+0000 7fd8db039740 1 -- 172.21.15.31:0/233742190 >> 172.21.15.31:0/233742190 conn(0x557581bae310 msgr2=0x557581bb1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:20:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.945+0000 7fd8b7400640 0 client.25523 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:20:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.945+0000 7fd8db039740 1 -- 172.21.15.31:0/233742190 shutdown_connections 2024-04-25T01:20:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.945+0000 7fd8db039740 1 -- 172.21.15.31:0/233742190 wait complete. 2024-04-25T01:20:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.946+0000 7fd8db039740 20 client.25523 trim_cache size 0 max 16384 2024-04-25T01:20:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:20:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:20:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:20:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:20:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.947+0000 7fd8d8200640 1 -- 172.21.15.31:0/2954794794 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd8b80a2f80 msgr2=0x7fd8b80a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.947+0000 7fd8d8200640 1 --2- 172.21.15.31:0/2954794794 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd8b80a2f80 0x7fd8b80a5440 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8013fd0 tx=0x7fd8c800d900 comp rx=0 tx=0).stop 2024-04-25T01:20:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.947+0000 7fd8d8200640 1 -- 172.21.15.31:0/2954794794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8d0108740 msgr2=0x7fd8d019ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.947+0000 7fd8d8200640 1 --2- 172.21.15.31:0/2954794794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8d0108740 0x7fd8d019ff40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc0042f0 tx=0x7fd8bc004320 comp rx=0 tx=0).stop 2024-04-25T01:20:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.948+0000 7fd8d8200640 1 -- 172.21.15.31:0/2954794794 shutdown_connections 2024-04-25T01:20:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.948+0000 7fd8d8200640 1 --2- 172.21.15.31:0/2954794794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8d010ffc0 0x7fd8d01a74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.948+0000 7fd8d8200640 1 --2- 172.21.15.31:0/2954794794 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd8b80a2f80 0x7fd8b80a5440 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.948+0000 7fd8d8200640 1 --2- 172.21.15.31:0/2954794794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8d01090a0 0x7fd8d01a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.948+0000 7fd8d8200640 1 --2- 172.21.15.31:0/2954794794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8d0108740 0x7fd8d019ff40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.948+0000 7fd8d8200640 1 -- 172.21.15.31:0/2954794794 >> 172.21.15.31:0/2954794794 conn(0x7fd8d006e7b0 msgr2=0x7fd8d010ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:20:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.948+0000 7fd8d8200640 1 -- 172.21.15.31:0/2954794794 shutdown_connections 2024-04-25T01:20:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:20.948+0000 7fd8d8200640 1 -- 172.21.15.31:0/2954794794 wait complete. 2024-04-25T01:20:22.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:21 smithi031 ceph-mon[28053]: pgmap v523: 97 pgs: 97 active+clean; 13 GiB data, 70 GiB used, 1002 GiB / 1.0 TiB avail; 61 KiB/s rd, 128 MiB/s wr, 19.01k op/s 2024-04-25T01:20:22.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:21 smithi169 ceph-mon[32759]: pgmap v523: 97 pgs: 97 active+clean; 13 GiB data, 70 GiB used, 1002 GiB / 1.0 TiB avail; 61 KiB/s rd, 128 MiB/s wr, 19.01k op/s 2024-04-25T01:20:22.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:21 smithi040 ceph-mon[32518]: pgmap v523: 97 pgs: 97 active+clean; 13 GiB data, 70 GiB used, 1002 GiB / 1.0 TiB avail; 61 KiB/s rd, 128 MiB/s wr, 19.01k op/s 2024-04-25T01:20:23.992 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:23 smithi169 ceph-mon[32759]: pgmap v524: 97 pgs: 97 active+clean; 13 GiB data, 71 GiB used, 1001 GiB / 1.0 TiB avail; 49 KiB/s rd, 130 MiB/s wr, 19.85k op/s 2024-04-25T01:20:24.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:23 smithi031 ceph-mon[28053]: pgmap v524: 97 pgs: 97 active+clean; 13 GiB data, 71 GiB used, 1001 GiB / 1.0 TiB avail; 49 KiB/s rd, 130 MiB/s wr, 19.85k op/s 2024-04-25T01:20:24.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:23 smithi040 ceph-mon[32518]: pgmap v524: 97 pgs: 97 active+clean; 13 GiB data, 71 GiB used, 1001 GiB / 1.0 TiB avail; 49 KiB/s rd, 130 MiB/s wr, 19.85k op/s 2024-04-25T01:20:26.003 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:25 smithi031 ceph-mon[28053]: pgmap v525: 97 pgs: 97 active+clean; 13 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 48 KiB/s rd, 111 MiB/s wr, 17.62k op/s 2024-04-25T01:20:26.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:25 smithi169 ceph-mon[32759]: pgmap v525: 97 pgs: 97 active+clean; 13 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 48 KiB/s rd, 111 MiB/s wr, 17.62k op/s 2024-04-25T01:20:26.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:25 smithi040 ceph-mon[32518]: pgmap v525: 97 pgs: 97 active+clean; 13 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 48 KiB/s rd, 111 MiB/s wr, 17.62k op/s 2024-04-25T01:20:28.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:27 smithi169 ceph-mon[32759]: pgmap v526: 97 pgs: 97 active+clean; 13 GiB data, 73 GiB used, 1000 GiB / 1.0 TiB avail; 65 KiB/s rd, 138 MiB/s wr, 21.70k op/s 2024-04-25T01:20:28.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:27 smithi040 ceph-mon[32518]: pgmap v526: 97 pgs: 97 active+clean; 13 GiB data, 73 GiB used, 1000 GiB / 1.0 TiB avail; 65 KiB/s rd, 138 MiB/s wr, 21.70k op/s 2024-04-25T01:20:28.251 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:27 smithi031 ceph-mon[28053]: pgmap v526: 97 pgs: 97 active+clean; 13 GiB data, 73 GiB used, 1000 GiB / 1.0 TiB avail; 65 KiB/s rd, 138 MiB/s wr, 21.70k op/s 2024-04-25T01:20:30.059 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:29 smithi031 ceph-mon[28053]: pgmap v527: 97 pgs: 97 active+clean; 13 GiB data, 73 GiB used, 999 GiB / 1.0 TiB avail; 51 KiB/s rd, 119 MiB/s wr, 18.37k op/s 2024-04-25T01:20:30.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:29 smithi169 ceph-mon[32759]: pgmap v527: 97 pgs: 97 active+clean; 13 GiB data, 73 GiB used, 999 GiB / 1.0 TiB avail; 51 KiB/s rd, 119 MiB/s wr, 18.37k op/s 2024-04-25T01:20:30.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:29 smithi040 ceph-mon[32518]: pgmap v527: 97 pgs: 97 active+clean; 13 GiB data, 73 GiB used, 999 GiB / 1.0 TiB avail; 51 KiB/s rd, 119 MiB/s wr, 18.37k op/s 2024-04-25T01:20:32.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:31 smithi169 ceph-mon[32759]: pgmap v528: 97 pgs: 97 active+clean; 13 GiB data, 73 GiB used, 999 GiB / 1.0 TiB avail; 70 KiB/s rd, 128 MiB/s wr, 19.44k op/s 2024-04-25T01:20:32.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:31 smithi040 ceph-mon[32518]: pgmap v528: 97 pgs: 97 active+clean; 13 GiB data, 73 GiB used, 999 GiB / 1.0 TiB avail; 70 KiB/s rd, 128 MiB/s wr, 19.44k op/s 2024-04-25T01:20:32.253 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:31 smithi031 ceph-mon[28053]: pgmap v528: 97 pgs: 97 active+clean; 13 GiB data, 73 GiB used, 999 GiB / 1.0 TiB avail; 70 KiB/s rd, 128 MiB/s wr, 19.44k op/s 2024-04-25T01:20:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.183+0000 7f04f6a00640 1 -- 172.21.15.31:0/3658872516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04f0107400 msgr2=0x7f04f01098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.183+0000 7f04f6a00640 1 --2- 172.21.15.31:0/3658872516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04f0107400 0x7f04f01098c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009900 tx=0x7f04d802a440 comp rx=0 tx=0).stop 2024-04-25T01:20:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.185+0000 7f04f6a00640 1 -- 172.21.15.31:0/3658872516 shutdown_connections 2024-04-25T01:20:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.185+0000 7f04f6a00640 1 --2- 172.21.15.31:0/3658872516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04f0107400 0x7f04f01098c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.185+0000 7f04f6a00640 1 --2- 172.21.15.31:0/3658872516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04f00ffe70 0x7f04f0106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.185+0000 7f04f6a00640 1 --2- 172.21.15.31:0/3658872516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f00ff510 0x7f04f00ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.185+0000 7f04f6a00640 1 -- 172.21.15.31:0/3658872516 >> 172.21.15.31:0/3658872516 conn(0x7f04f00faf60 msgr2=0x7f04f00fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:20:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.185+0000 7f04f6a00640 1 -- 172.21.15.31:0/3658872516 shutdown_connections 2024-04-25T01:20:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.185+0000 7f04f6a00640 1 -- 172.21.15.31:0/3658872516 wait complete. 2024-04-25T01:20:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.186+0000 7f04f6a00640 1 Processor -- start 2024-04-25T01:20:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.187+0000 7f04f6a00640 1 -- start start 2024-04-25T01:20:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.187+0000 7f04f6a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f00ff510 0x7f04f01027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.187+0000 7f04f6a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04f00ffe70 0x7f04f0100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.188+0000 7f04f6a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04f0107400 0x7f04f0101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.188+0000 7f04ef400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04f00ffe70 0x7f04f0100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.188+0000 7f04ef400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04f00ffe70 0x7f04f0100e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:44658/0 (socket says 172.21.15.31:44658) 2024-04-25T01:20:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.188+0000 7f04ef400640 1 -- 172.21.15.31:0/2647947652 learned_addr learned my addr 172.21.15.31:0/2647947652 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:20:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.188+0000 7f04f4a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04f0107400 0x7f04f0101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.188+0000 7f04f6a00640 1 -- 172.21.15.31:0/2647947652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f04f010d440 con 0x7f04f00ff510 2024-04-25T01:20:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.188+0000 7f04f6a00640 1 -- 172.21.15.31:0/2647947652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f04f010d2c0 con 0x7f04f0107400 2024-04-25T01:20:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.188+0000 7f04f6a00640 1 -- 172.21.15.31:0/2647947652 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f04f010d5c0 con 0x7f04f00ffe70 2024-04-25T01:20:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.189+0000 7f04efe00640 1 --2- 172.21.15.31:0/2647947652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f00ff510 0x7f04f01027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.189+0000 7f04ef400640 1 -- 172.21.15.31:0/2647947652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04f0107400 msgr2=0x7f04f0101360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.189+0000 7f04ef400640 1 --2- 172.21.15.31:0/2647947652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04f0107400 0x7f04f0101360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.189+0000 7f04ef400640 1 -- 172.21.15.31:0/2647947652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f00ff510 msgr2=0x7f04f01027e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.189+0000 7f04ef400640 1 --2- 172.21.15.31:0/2647947652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f00ff510 0x7f04f01027e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.189+0000 7f04ef400640 1 -- 172.21.15.31:0/2647947652 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f01a7ed0 con 0x7f04f00ffe70 2024-04-25T01:20:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.189+0000 7f04efe00640 1 --2- 172.21.15.31:0/2647947652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f00ff510 0x7f04f01027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:20:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.190+0000 7f04ef400640 1 --2- 172.21.15.31:0/2647947652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04f00ffe70 0x7f04f0100e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f04e000e9c0 tx=0x7f04e000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:20:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.190+0000 7f04ecc00640 1 -- 172.21.15.31:0/2647947652 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e000cd80 con 0x7f04f00ffe70 2024-04-25T01:20:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.192+0000 7f04f6a00640 1 -- 172.21.15.31:0/2647947652 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f01a81c0 con 0x7f04f00ffe70 2024-04-25T01:20:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.192+0000 7f04f6a00640 1 -- 172.21.15.31:0/2647947652 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f01a8730 con 0x7f04f00ffe70 2024-04-25T01:20:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.192+0000 7f04ecc00640 1 -- 172.21.15.31:0/2647947652 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04e00035f0 con 0x7f04f00ffe70 2024-04-25T01:20:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.193+0000 7f04ecc00640 1 -- 172.21.15.31:0/2647947652 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0010690 con 0x7f04f00ffe70 2024-04-25T01:20:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.205+0000 7f04ecc00640 1 -- 172.21.15.31:0/2647947652 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f04e00037a0 con 0x7f04f00ffe70 2024-04-25T01:20:34.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:33 smithi169 ceph-mon[32759]: pgmap v529: 97 pgs: 97 active+clean; 13 GiB data, 74 GiB used, 999 GiB / 1.0 TiB avail; 81 KiB/s rd, 133 MiB/s wr, 19.72k op/s 2024-04-25T01:20:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.205+0000 7f04f97b3740 1 Processor -- start 2024-04-25T01:20:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.208+0000 7f04f97b3740 1 -- start start 2024-04-25T01:20:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.209+0000 7f04ecc00640 1 --2- 172.21.15.31:0/2647947652 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f04d40a2ba0 0x7f04d40a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.209+0000 7f04ecc00640 1 -- 172.21.15.31:0/2647947652 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f04e00cf410 con 0x7f04f00ffe70 2024-04-25T01:20:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.210+0000 7f04f97b3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cc870aca0 0x555cc870b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.210+0000 7f04f97b3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555cc870b690 0x555cc8710190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.210+0000 7f04f97b3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cc8710800 0x555cc8712c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.210+0000 7f04f97b3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x555cc866dab0 con 0x555cc8710800 2024-04-25T01:20:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.210+0000 7f04f97b3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x555cc8616640 con 0x555cc870b690 2024-04-25T01:20:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.210+0000 7f04f97b3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555cc8669a70 con 0x555cc870aca0 2024-04-25T01:20:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.210+0000 7f04efe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cc870aca0 0x555cc870b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.210+0000 7f04efe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cc870aca0 0x555cc870b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:44668/0 (socket says 172.21.15.31:44668) 2024-04-25T01:20:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.210+0000 7f04efe00640 1 -- 172.21.15.31:0/1395176225 learned_addr learned my addr 172.21.15.31:0/1395176225 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:20:34.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:33 smithi040 ceph-mon[32518]: pgmap v529: 97 pgs: 97 active+clean; 13 GiB data, 74 GiB used, 999 GiB / 1.0 TiB avail; 81 KiB/s rd, 133 MiB/s wr, 19.72k op/s 2024-04-25T01:20:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.211+0000 7f04efe00640 1 -- 172.21.15.31:0/1395176225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555cc870b690 msgr2=0x555cc8710190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:20:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.211+0000 7f04efe00640 1 --2- 172.21.15.31:0/1395176225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555cc870b690 0x555cc8710190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.211+0000 7f04efe00640 1 -- 172.21.15.31:0/1395176225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cc8710800 msgr2=0x555cc8712c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.211+0000 7f04efe00640 1 --2- 172.21.15.31:0/1395176225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cc8710800 0x555cc8712c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.211+0000 7f04efe00640 1 -- 172.21.15.31:0/1395176225 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cc8382210 con 0x555cc870aca0 2024-04-25T01:20:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.211+0000 7f04f4a00640 1 --2- 172.21.15.31:0/2647947652 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f04d40a2ba0 0x7f04d40a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.213+0000 7f04efe00640 1 --2- 172.21.15.31:0/1395176225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cc870aca0 0x555cc870b0c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f04e80104a0 tx=0x7f04e8010960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a217eecd1a36af8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:20:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.213+0000 7f04e5200640 1 -- 172.21.15.31:0/1395176225 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8017070 con 0x555cc870aca0 2024-04-25T01:20:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.213+0000 7f04e5200640 1 -- 172.21.15.31:0/1395176225 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04e8011be0 con 0x555cc870aca0 2024-04-25T01:20:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.213+0000 7f04e5200640 1 -- 172.21.15.31:0/1395176225 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8017070 con 0x555cc870aca0 2024-04-25T01:20:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.215+0000 7f04f97b3740 1 -- 172.21.15.31:0/1395176225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cc870aca0 msgr2=0x555cc870b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.215+0000 7f04f97b3740 1 --2- 172.21.15.31:0/1395176225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cc870aca0 0x555cc870b0c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f04e80104a0 tx=0x7f04e8010960 comp rx=0 tx=0).stop 2024-04-25T01:20:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.217+0000 7f04f4a00640 1 --2- 172.21.15.31:0/2647947652 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f04d40a2ba0 0x7f04d40a5060 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f04d8004b40 tx=0x7f04d8004b70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:20:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.217+0000 7f04f97b3740 1 -- 172.21.15.31:0/1395176225 shutdown_connections 2024-04-25T01:20:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.217+0000 7f04f97b3740 1 --2- 172.21.15.31:0/1395176225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cc8710800 0x555cc8712c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.217+0000 7f04f97b3740 1 --2- 172.21.15.31:0/1395176225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555cc870b690 0x555cc8710190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.217+0000 7f04f97b3740 1 --2- 172.21.15.31:0/1395176225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cc870aca0 0x555cc870b0c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.217+0000 7f04f97b3740 1 -- 172.21.15.31:0/1395176225 >> 172.21.15.31:0/1395176225 conn(0x555cc8706310 msgr2=0x555cc8706710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:20:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.219+0000 7f04f97b3740 1 -- 172.21.15.31:0/1395176225 shutdown_connections 2024-04-25T01:20:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.219+0000 7f04f97b3740 1 -- 172.21.15.31:0/1395176225 wait complete. 2024-04-25T01:20:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.219+0000 7f04f97b3740 1 -- start start 2024-04-25T01:20:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.220+0000 7f04f97b3740 1 Processor -- start 2024-04-25T01:20:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.225+0000 7f04f97b3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555cc870aca0 0x555cc8714f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.225+0000 7f04f97b3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cc870b690 0x555cc87154b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.225+0000 7f04efe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555cc870aca0 0x555cc8714f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.225+0000 7f04f97b3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cc8710800 0x555cc871c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.225+0000 7f04f97b3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x555cc87137b0 con 0x555cc870b690 2024-04-25T01:20:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.225+0000 7f04efe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555cc870aca0 0x555cc8714f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:43236/0 (socket says 172.21.15.31:43236) 2024-04-25T01:20:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.225+0000 7f04efe00640 1 -- 172.21.15.31:0/854861534 learned_addr learned my addr 172.21.15.31:0/854861534 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:20:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.225+0000 7f04f97b3740 1 -- 172.21.15.31:0/854861534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x555cc8629990 con 0x555cc870aca0 2024-04-25T01:20:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.225+0000 7f04f97b3740 1 -- 172.21.15.31:0/854861534 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555cc8713930 con 0x555cc8710800 2024-04-25T01:20:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.226+0000 7f04ef400640 1 --2- 172.21.15.31:0/854861534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cc870b690 0x555cc87154b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.226+0000 7f04f97b3740 10 client.0 _ll_register_callbacks cb 0x555cc862b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:20:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.226+0000 7f04f4a00640 1 --2- 172.21.15.31:0/854861534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cc8710800 0x555cc871c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.227+0000 7f04ef400640 1 -- 172.21.15.31:0/854861534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cc8710800 msgr2=0x555cc871c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.227+0000 7f04ef400640 1 --2- 172.21.15.31:0/854861534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cc8710800 0x555cc871c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.227+0000 7f04ef400640 1 -- 172.21.15.31:0/854861534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555cc870aca0 msgr2=0x555cc8714f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.227+0000 7f04ef400640 1 --2- 172.21.15.31:0/854861534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555cc870aca0 0x555cc8714f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.227+0000 7f04ef400640 1 -- 172.21.15.31:0/854861534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cc844e620 con 0x555cc870b690 2024-04-25T01:20:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.228+0000 7f04ef400640 1 --2- 172.21.15.31:0/854861534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cc870b690 0x555cc87154b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x555cc870c100 tx=0x7f04e009c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e6825ef55a0b97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:20:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.228+0000 7f04dd600640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T01:20:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.229+0000 7f04dd600640 1 -- 172.21.15.31:0/854861534 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e00b4e60 con 0x555cc870b690 2024-04-25T01:20:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.229+0000 7f04f97b3740 1 -- 172.21.15.31:0/854861534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04e8011be0 con 0x555cc870b690 2024-04-25T01:20:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.229+0000 7f04dd600640 1 -- 172.21.15.31:0/854861534 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f04e009b0b0 con 0x555cc870b690 2024-04-25T01:20:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.230+0000 7f04dd600640 1 -- 172.21.15.31:0/854861534 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f04e0003c90 con 0x555cc870b690 2024-04-25T01:20:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.230+0000 7f04f97b3740 10 client.15324 fetch_fsmap learned FSMap version 27 2024-04-25T01:20:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.230+0000 7f04f97b3740 1 -- 172.21.15.31:0/854861534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555cc871f590 con 0x555cc870b690 2024-04-25T01:20:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.230+0000 7f04dd600640 1 -- 172.21.15.31:0/854861534 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04e8011be0 con 0x555cc870b690 2024-04-25T01:20:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.231+0000 7f04dd600640 1 -- 172.21.15.31:0/854861534 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7f04e009ab20 con 0x555cc870b690 2024-04-25T01:20:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.231+0000 7f04f97b3740 10 client.15324 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:20:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.231+0000 7f04f97b3740 10 client.15324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:20:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.231+0000 7f04f97b3740 20 client.15324 populate_metadata read hostname 'smithi031' 2024-04-25T01:20:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.231+0000 7f04f97b3740 1 --2- 172.21.15.31:0/854861534 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x555cc8721110 0x555cc8723530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.231+0000 7f04f97b3740 4 client.15324 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:20:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.231+0000 7f04f97b3740 1 -- 172.21.15.31:0/854861534 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cc867aff0 con 0x555cc8721110 2024-04-25T01:20:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.232+0000 7f04efe00640 1 --2- 172.21.15.31:0/854861534 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x555cc8721110 0x555cc8723530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.233+0000 7f04efe00640 1 --2- 172.21.15.31:0/854861534 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x555cc8721110 0x555cc8723530 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4b796fa0f56486a server_cookie=59dc6688422edfbb in_seq=0 out_seq=0 2024-04-25T01:20:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.233+0000 7f04dd600640 10 client.15324 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:20:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.234+0000 7f04dd600640 1 -- 172.21.15.31:0/854861534 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555cc867aff0 con 0x555cc8721110 2024-04-25T01:20:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.234+0000 7f04dd600640 10 client.15324 handle_command_reply: tid=0 2024-04-25T01:20:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.234+0000 7f04f97b3740 1 client.15324 shutdown 2024-04-25T01:20:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.236+0000 7f04f97b3740 1 -- 172.21.15.31:0/854861534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cc870b690 msgr2=0x555cc87154b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.236+0000 7f04f97b3740 1 --2- 172.21.15.31:0/854861534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cc870b690 0x555cc87154b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x555cc870c100 tx=0x7f04e009c4a0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.237+0000 7f04f97b3740 1 -- 172.21.15.31:0/854861534 shutdown_connections 2024-04-25T01:20:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.237+0000 7f04f97b3740 1 --2- 172.21.15.31:0/854861534 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x555cc8721110 0x555cc8723530 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.237+0000 7f04f97b3740 1 --2- 172.21.15.31:0/854861534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cc8710800 0x555cc871c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.237+0000 7f04f97b3740 1 --2- 172.21.15.31:0/854861534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cc870b690 0x555cc87154b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.237+0000 7f04f97b3740 1 --2- 172.21.15.31:0/854861534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555cc870aca0 0x555cc8714f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.237+0000 7f04f97b3740 1 -- 172.21.15.31:0/854861534 >> 172.21.15.31:0/854861534 conn(0x555cc8706310 msgr2=0x555cc8705480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:20:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.237+0000 7f04dd600640 0 client.15324 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:20:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.237+0000 7f04f97b3740 1 -- 172.21.15.31:0/854861534 shutdown_connections 2024-04-25T01:20:34.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.238+0000 7f04f97b3740 1 -- 172.21.15.31:0/854861534 wait complete. 2024-04-25T01:20:34.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.238+0000 7f04f97b3740 20 client.15324 trim_cache size 0 max 16384 2024-04-25T01:20:34.252 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:33 smithi031 ceph-mon[28053]: pgmap v529: 97 pgs: 97 active+clean; 13 GiB data, 74 GiB used, 999 GiB / 1.0 TiB avail; 81 KiB/s rd, 133 MiB/s wr, 19.72k op/s 2024-04-25T01:20:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.278+0000 7f04f97b3740 1 Processor -- start 2024-04-25T01:20:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.278+0000 7f04f97b3740 1 -- start start 2024-04-25T01:20:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.279+0000 7f04f97b3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cc870aca0 0x555cc8713d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:34.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.279+0000 7f04f97b3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555cc870b690 0x555cc871c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:34.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.280+0000 7f04f97b3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cc8710800 0x555cc87142a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:34.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.280+0000 7f04f97b3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc010c50 con 0x555cc870aca0 2024-04-25T01:20:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.280+0000 7f04f97b3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc00e4c0 con 0x555cc870b690 2024-04-25T01:20:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.280+0000 7f04f97b3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc00c310 con 0x555cc8710800 2024-04-25T01:20:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.280+0000 7f04f4a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cc8710800 0x555cc87142a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.281+0000 7f04efe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cc870aca0 0x555cc8713d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.281+0000 7f04efe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cc870aca0 0x555cc8713d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55446/0 (socket says 172.21.15.31:55446) 2024-04-25T01:20:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.281+0000 7f04efe00640 1 -- 172.21.15.31:0/15765033 learned_addr learned my addr 172.21.15.31:0/15765033 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:20:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.281+0000 7f04ef400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555cc870b690 0x555cc871c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.282+0000 7f04ef400640 1 -- 172.21.15.31:0/15765033 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cc8710800 msgr2=0x555cc87142a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.282+0000 7f04ef400640 1 --2- 172.21.15.31:0/15765033 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cc8710800 0x555cc87142a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.282+0000 7f04ef400640 1 -- 172.21.15.31:0/15765033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cc870aca0 msgr2=0x555cc8713d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.282+0000 7f04ef400640 1 --2- 172.21.15.31:0/15765033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cc870aca0 0x555cc8713d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.282+0000 7f04ef400640 1 -- 172.21.15.31:0/15765033 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cc8716000 con 0x555cc870b690 2024-04-25T01:20:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.283+0000 7f04ef400640 1 --2- 172.21.15.31:0/15765033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555cc870b690 0x555cc871c4b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f04e009bbf0 tx=0x7f04e0009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a308d484a54faea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:20:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.284+0000 7f04dfe00640 1 -- 172.21.15.31:0/15765033 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e00b4e60 con 0x555cc870b690 2024-04-25T01:20:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.284+0000 7f04dfe00640 1 -- 172.21.15.31:0/15765033 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04e00b1070 con 0x555cc870b690 2024-04-25T01:20:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.284+0000 7f04dfe00640 1 -- 172.21.15.31:0/15765033 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e007f080 con 0x555cc870b690 2024-04-25T01:20:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.285+0000 7f04f97b3740 1 -- 172.21.15.31:0/15765033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555cc870b690 msgr2=0x555cc871c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.285+0000 7f04f97b3740 1 --2- 172.21.15.31:0/15765033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555cc870b690 0x555cc871c4b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f04e009bbf0 tx=0x7f04e0009740 comp rx=0 tx=0).stop 2024-04-25T01:20:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.286+0000 7f04f97b3740 1 -- 172.21.15.31:0/15765033 shutdown_connections 2024-04-25T01:20:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.286+0000 7f04f97b3740 1 --2- 172.21.15.31:0/15765033 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cc8710800 0x555cc87142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.286+0000 7f04f97b3740 1 --2- 172.21.15.31:0/15765033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555cc870b690 0x555cc871c4b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.286+0000 7f04f97b3740 1 --2- 172.21.15.31:0/15765033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cc870aca0 0x555cc8713d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.286+0000 7f04f97b3740 1 -- 172.21.15.31:0/15765033 >> 172.21.15.31:0/15765033 conn(0x555cc8706310 msgr2=0x555cc8709880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:20:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.286+0000 7f04f97b3740 1 -- 172.21.15.31:0/15765033 shutdown_connections 2024-04-25T01:20:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.286+0000 7f04f97b3740 1 -- 172.21.15.31:0/15765033 wait complete. 2024-04-25T01:20:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.287+0000 7f04f97b3740 1 -- start start 2024-04-25T01:20:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.287+0000 7f04f97b3740 1 Processor -- start 2024-04-25T01:20:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.289+0000 7f04f97b3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cc870aca0 0x555cc8719bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.289+0000 7f04f97b3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cc870b690 0x555cc871a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.290+0000 7f04f97b3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555cc8710800 0x555cc871b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.290+0000 7f04f97b3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc00c980 con 0x555cc870aca0 2024-04-25T01:20:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.290+0000 7f04f97b3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc00ddf0 con 0x555cc8710800 2024-04-25T01:20:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.290+0000 7f04f97b3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc00d050 con 0x555cc870b690 2024-04-25T01:20:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.290+0000 7f04f97b3740 10 client.0 _ll_register_callbacks cb 0x7f04e0097820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:20:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.290+0000 7f04ef400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cc870b690 0x555cc871a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.290+0000 7f04f4a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555cc8710800 0x555cc871b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.290+0000 7f04f4a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555cc8710800 0x555cc871b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:43248/0 (socket says 172.21.15.31:43248) 2024-04-25T01:20:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.290+0000 7f04f4a00640 1 -- 172.21.15.31:0/3633264508 learned_addr learned my addr 172.21.15.31:0/3633264508 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:20:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.290+0000 7f04efe00640 1 --2- 172.21.15.31:0/3633264508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cc870aca0 0x555cc8719bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.292+0000 7f04ef400640 1 -- 172.21.15.31:0/3633264508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555cc8710800 msgr2=0x555cc871b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.292+0000 7f04ef400640 1 --2- 172.21.15.31:0/3633264508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555cc8710800 0x555cc871b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.292+0000 7f04ef400640 1 -- 172.21.15.31:0/3633264508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cc870aca0 msgr2=0x555cc8719bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.292+0000 7f04ef400640 1 --2- 172.21.15.31:0/3633264508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cc870aca0 0x555cc8719bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.292+0000 7f04ef400640 1 -- 172.21.15.31:0/3633264508 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cc871b6a0 con 0x555cc870b690 2024-04-25T01:20:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.292+0000 7f04dd600640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:20:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.292+0000 7f04ef400640 1 --2- 172.21.15.31:0/3633264508 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cc870b690 0x555cc871a110 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x555cc87138e0 tx=0x7f04e007af30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5247d2b209955873 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:20:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.292+0000 7f04dd600640 1 -- 172.21.15.31:0/3633264508 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e00b4e60 con 0x555cc870b690 2024-04-25T01:20:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.293+0000 7f04dd600640 1 -- 172.21.15.31:0/3633264508 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f04e0003a90 con 0x555cc870b690 2024-04-25T01:20:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.293+0000 7f04f97b3740 1 -- 172.21.15.31:0/3633264508 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04e00b1070 con 0x555cc870b690 2024-04-25T01:20:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.293+0000 7f04dd600640 1 -- 172.21.15.31:0/3633264508 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f04e00b4e60 con 0x555cc870b690 2024-04-25T01:20:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.293+0000 7f04f97b3740 10 client.25564 fetch_fsmap learned FSMap version 27 2024-04-25T01:20:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.293+0000 7f04f97b3740 1 -- 172.21.15.31:0/3633264508 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555cc87876b0 con 0x555cc870b690 2024-04-25T01:20:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.294+0000 7f04dd600640 1 -- 172.21.15.31:0/3633264508 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04e009b2c0 con 0x555cc870b690 2024-04-25T01:20:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.295+0000 7f04dd600640 1 -- 172.21.15.31:0/3633264508 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7f04e00a6bd0 con 0x555cc870b690 2024-04-25T01:20:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.295+0000 7f04f97b3740 10 client.25564 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:20:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.295+0000 7f04f97b3740 10 client.25564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:20:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.295+0000 7f04f97b3740 20 client.25564 populate_metadata read hostname 'smithi031' 2024-04-25T01:20:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.295+0000 7f04f97b3740 1 --2- 172.21.15.31:0/3633264508 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x555cc8787b90 0x555cc8789ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.295+0000 7f04f97b3740 4 client.25564 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:20:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.295+0000 7f04f97b3740 1 -- 172.21.15.31:0/3633264508 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f04e00b1070 con 0x555cc8787b90 2024-04-25T01:20:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.296+0000 7f04efe00640 1 --2- 172.21.15.31:0/3633264508 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x555cc8787b90 0x555cc8789ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.298+0000 7f04efe00640 1 --2- 172.21.15.31:0/3633264508 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x555cc8787b90 0x555cc8789ff0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c073d998e9f2256 server_cookie=899b917de26de11e in_seq=0 out_seq=0 2024-04-25T01:20:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.298+0000 7f04dd600640 10 client.25564 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:20:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.444+0000 7f04dd600640 1 -- 172.21.15.31:0/3633264508 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f04e00b1070 con 0x555cc8787b90 2024-04-25T01:20:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.444+0000 7f04dd600640 10 client.25564 handle_command_reply: tid=0 2024-04-25T01:20:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.445+0000 7f04f97b3740 1 client.25564 shutdown 2024-04-25T01:20:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.446+0000 7f04f97b3740 1 -- 172.21.15.31:0/3633264508 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cc870b690 msgr2=0x555cc871a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.446+0000 7f04f97b3740 1 --2- 172.21.15.31:0/3633264508 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cc870b690 0x555cc871a110 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x555cc87138e0 tx=0x7f04e007af30 comp rx=0 tx=0).stop 2024-04-25T01:20:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.448+0000 7f04f97b3740 1 -- 172.21.15.31:0/3633264508 shutdown_connections 2024-04-25T01:20:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.448+0000 7f04f97b3740 1 --2- 172.21.15.31:0/3633264508 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x555cc8787b90 0x555cc8789ff0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.448+0000 7f04dd600640 0 client.25564 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:20:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.448+0000 7f04f97b3740 1 --2- 172.21.15.31:0/3633264508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555cc8710800 0x555cc871b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.448+0000 7f04f97b3740 1 --2- 172.21.15.31:0/3633264508 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555cc870b690 0x555cc871a110 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.448+0000 7f04f97b3740 1 --2- 172.21.15.31:0/3633264508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cc870aca0 0x555cc8719bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.448+0000 7f04f97b3740 1 -- 172.21.15.31:0/3633264508 >> 172.21.15.31:0/3633264508 conn(0x555cc8706310 msgr2=0x555cc8709880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:20:34.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.449+0000 7f04f97b3740 1 -- 172.21.15.31:0/3633264508 shutdown_connections 2024-04-25T01:20:34.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.449+0000 7f04f97b3740 1 -- 172.21.15.31:0/3633264508 wait complete. 2024-04-25T01:20:34.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.450+0000 7f04f97b3740 20 client.25564 trim_cache size 0 max 16384 2024-04-25T01:20:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:20:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:20:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:20:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:20:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.451+0000 7f04f6a00640 1 -- 172.21.15.31:0/2647947652 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f04d40a2ba0 msgr2=0x7f04d40a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:34.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.451+0000 7f04f6a00640 1 --2- 172.21.15.31:0/2647947652 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f04d40a2ba0 0x7f04d40a5060 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f04d8004b40 tx=0x7f04d8004b70 comp rx=0 tx=0).stop 2024-04-25T01:20:34.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.451+0000 7f04f6a00640 1 -- 172.21.15.31:0/2647947652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04f00ffe70 msgr2=0x7f04f0100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:34.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.451+0000 7f04f6a00640 1 --2- 172.21.15.31:0/2647947652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04f00ffe70 0x7f04f0100e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f04e000e9c0 tx=0x7f04e000ee80 comp rx=0 tx=0).stop 2024-04-25T01:20:34.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.451+0000 7f04f6a00640 1 -- 172.21.15.31:0/2647947652 shutdown_connections 2024-04-25T01:20:34.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.451+0000 7f04f6a00640 1 --2- 172.21.15.31:0/2647947652 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f04d40a2ba0 0x7f04d40a5060 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.451+0000 7f04f6a00640 1 --2- 172.21.15.31:0/2647947652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04f0107400 0x7f04f0101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.451+0000 7f04f6a00640 1 --2- 172.21.15.31:0/2647947652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04f00ffe70 0x7f04f0100e20 secure :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f04e000e9c0 tx=0x7f04e000ee80 comp rx=0 tx=0).stop 2024-04-25T01:20:34.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.451+0000 7f04f6a00640 1 --2- 172.21.15.31:0/2647947652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f00ff510 0x7f04f01027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:34.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.451+0000 7f04f6a00640 1 -- 172.21.15.31:0/2647947652 >> 172.21.15.31:0/2647947652 conn(0x7f04f00faf60 msgr2=0x7f04f0102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:20:34.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.453+0000 7f04f6a00640 1 -- 172.21.15.31:0/2647947652 shutdown_connections 2024-04-25T01:20:34.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:34.453+0000 7f04f6a00640 1 -- 172.21.15.31:0/2647947652 wait complete. 2024-04-25T01:20:36.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:35 smithi169 ceph-mon[32759]: pgmap v530: 97 pgs: 97 active+clean; 13 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 77 KiB/s rd, 120 MiB/s wr, 17.22k op/s 2024-04-25T01:20:36.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:35 smithi040 ceph-mon[32518]: pgmap v530: 97 pgs: 97 active+clean; 13 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 77 KiB/s rd, 120 MiB/s wr, 17.22k op/s 2024-04-25T01:20:36.251 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:35 smithi031 ceph-mon[28053]: pgmap v530: 97 pgs: 97 active+clean; 13 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 77 KiB/s rd, 120 MiB/s wr, 17.22k op/s 2024-04-25T01:20:38.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:37 smithi169 ceph-mon[32759]: pgmap v531: 97 pgs: 97 active+clean; 13 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 112 KiB/s rd, 144 MiB/s wr, 20.12k op/s 2024-04-25T01:20:38.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:37 smithi040 ceph-mon[32518]: pgmap v531: 97 pgs: 97 active+clean; 13 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 112 KiB/s rd, 144 MiB/s wr, 20.12k op/s 2024-04-25T01:20:38.252 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:37 smithi031 ceph-mon[28053]: pgmap v531: 97 pgs: 97 active+clean; 13 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 112 KiB/s rd, 144 MiB/s wr, 20.12k op/s 2024-04-25T01:20:40.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:39 smithi169 ceph-mon[32759]: pgmap v532: 97 pgs: 97 active+clean; 13 GiB data, 77 GiB used, 996 GiB / 1.0 TiB avail; 115 KiB/s rd, 121 MiB/s wr, 16.61k op/s 2024-04-25T01:20:40.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:39 smithi040 ceph-mon[32518]: pgmap v532: 97 pgs: 97 active+clean; 13 GiB data, 77 GiB used, 996 GiB / 1.0 TiB avail; 115 KiB/s rd, 121 MiB/s wr, 16.61k op/s 2024-04-25T01:20:40.251 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:39 smithi031 ceph-mon[28053]: pgmap v532: 97 pgs: 97 active+clean; 13 GiB data, 77 GiB used, 996 GiB / 1.0 TiB avail; 115 KiB/s rd, 121 MiB/s wr, 16.61k op/s 2024-04-25T01:20:42.440 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:42 smithi169 ceph-mon[32759]: pgmap v533: 97 pgs: 97 active+clean; 13 GiB data, 77 GiB used, 995 GiB / 1.0 TiB avail; 159 KiB/s rd, 130 MiB/s wr, 17.68k op/s 2024-04-25T01:20:42.443 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:42 smithi040 ceph-mon[32518]: pgmap v533: 97 pgs: 97 active+clean; 13 GiB data, 77 GiB used, 995 GiB / 1.0 TiB avail; 159 KiB/s rd, 130 MiB/s wr, 17.68k op/s 2024-04-25T01:20:42.502 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:42 smithi031 ceph-mon[28053]: pgmap v533: 97 pgs: 97 active+clean; 13 GiB data, 77 GiB used, 995 GiB / 1.0 TiB avail; 159 KiB/s rd, 130 MiB/s wr, 17.68k op/s 2024-04-25T01:20:44.440 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:44 smithi169 ceph-mon[32759]: pgmap v534: 97 pgs: 97 active+clean; 13 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 149 KiB/s rd, 132 MiB/s wr, 17.58k op/s 2024-04-25T01:20:44.443 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:44 smithi040 ceph-mon[32518]: pgmap v534: 97 pgs: 97 active+clean; 13 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 149 KiB/s rd, 132 MiB/s wr, 17.58k op/s 2024-04-25T01:20:44.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:44 smithi031 ceph-mon[28053]: pgmap v534: 97 pgs: 97 active+clean; 13 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 149 KiB/s rd, 132 MiB/s wr, 17.58k op/s 2024-04-25T01:20:46.440 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:46 smithi169 ceph-mon[32759]: pgmap v535: 97 pgs: 97 active+clean; 13 GiB data, 79 GiB used, 994 GiB / 1.0 TiB avail; 125 KiB/s rd, 120 MiB/s wr, 15.70k op/s 2024-04-25T01:20:46.443 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:46 smithi040 ceph-mon[32518]: pgmap v535: 97 pgs: 97 active+clean; 13 GiB data, 79 GiB used, 994 GiB / 1.0 TiB avail; 125 KiB/s rd, 120 MiB/s wr, 15.70k op/s 2024-04-25T01:20:46.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:46 smithi031 ceph-mon[28053]: pgmap v535: 97 pgs: 97 active+clean; 13 GiB data, 79 GiB used, 994 GiB / 1.0 TiB avail; 125 KiB/s rd, 120 MiB/s wr, 15.70k op/s 2024-04-25T01:20:47.440 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:47 smithi169 ceph-mon[32759]: pgmap v536: 97 pgs: 97 active+clean; 13 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 148 KiB/s rd, 136 MiB/s wr, 17.54k op/s 2024-04-25T01:20:47.443 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:47 smithi040 ceph-mon[32518]: pgmap v536: 97 pgs: 97 active+clean; 13 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 148 KiB/s rd, 136 MiB/s wr, 17.54k op/s 2024-04-25T01:20:47.513 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:47 smithi031 ceph-mon[28053]: pgmap v536: 97 pgs: 97 active+clean; 13 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 148 KiB/s rd, 136 MiB/s wr, 17.54k op/s 2024-04-25T01:20:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.719+0000 7fd734c00640 1 -- 172.21.15.31:0/2127007616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7301087d0 msgr2=0x7fd73010abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.719+0000 7fd734c00640 1 --2- 172.21.15.31:0/2127007616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7301087d0 0x7fd73010abc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd728005fd0 tx=0x7fd72802b3e0 comp rx=0 tx=0).stop 2024-04-25T01:20:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.719+0000 7fd734c00640 1 -- 172.21.15.31:0/2127007616 shutdown_connections 2024-04-25T01:20:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.719+0000 7fd734c00640 1 --2- 172.21.15.31:0/2127007616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd73010dc70 0x7fd730110130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.719+0000 7fd734c00640 1 --2- 172.21.15.31:0/2127007616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd73010b190 0x7fd73010d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.719+0000 7fd734c00640 1 --2- 172.21.15.31:0/2127007616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7301087d0 0x7fd73010abc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.719+0000 7fd734c00640 1 -- 172.21.15.31:0/2127007616 >> 172.21.15.31:0/2127007616 conn(0x7fd73006e5e0 msgr2=0x7fd730070a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:20:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.719+0000 7fd734c00640 1 -- 172.21.15.31:0/2127007616 shutdown_connections 2024-04-25T01:20:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.720+0000 7fd734c00640 1 -- 172.21.15.31:0/2127007616 wait complete. 2024-04-25T01:20:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.725+0000 7fd734c00640 1 Processor -- start 2024-04-25T01:20:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.725+0000 7fd734c00640 1 -- start start 2024-04-25T01:20:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.726+0000 7fd734c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd73010b190 0x7fd7301a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.726+0000 7fd734c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd73010dc70 0x7fd7301a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.726+0000 7fd734c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd730077990 0x7fd730079e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.726+0000 7fd734c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd7301138d0 con 0x7fd73010b190 2024-04-25T01:20:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.726+0000 7fd734c00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd730113750 con 0x7fd730077990 2024-04-25T01:20:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.726+0000 7fd734c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd730113a50 con 0x7fd73010dc70 2024-04-25T01:20:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.726+0000 7fd72ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd73010dc70 0x7fd7301a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.726+0000 7fd72ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd73010dc70 0x7fd7301a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:38912/0 (socket says 172.21.15.31:38912) 2024-04-25T01:20:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.726+0000 7fd72ea00640 1 -- 172.21.15.31:0/3671622064 learned_addr learned my addr 172.21.15.31:0/3671622064 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:20:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.726+0000 7fd72fe00640 1 --2- 172.21.15.31:0/3671622064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd730077990 0x7fd730079e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.727+0000 7fd72ea00640 1 -- 172.21.15.31:0/3671622064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd730077990 msgr2=0x7fd730079e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.727+0000 7fd72ea00640 1 --2- 172.21.15.31:0/3671622064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd730077990 0x7fd730079e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.727+0000 7fd72ea00640 1 -- 172.21.15.31:0/3671622064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd73010b190 msgr2=0x7fd7301a0330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:20:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.731+0000 7fd72f400640 1 --2- 172.21.15.31:0/3671622064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd73010b190 0x7fd7301a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.732+0000 7fd72ea00640 1 --2- 172.21.15.31:0/3671622064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd73010b190 0x7fd7301a0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.732+0000 7fd72ea00640 1 -- 172.21.15.31:0/3671622064 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73007a410 con 0x7fd73010dc70 2024-04-25T01:20:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.732+0000 7fd72ea00640 1 --2- 172.21.15.31:0/3671622064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd73010dc70 0x7fd7301a0870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd71c00d950 tx=0x7fd71c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:20:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.733+0000 7fd727e00640 1 -- 172.21.15.31:0/3671622064 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c018070 con 0x7fd73010dc70 2024-04-25T01:20:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.733+0000 7fd727e00640 1 -- 172.21.15.31:0/3671622064 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd71c005500 con 0x7fd73010dc70 2024-04-25T01:20:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.733+0000 7fd727e00640 1 -- 172.21.15.31:0/3671622064 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c012760 con 0x7fd73010dc70 2024-04-25T01:20:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.733+0000 7fd734c00640 1 -- 172.21.15.31:0/3671622064 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd73010b630 con 0x7fd73010dc70 2024-04-25T01:20:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.734+0000 7fd734c00640 1 -- 172.21.15.31:0/3671622064 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd73010bbb0 con 0x7fd73010dc70 2024-04-25T01:20:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.750+0000 7fd727e00640 1 -- 172.21.15.31:0/3671622064 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd71c004ea0 con 0x7fd73010dc70 2024-04-25T01:20:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.755+0000 7fd739018740 1 Processor -- start 2024-04-25T01:20:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.756+0000 7fd739018740 1 -- start start 2024-04-25T01:20:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.761+0000 7fd739018740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f30bca660 0x562f30bcaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.761+0000 7fd739018740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562f30bcb050 0x562f30bcfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.761+0000 7fd739018740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bd01c0 0x562f30bd2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.761+0000 7fd739018740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562f30b24ab0 con 0x562f30bca660 2024-04-25T01:20:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.761+0000 7fd739018740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562f30acd640 con 0x562f30bcb050 2024-04-25T01:20:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.761+0000 7fd739018740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562f30b20a70 con 0x562f30bd01c0 2024-04-25T01:20:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.762+0000 7fd72ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bd01c0 0x562f30bd2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.763+0000 7fd72ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bd01c0 0x562f30bd2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:38926/0 (socket says 172.21.15.31:38926) 2024-04-25T01:20:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.763+0000 7fd72ea00640 1 -- 172.21.15.31:0/2616128659 learned_addr learned my addr 172.21.15.31:0/2616128659 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:20:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.763+0000 7fd72f400640 1 --2- 172.21.15.31:0/2616128659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562f30bcb050 0x562f30bcfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.764+0000 7fd727e00640 1 --2- 172.21.15.31:0/3671622064 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd7140a2c00 0x7fd7140a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.764+0000 7fd72fe00640 1 --2- 172.21.15.31:0/2616128659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f30bca660 0x562f30bcaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.765+0000 7fd72fe00640 1 --2- 172.21.15.31:0/3671622064 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd7140a2c00 0x7fd7140a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.765+0000 7fd72f400640 1 -- 172.21.15.31:0/2616128659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bd01c0 msgr2=0x562f30bd2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.765+0000 7fd72f400640 1 --2- 172.21.15.31:0/2616128659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bd01c0 0x562f30bd2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.765+0000 7fd72f400640 1 -- 172.21.15.31:0/2616128659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f30bca660 msgr2=0x562f30bcaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.765+0000 7fd72f400640 1 --2- 172.21.15.31:0/2616128659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f30bca660 0x562f30bcaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.765+0000 7fd72f400640 1 -- 172.21.15.31:0/2616128659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f30839210 con 0x562f30bcb050 2024-04-25T01:20:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.765+0000 7fd727e00640 1 -- 172.21.15.31:0/3671622064 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fd71c0d2b50 con 0x7fd73010dc70 2024-04-25T01:20:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.766+0000 7fd72f400640 1 --2- 172.21.15.31:0/2616128659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562f30bcb050 0x562f30bcfb50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd728009000 tx=0x7fd728004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=249525a2e55fb14b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:20:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.767+0000 7fd72fe00640 1 --2- 172.21.15.31:0/3671622064 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd7140a2c00 0x7fd7140a50c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd720006aa0 tx=0x7fd720006da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:20:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.767+0000 7fd71be00640 1 -- 172.21.15.31:0/2616128659 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728037070 con 0x562f30bcb050 2024-04-25T01:20:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.768+0000 7fd71be00640 1 -- 172.21.15.31:0/2616128659 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd728032400 con 0x562f30bcb050 2024-04-25T01:20:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.768+0000 7fd71be00640 1 -- 172.21.15.31:0/2616128659 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728003b60 con 0x562f30bcb050 2024-04-25T01:20:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.770+0000 7fd739018740 1 -- 172.21.15.31:0/2616128659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562f30bcb050 msgr2=0x562f30bcfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.770+0000 7fd739018740 1 --2- 172.21.15.31:0/2616128659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562f30bcb050 0x562f30bcfb50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd728009000 tx=0x7fd728004a40 comp rx=0 tx=0).stop 2024-04-25T01:20:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.771+0000 7fd739018740 1 -- 172.21.15.31:0/2616128659 shutdown_connections 2024-04-25T01:20:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.771+0000 7fd739018740 1 --2- 172.21.15.31:0/2616128659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bd01c0 0x562f30bd2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.774+0000 7fd739018740 1 --2- 172.21.15.31:0/2616128659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562f30bcb050 0x562f30bcfb50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.774+0000 7fd739018740 1 --2- 172.21.15.31:0/2616128659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f30bca660 0x562f30bcaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.774+0000 7fd739018740 1 -- 172.21.15.31:0/2616128659 >> 172.21.15.31:0/2616128659 conn(0x562f30bc5cd0 msgr2=0x562f30bc60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:20:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.774+0000 7fd739018740 1 -- 172.21.15.31:0/2616128659 shutdown_connections 2024-04-25T01:20:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.774+0000 7fd739018740 1 -- 172.21.15.31:0/2616128659 wait complete. 2024-04-25T01:20:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.775+0000 7fd739018740 1 -- start start 2024-04-25T01:20:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.775+0000 7fd739018740 1 Processor -- start 2024-04-25T01:20:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.777+0000 7fd739018740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bca660 0x562f30bd4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.777+0000 7fd739018740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f30bcb050 0x562f30bd4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.777+0000 7fd72f400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bca660 0x562f30bd4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.777+0000 7fd72f400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bca660 0x562f30bd4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:38932/0 (socket says 172.21.15.31:38932) 2024-04-25T01:20:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.778+0000 7fd72f400640 1 -- 172.21.15.31:0/4242776296 learned_addr learned my addr 172.21.15.31:0/4242776296 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:20:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.778+0000 7fd739018740 1 --2- 172.21.15.31:0/4242776296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562f30bd01c0 0x562f30bdbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.778+0000 7fd739018740 1 -- 172.21.15.31:0/4242776296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562f30bd3110 con 0x562f30bcb050 2024-04-25T01:20:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.778+0000 7fd72fe00640 1 --2- 172.21.15.31:0/4242776296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562f30bd01c0 0x562f30bdbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.779+0000 7fd739018740 1 -- 172.21.15.31:0/4242776296 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562f30ae0990 con 0x562f30bd01c0 2024-04-25T01:20:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.779+0000 7fd739018740 1 -- 172.21.15.31:0/4242776296 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562f30bd3290 con 0x562f30bca660 2024-04-25T01:20:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.779+0000 7fd739018740 10 client.0 _ll_register_callbacks cb 0x562f30ae2950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:20:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.780+0000 7fd72f400640 1 -- 172.21.15.31:0/4242776296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562f30bd01c0 msgr2=0x562f30bdbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.780+0000 7fd72f400640 1 --2- 172.21.15.31:0/4242776296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562f30bd01c0 0x562f30bdbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.780+0000 7fd72f400640 1 -- 172.21.15.31:0/4242776296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f30bcb050 msgr2=0x562f30bd4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.780+0000 7fd72f400640 1 --2- 172.21.15.31:0/4242776296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f30bcb050 0x562f30bd4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:47.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.780+0000 7fd72f400640 1 -- 172.21.15.31:0/4242776296 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f30905620 con 0x562f30bca660 2024-04-25T01:20:47.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.780+0000 7fd72fe00640 1 --2- 172.21.15.31:0/4242776296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562f30bd01c0 0x562f30bdbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:20:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.783+0000 7fd718c00640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:20:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.784+0000 7fd72f400640 1 --2- 172.21.15.31:0/4242776296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bca660 0x562f30bd4870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd728005e30 tx=0x7fd72800aec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e26a3875cd9c2cd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:20:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.784+0000 7fd718c00640 1 -- 172.21.15.31:0/4242776296 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72804b020 con 0x562f30bca660 2024-04-25T01:20:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.784+0000 7fd718c00640 1 -- 172.21.15.31:0/4242776296 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fd728008d70 con 0x562f30bca660 2024-04-25T01:20:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.785+0000 7fd739018740 1 -- 172.21.15.31:0/4242776296 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd728032400 con 0x562f30bca660 2024-04-25T01:20:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.785+0000 7fd718c00640 1 -- 172.21.15.31:0/4242776296 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd728037070 con 0x562f30bca660 2024-04-25T01:20:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.785+0000 7fd739018740 10 client.25576 fetch_fsmap learned FSMap version 27 2024-04-25T01:20:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.785+0000 7fd739018740 1 -- 172.21.15.31:0/4242776296 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562f30bd3140 con 0x562f30bca660 2024-04-25T01:20:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.786+0000 7fd718c00640 1 -- 172.21.15.31:0/4242776296 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd728032400 con 0x562f30bca660 2024-04-25T01:20:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.788+0000 7fd718c00640 1 -- 172.21.15.31:0/4242776296 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7fd72803a070 con 0x562f30bca660 2024-04-25T01:20:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.788+0000 7fd739018740 10 client.25576 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:20:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.788+0000 7fd739018740 10 client.25576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:20:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.788+0000 7fd739018740 20 client.25576 populate_metadata read hostname 'smithi031' 2024-04-25T01:20:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.788+0000 7fd739018740 1 --2- 172.21.15.31:0/4242776296 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x562f30be01f0 0x562f30be2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.789+0000 7fd739018740 4 client.25576 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:20:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.789+0000 7fd739018740 1 -- 172.21.15.31:0/4242776296 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f30b31ff0 con 0x562f30be01f0 2024-04-25T01:20:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.789+0000 7fd72ea00640 1 --2- 172.21.15.31:0/4242776296 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x562f30be01f0 0x562f30be2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:47.772 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.790+0000 7fd72ea00640 1 --2- 172.21.15.31:0/4242776296 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x562f30be01f0 0x562f30be2620 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28e62ff19d131d8f server_cookie=662dfc20915133f3 in_seq=0 out_seq=0 2024-04-25T01:20:47.772 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.790+0000 7fd718c00640 10 client.25576 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:20:47.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.791+0000 7fd718c00640 1 -- 172.21.15.31:0/4242776296 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562f30b31ff0 con 0x562f30be01f0 2024-04-25T01:20:47.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.791+0000 7fd718c00640 10 client.25576 handle_command_reply: tid=0 2024-04-25T01:20:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.791+0000 7fd739018740 1 client.25576 shutdown 2024-04-25T01:20:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.792+0000 7fd739018740 1 -- 172.21.15.31:0/4242776296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bca660 msgr2=0x562f30bd4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.792+0000 7fd739018740 1 --2- 172.21.15.31:0/4242776296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bca660 0x562f30bd4870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd728005e30 tx=0x7fd72800aec0 comp rx=0 tx=0).stop 2024-04-25T01:20:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.793+0000 7fd739018740 1 -- 172.21.15.31:0/4242776296 shutdown_connections 2024-04-25T01:20:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.793+0000 7fd739018740 1 --2- 172.21.15.31:0/4242776296 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x562f30be01f0 0x562f30be2620 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.793+0000 7fd739018740 1 --2- 172.21.15.31:0/4242776296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562f30bd01c0 0x562f30bdbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.794+0000 7fd739018740 1 --2- 172.21.15.31:0/4242776296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f30bcb050 0x562f30bd4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.794+0000 7fd718c00640 0 client.25576 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:20:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.794+0000 7fd739018740 1 --2- 172.21.15.31:0/4242776296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bca660 0x562f30bd4870 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.794+0000 7fd739018740 1 -- 172.21.15.31:0/4242776296 >> 172.21.15.31:0/4242776296 conn(0x562f30bc5cd0 msgr2=0x562f30bbc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:20:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.794+0000 7fd739018740 1 -- 172.21.15.31:0/4242776296 shutdown_connections 2024-04-25T01:20:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.795+0000 7fd739018740 1 -- 172.21.15.31:0/4242776296 wait complete. 2024-04-25T01:20:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.795+0000 7fd739018740 20 client.25576 trim_cache size 0 max 16384 2024-04-25T01:20:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.841+0000 7fd739018740 1 Processor -- start 2024-04-25T01:20:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.841+0000 7fd739018740 1 -- start start 2024-04-25T01:20:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.842+0000 7fd739018740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562f30bca660 0x562f30bc89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.842+0000 7fd739018740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bcb050 0x562f30bc8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.842+0000 7fd739018740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f30bd01c0 0x562f30bc9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.842+0000 7fd739018740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd710010ac0 con 0x562f30bd01c0 2024-04-25T01:20:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.842+0000 7fd739018740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd71000e330 con 0x562f30bca660 2024-04-25T01:20:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.842+0000 7fd739018740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd71000c230 con 0x562f30bcb050 2024-04-25T01:20:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.843+0000 7fd72f400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562f30bca660 0x562f30bc89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.843+0000 7fd72ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bcb050 0x562f30bc8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.843+0000 7fd72ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bcb050 0x562f30bc8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:38934/0 (socket says 172.21.15.31:38934) 2024-04-25T01:20:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.843+0000 7fd72ea00640 1 -- 172.21.15.31:0/2352970117 learned_addr learned my addr 172.21.15.31:0/2352970117 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:20:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.844+0000 7fd72ea00640 1 -- 172.21.15.31:0/2352970117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562f30bca660 msgr2=0x562f30bc89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.844+0000 7fd72ea00640 1 --2- 172.21.15.31:0/2352970117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562f30bca660 0x562f30bc89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.845+0000 7fd72ea00640 1 -- 172.21.15.31:0/2352970117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f30bd01c0 msgr2=0x562f30bc9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.845+0000 7fd72fe00640 1 --2- 172.21.15.31:0/2352970117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f30bd01c0 0x562f30bc9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.846+0000 7fd72ea00640 1 --2- 172.21.15.31:0/2352970117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f30bd01c0 0x562f30bc9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.846+0000 7fd72ea00640 1 -- 172.21.15.31:0/2352970117 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f30bc9970 con 0x562f30bcb050 2024-04-25T01:20:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.846+0000 7fd72fe00640 1 --2- 172.21.15.31:0/2352970117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f30bd01c0 0x562f30bc9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:20:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.847+0000 7fd72ea00640 1 --2- 172.21.15.31:0/2352970117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bcb050 0x562f30bc8ef0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009610 tx=0x7fd71c010040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62c370520efa1b41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:20:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.847+0000 7fd71b400640 1 -- 172.21.15.31:0/2352970117 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c0a37f0 con 0x562f30bcb050 2024-04-25T01:20:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.848+0000 7fd71b400640 1 -- 172.21.15.31:0/2352970117 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd71c0d2e50 con 0x562f30bcb050 2024-04-25T01:20:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.848+0000 7fd71b400640 1 -- 172.21.15.31:0/2352970117 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c09bab0 con 0x562f30bcb050 2024-04-25T01:20:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.851+0000 7fd739018740 1 -- 172.21.15.31:0/2352970117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bcb050 msgr2=0x562f30bc8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.852+0000 7fd739018740 1 --2- 172.21.15.31:0/2352970117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bcb050 0x562f30bc8ef0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009610 tx=0x7fd71c010040 comp rx=0 tx=0).stop 2024-04-25T01:20:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.852+0000 7fd739018740 1 -- 172.21.15.31:0/2352970117 shutdown_connections 2024-04-25T01:20:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.853+0000 7fd739018740 1 --2- 172.21.15.31:0/2352970117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f30bd01c0 0x562f30bc9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.853+0000 7fd739018740 1 --2- 172.21.15.31:0/2352970117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bcb050 0x562f30bc8ef0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.853+0000 7fd739018740 1 --2- 172.21.15.31:0/2352970117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562f30bca660 0x562f30bc89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.853+0000 7fd739018740 1 -- 172.21.15.31:0/2352970117 >> 172.21.15.31:0/2352970117 conn(0x562f30bc5cd0 msgr2=0x562f30bcf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:20:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.854+0000 7fd739018740 1 -- 172.21.15.31:0/2352970117 shutdown_connections 2024-04-25T01:20:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.854+0000 7fd739018740 1 -- 172.21.15.31:0/2352970117 wait complete. 2024-04-25T01:20:47.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.855+0000 7fd739018740 1 -- start start 2024-04-25T01:20:47.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.856+0000 7fd739018740 1 Processor -- start 2024-04-25T01:20:47.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.858+0000 7fd739018740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bca660 0x562f30bd85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.859+0000 7fd739018740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562f30bcb050 0x562f30bd8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.859+0000 7fd739018740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f30bd01c0 0x562f30bd9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.860+0000 7fd739018740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd71000c7f0 con 0x562f30bd01c0 2024-04-25T01:20:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.860+0000 7fd739018740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd71000dc60 con 0x562f30bcb050 2024-04-25T01:20:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.860+0000 7fd739018740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd71000cec0 con 0x562f30bca660 2024-04-25T01:20:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.860+0000 7fd739018740 10 client.0 _ll_register_callbacks cb 0x7fd7280076b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:20:47.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.861+0000 7fd72f400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bca660 0x562f30bd85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:47.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.861+0000 7fd72ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562f30bcb050 0x562f30bd8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.863+0000 7fd72ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562f30bcb050 0x562f30bd8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:57374/0 (socket says 172.21.15.31:57374) 2024-04-25T01:20:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.863+0000 7fd72ea00640 1 -- 172.21.15.31:0/3849569577 learned_addr learned my addr 172.21.15.31:0/3849569577 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:20:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.863+0000 7fd72f400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bca660 0x562f30bd85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:38950/0 (socket says 172.21.15.31:38950) 2024-04-25T01:20:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.863+0000 7fd72fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f30bd01c0 0x562f30bd9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:47.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.864+0000 7fd72ea00640 1 -- 172.21.15.31:0/3849569577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bca660 msgr2=0x562f30bd85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:47.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.864+0000 7fd72ea00640 1 --2- 172.21.15.31:0/3849569577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bca660 0x562f30bd85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:47.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.864+0000 7fd72ea00640 1 -- 172.21.15.31:0/3849569577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f30bd01c0 msgr2=0x562f30bd9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:20:47.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.865+0000 7fd72ea00640 1 --2- 172.21.15.31:0/3849569577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f30bd01c0 0x562f30bd9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:47.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.865+0000 7fd72ea00640 1 -- 172.21.15.31:0/3849569577 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f30bd9ff0 con 0x562f30bcb050 2024-04-25T01:20:47.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.865+0000 7fd72fe00640 1 --2- 172.21.15.31:0/3849569577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f30bd01c0 0x562f30bd9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:20:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.867+0000 7fd72ea00640 1 --2- 172.21.15.31:0/3849569577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562f30bcb050 0x562f30bd8b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0097f0 tx=0x7fd71c09f2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe7ef5250828e487 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:20:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.869+0000 7fd718c00640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T01:20:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.869+0000 7fd718c00640 1 -- 172.21.15.31:0/3849569577 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c0b8060 con 0x562f30bcb050 2024-04-25T01:20:47.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.869+0000 7fd739018740 1 -- 172.21.15.31:0/3849569577 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd71c0d2e50 con 0x562f30bcb050 2024-04-25T01:20:47.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.869+0000 7fd718c00640 1 -- 172.21.15.31:0/3849569577 <== mon.2 v2:172.21.15.40:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fd71c09bab0 con 0x562f30bcb050 2024-04-25T01:20:47.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.870+0000 7fd718c00640 1 -- 172.21.15.31:0/3849569577 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd71c0a80b0 con 0x562f30bcb050 2024-04-25T01:20:47.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.871+0000 7fd739018740 10 client.25556 fetch_fsmap learned FSMap version 27 2024-04-25T01:20:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.871+0000 7fd739018740 1 -- 172.21.15.31:0/3849569577 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562f30bdb2a0 con 0x562f30bcb050 2024-04-25T01:20:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.871+0000 7fd718c00640 1 -- 172.21.15.31:0/3849569577 <== mon.2 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd71c0d2e50 con 0x562f30bcb050 2024-04-25T01:20:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.871+0000 7fd718c00640 1 -- 172.21.15.31:0/3849569577 <== mon.2 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7fd71c005190 con 0x562f30bcb050 2024-04-25T01:20:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.873+0000 7fd739018740 10 client.25556 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:20:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.874+0000 7fd739018740 10 client.25556 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:20:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.874+0000 7fd739018740 20 client.25556 populate_metadata read hostname 'smithi031' 2024-04-25T01:20:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.874+0000 7fd739018740 1 --2- 172.21.15.31:0/3849569577 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x562f30be01f0 0x562f30c48b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:20:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.874+0000 7fd739018740 4 client.25556 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:20:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.874+0000 7fd739018740 1 -- 172.21.15.31:0/3849569577 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562f30b2c340 con 0x562f30be01f0 2024-04-25T01:20:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.875+0000 7fd72f400640 1 --2- 172.21.15.31:0/3849569577 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x562f30be01f0 0x562f30c48b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:20:47.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.877+0000 7fd72f400640 1 --2- 172.21.15.31:0/3849569577 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x562f30be01f0 0x562f30c48b70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8e322012c1163c6 server_cookie=ac9c5a79e7815383 in_seq=0 out_seq=0 2024-04-25T01:20:47.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:47.878+0000 7fd718c00640 10 client.25556 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:20:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.142+0000 7fd718c00640 1 -- 172.21.15.31:0/3849569577 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562f30b2c340 con 0x562f30be01f0 2024-04-25T01:20:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.142+0000 7fd718c00640 10 client.25556 handle_command_reply: tid=0 2024-04-25T01:20:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.142+0000 7fd739018740 1 client.25556 shutdown 2024-04-25T01:20:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.144+0000 7fd739018740 1 -- 172.21.15.31:0/3849569577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562f30bcb050 msgr2=0x562f30bd8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.144+0000 7fd739018740 1 --2- 172.21.15.31:0/3849569577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562f30bcb050 0x562f30bd8b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0097f0 tx=0x7fd71c09f2e0 comp rx=0 tx=0).stop 2024-04-25T01:20:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.144+0000 7fd739018740 1 -- 172.21.15.31:0/3849569577 shutdown_connections 2024-04-25T01:20:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.144+0000 7fd739018740 1 --2- 172.21.15.31:0/3849569577 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x562f30be01f0 0x562f30c48b70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.144+0000 7fd739018740 1 --2- 172.21.15.31:0/3849569577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f30bd01c0 0x562f30bd9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.144+0000 7fd739018740 1 --2- 172.21.15.31:0/3849569577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562f30bcb050 0x562f30bd8b00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.144+0000 7fd739018740 1 --2- 172.21.15.31:0/3849569577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562f30bca660 0x562f30bd85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.144+0000 7fd739018740 1 -- 172.21.15.31:0/3849569577 >> 172.21.15.31:0/3849569577 conn(0x562f30bc5cd0 msgr2=0x562f30bc89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:20:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.144+0000 7fd718c00640 0 client.25556 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:20:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.144+0000 7fd739018740 1 -- 172.21.15.31:0/3849569577 shutdown_connections 2024-04-25T01:20:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.144+0000 7fd739018740 1 -- 172.21.15.31:0/3849569577 wait complete. 2024-04-25T01:20:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.145+0000 7fd739018740 20 client.25556 trim_cache size 0 max 16384 2024-04-25T01:20:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:20:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:20:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:20:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:20:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.146+0000 7fd734c00640 1 -- 172.21.15.31:0/3671622064 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd7140a2c00 msgr2=0x7fd7140a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.146+0000 7fd734c00640 1 --2- 172.21.15.31:0/3671622064 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd7140a2c00 0x7fd7140a50c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd720006aa0 tx=0x7fd720006da0 comp rx=0 tx=0).stop 2024-04-25T01:20:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.146+0000 7fd734c00640 1 -- 172.21.15.31:0/3671622064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd73010dc70 msgr2=0x7fd7301a0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:20:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.146+0000 7fd734c00640 1 --2- 172.21.15.31:0/3671622064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd73010dc70 0x7fd7301a0870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd71c00d950 tx=0x7fd71c00de10 comp rx=0 tx=0).stop 2024-04-25T01:20:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.147+0000 7fd734c00640 1 -- 172.21.15.31:0/3671622064 shutdown_connections 2024-04-25T01:20:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.147+0000 7fd734c00640 1 --2- 172.21.15.31:0/3671622064 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd7140a2c00 0x7fd7140a50c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.147+0000 7fd734c00640 1 --2- 172.21.15.31:0/3671622064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd730077990 0x7fd730079e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.147+0000 7fd734c00640 1 --2- 172.21.15.31:0/3671622064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd73010dc70 0x7fd7301a0870 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.147+0000 7fd734c00640 1 --2- 172.21.15.31:0/3671622064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd73010b190 0x7fd7301a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:20:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.147+0000 7fd734c00640 1 -- 172.21.15.31:0/3671622064 >> 172.21.15.31:0/3671622064 conn(0x7fd73006e5e0 msgr2=0x7fd73010e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:20:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.147+0000 7fd734c00640 1 -- 172.21.15.31:0/3671622064 shutdown_connections 2024-04-25T01:20:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:20:48.147+0000 7fd734c00640 1 -- 172.21.15.31:0/3671622064 wait complete. 2024-04-25T01:20:49.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:49 smithi169 ceph-mon[32759]: pgmap v537: 97 pgs: 97 active+clean; 13 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 126 KiB/s rd, 125 MiB/s wr, 16.26k op/s 2024-04-25T01:20:49.693 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:49 smithi040 ceph-mon[32518]: pgmap v537: 97 pgs: 97 active+clean; 13 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 126 KiB/s rd, 125 MiB/s wr, 16.26k op/s 2024-04-25T01:20:49.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:49 smithi031 ceph-mon[28053]: pgmap v537: 97 pgs: 97 active+clean; 13 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 126 KiB/s rd, 125 MiB/s wr, 16.26k op/s 2024-04-25T01:20:51.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:51 smithi169 ceph-mon[32759]: pgmap v538: 97 pgs: 97 active+clean; 13 GiB data, 81 GiB used, 991 GiB / 1.0 TiB avail; 145 KiB/s rd, 135 MiB/s wr, 17.50k op/s 2024-04-25T01:20:51.693 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:51 smithi040 ceph-mon[32518]: pgmap v538: 97 pgs: 97 active+clean; 13 GiB data, 81 GiB used, 991 GiB / 1.0 TiB avail; 145 KiB/s rd, 135 MiB/s wr, 17.50k op/s 2024-04-25T01:20:51.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:51 smithi031 ceph-mon[28053]: pgmap v538: 97 pgs: 97 active+clean; 13 GiB data, 81 GiB used, 991 GiB / 1.0 TiB avail; 145 KiB/s rd, 135 MiB/s wr, 17.50k op/s 2024-04-25T01:20:53.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:53 smithi031 ceph-mon[28053]: pgmap v539: 97 pgs: 97 active+clean; 13 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 107 KiB/s rd, 128 MiB/s wr, 16.83k op/s 2024-04-25T01:20:53.940 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:53 smithi169 ceph-mon[32759]: pgmap v539: 97 pgs: 97 active+clean; 13 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 107 KiB/s rd, 128 MiB/s wr, 16.83k op/s 2024-04-25T01:20:53.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:53 smithi040 ceph-mon[32518]: pgmap v539: 97 pgs: 97 active+clean; 13 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 107 KiB/s rd, 128 MiB/s wr, 16.83k op/s 2024-04-25T01:20:55.940 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:55 smithi169 ceph-mon[32759]: pgmap v540: 97 pgs: 97 active+clean; 13 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 89 KiB/s rd, 115 MiB/s wr, 15.44k op/s 2024-04-25T01:20:55.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:55 smithi040 ceph-mon[32518]: pgmap v540: 97 pgs: 97 active+clean; 13 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 89 KiB/s rd, 115 MiB/s wr, 15.44k op/s 2024-04-25T01:20:56.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:55 smithi031 ceph-mon[28053]: pgmap v540: 97 pgs: 97 active+clean; 13 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 89 KiB/s rd, 115 MiB/s wr, 15.44k op/s 2024-04-25T01:20:57.940 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:57 smithi169 ceph-mon[32759]: pgmap v541: 97 pgs: 97 active+clean; 13 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 124 KiB/s rd, 141 MiB/s wr, 19.32k op/s 2024-04-25T01:20:57.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:57 smithi040 ceph-mon[32518]: pgmap v541: 97 pgs: 97 active+clean; 13 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 124 KiB/s rd, 141 MiB/s wr, 19.32k op/s 2024-04-25T01:20:58.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:57 smithi031 ceph-mon[28053]: pgmap v541: 97 pgs: 97 active+clean; 13 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 124 KiB/s rd, 141 MiB/s wr, 19.32k op/s 2024-04-25T01:20:59.940 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:20:59 smithi169 ceph-mon[32759]: pgmap v542: 97 pgs: 97 active+clean; 13 GiB data, 83 GiB used, 989 GiB / 1.0 TiB avail; 125 KiB/s rd, 131 MiB/s wr, 18.23k op/s 2024-04-25T01:20:59.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:20:59 smithi040 ceph-mon[32518]: pgmap v542: 97 pgs: 97 active+clean; 13 GiB data, 83 GiB used, 989 GiB / 1.0 TiB avail; 125 KiB/s rd, 131 MiB/s wr, 18.23k op/s 2024-04-25T01:21:00.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:20:59 smithi031 ceph-mon[28053]: pgmap v542: 97 pgs: 97 active+clean; 13 GiB data, 83 GiB used, 989 GiB / 1.0 TiB avail; 125 KiB/s rd, 131 MiB/s wr, 18.23k op/s 2024-04-25T01:21:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.385+0000 7f5066200640 1 -- 172.21.15.31:0/965878223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5060108fc0 msgr2=0x7f5060109440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.385+0000 7f5066200640 1 --2- 172.21.15.31:0/965878223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5060108fc0 0x7f5060109440 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f504c009900 tx=0x7f504c02a400 comp rx=0 tx=0).stop 2024-04-25T01:21:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.386+0000 7f5066200640 1 -- 172.21.15.31:0/965878223 shutdown_connections 2024-04-25T01:21:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.386+0000 7f5066200640 1 --2- 172.21.15.31:0/965878223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5060109980 0x7f5060110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.386+0000 7f5066200640 1 --2- 172.21.15.31:0/965878223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5060108fc0 0x7f5060109440 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.390+0000 7f5066200640 1 --2- 172.21.15.31:0/965878223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5060107dd0 0x7f50601081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.390+0000 7f5066200640 1 -- 172.21.15.31:0/965878223 >> 172.21.15.31:0/965878223 conn(0x7f5060076d90 msgr2=0x7f50600791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.393+0000 7f5066200640 1 -- 172.21.15.31:0/965878223 shutdown_connections 2024-04-25T01:21:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.394+0000 7f5066200640 1 -- 172.21.15.31:0/965878223 wait complete. 2024-04-25T01:21:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.396+0000 7f5066200640 1 Processor -- start 2024-04-25T01:21:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.396+0000 7f5066200640 1 -- start start 2024-04-25T01:21:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.397+0000 7f5066200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5060107dd0 0x7f506019e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.397+0000 7f5066200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5060108fc0 0x7f506019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.398+0000 7f505ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5060108fc0 0x7f506019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.398+0000 7f505ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5060108fc0 0x7f506019e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:53746/0 (socket says 172.21.15.31:53746) 2024-04-25T01:21:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.398+0000 7f505ea00640 1 -- 172.21.15.31:0/671317654 learned_addr learned my addr 172.21.15.31:0/671317654 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:21:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.398+0000 7f5066200640 1 --2- 172.21.15.31:0/671317654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5060109980 0x7f50601a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.399+0000 7f5066200640 1 -- 172.21.15.31:0/671317654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5060113b00 con 0x7f5060107dd0 2024-04-25T01:21:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.399+0000 7f5066200640 1 -- 172.21.15.31:0/671317654 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5060113980 con 0x7f5060108fc0 2024-04-25T01:21:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.399+0000 7f5066200640 1 -- 172.21.15.31:0/671317654 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5060113c80 con 0x7f5060109980 2024-04-25T01:21:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.399+0000 7f505f400640 1 --2- 172.21.15.31:0/671317654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5060107dd0 0x7f506019e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.399+0000 7f505fe00640 1 --2- 172.21.15.31:0/671317654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5060109980 0x7f50601a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.400+0000 7f505ea00640 1 -- 172.21.15.31:0/671317654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5060109980 msgr2=0x7f50601a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.400+0000 7f505ea00640 1 --2- 172.21.15.31:0/671317654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5060109980 0x7f50601a5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.400+0000 7f505ea00640 1 -- 172.21.15.31:0/671317654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5060107dd0 msgr2=0x7f506019e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.400+0000 7f505ea00640 1 --2- 172.21.15.31:0/671317654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5060107dd0 0x7f506019e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.400+0000 7f505ea00640 1 -- 172.21.15.31:0/671317654 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50601a5ca0 con 0x7f5060108fc0 2024-04-25T01:21:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.401+0000 7f505fe00640 1 --2- 172.21.15.31:0/671317654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5060109980 0x7f50601a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:21:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.402+0000 7f505ea00640 1 --2- 172.21.15.31:0/671317654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5060108fc0 0x7f506019e5c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f504c031440 tx=0x7f504c032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.403+0000 7f5057e00640 1 -- 172.21.15.31:0/671317654 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c03f030 con 0x7f5060108fc0 2024-04-25T01:21:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.404+0000 7f5057e00640 1 -- 172.21.15.31:0/671317654 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f504c003ce0 con 0x7f5060108fc0 2024-04-25T01:21:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.404+0000 7f5057e00640 1 -- 172.21.15.31:0/671317654 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c035070 con 0x7f5060108fc0 2024-04-25T01:21:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.405+0000 7f5066200640 1 -- 172.21.15.31:0/671317654 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50601a5f30 con 0x7f5060108fc0 2024-04-25T01:21:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.405+0000 7f5066200640 1 -- 172.21.15.31:0/671317654 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50601a6420 con 0x7f5060108fc0 2024-04-25T01:21:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.409+0000 7f506905b740 1 Processor -- start 2024-04-25T01:21:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.413+0000 7f506905b740 1 -- start start 2024-04-25T01:21:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.413+0000 7f506905b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560135ca2ca0 0x560135ca30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.413+0000 7f506905b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560135ca3690 0x560135ca8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.413+0000 7f506905b740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca8800 0x560135caac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.413+0000 7f506905b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560135c05ab0 con 0x560135ca3690 2024-04-25T01:21:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.413+0000 7f506905b740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560135bae640 con 0x560135ca8800 2024-04-25T01:21:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.413+0000 7f506905b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560135c01a70 con 0x560135ca2ca0 2024-04-25T01:21:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.416+0000 7f5057e00640 1 -- 172.21.15.31:0/671317654 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f504c042030 con 0x7f5060108fc0 2024-04-25T01:21:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.416+0000 7f505ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca8800 0x560135caac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.416+0000 7f505ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca8800 0x560135caac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:53748/0 (socket says 172.21.15.31:53748) 2024-04-25T01:21:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.416+0000 7f505ea00640 1 -- 172.21.15.31:0/2219431312 learned_addr learned my addr 172.21.15.31:0/2219431312 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:21:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.417+0000 7f505fe00640 1 --2- 172.21.15.31:0/2219431312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560135ca2ca0 0x560135ca30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.417+0000 7f505f400640 1 --2- 172.21.15.31:0/2219431312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560135ca3690 0x560135ca8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.419+0000 7f505f400640 1 -- 172.21.15.31:0/2219431312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560135ca2ca0 msgr2=0x560135ca30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.419+0000 7f505f400640 1 --2- 172.21.15.31:0/2219431312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560135ca2ca0 0x560135ca30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.419+0000 7f505f400640 1 -- 172.21.15.31:0/2219431312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca8800 msgr2=0x560135caac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.419+0000 7f505f400640 1 --2- 172.21.15.31:0/2219431312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca8800 0x560135caac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.422+0000 7f505f400640 1 -- 172.21.15.31:0/2219431312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56013591a210 con 0x560135ca3690 2024-04-25T01:21:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.422+0000 7f505ea00640 1 --2- 172.21.15.31:0/2219431312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca8800 0x560135caac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:21:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.423+0000 7f505f400640 1 --2- 172.21.15.31:0/2219431312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560135ca3690 0x560135ca8190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f50500058b0 tx=0x7f5050011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d9994254566f89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.424+0000 7f504be00640 1 -- 172.21.15.31:0/2219431312 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050004030 con 0x560135ca3690 2024-04-25T01:21:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.424+0000 7f504be00640 1 -- 172.21.15.31:0/2219431312 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5050005500 con 0x560135ca3690 2024-04-25T01:21:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.424+0000 7f504be00640 1 -- 172.21.15.31:0/2219431312 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50500035f0 con 0x560135ca3690 2024-04-25T01:21:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.425+0000 7f506905b740 1 -- 172.21.15.31:0/2219431312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560135ca3690 msgr2=0x560135ca8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.425+0000 7f506905b740 1 --2- 172.21.15.31:0/2219431312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560135ca3690 0x560135ca8190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f50500058b0 tx=0x7f5050011810 comp rx=0 tx=0).stop 2024-04-25T01:21:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.426+0000 7f506905b740 1 -- 172.21.15.31:0/2219431312 shutdown_connections 2024-04-25T01:21:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.426+0000 7f506905b740 1 --2- 172.21.15.31:0/2219431312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca8800 0x560135caac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.426+0000 7f506905b740 1 --2- 172.21.15.31:0/2219431312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560135ca3690 0x560135ca8190 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.426+0000 7f506905b740 1 --2- 172.21.15.31:0/2219431312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560135ca2ca0 0x560135ca30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.426+0000 7f506905b740 1 -- 172.21.15.31:0/2219431312 >> 172.21.15.31:0/2219431312 conn(0x560135c9e310 msgr2=0x560135c9e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.426+0000 7f5057e00640 1 --2- 172.21.15.31:0/671317654 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f50440a2d30 0x7f50440a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.426+0000 7f5057e00640 1 -- 172.21.15.31:0/671317654 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f504c0ef4c0 con 0x7f5060108fc0 2024-04-25T01:21:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.427+0000 7f505fe00640 1 --2- 172.21.15.31:0/671317654 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f50440a2d30 0x7f50440a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.427+0000 7f506905b740 1 -- 172.21.15.31:0/2219431312 shutdown_connections 2024-04-25T01:21:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.427+0000 7f506905b740 1 -- 172.21.15.31:0/2219431312 wait complete. 2024-04-25T01:21:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.428+0000 7f506905b740 1 -- start start 2024-04-25T01:21:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.429+0000 7f506905b740 1 Processor -- start 2024-04-25T01:21:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.429+0000 7f505fe00640 1 --2- 172.21.15.31:0/671317654 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f50440a2d30 0x7f50440a51f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x560135ca3ef0 tx=0x7f5058010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.430+0000 7f506905b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560135ca2ca0 0x560135cacf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.430+0000 7f506905b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560135ca3690 0x560135cad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.430+0000 7f506905b740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca8800 0x560135cb44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.430+0000 7f506905b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560135cab7b0 con 0x560135ca2ca0 2024-04-25T01:21:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.430+0000 7f506905b740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560135bc1990 con 0x560135ca8800 2024-04-25T01:21:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.430+0000 7f506905b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560135cab930 con 0x560135ca3690 2024-04-25T01:21:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.431+0000 7f506905b740 10 client.0 _ll_register_callbacks cb 0x560135bc3950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:21:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.431+0000 7f505fe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca8800 0x560135cb44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.431+0000 7f505fe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca8800 0x560135cb44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:53754/0 (socket says 172.21.15.31:53754) 2024-04-25T01:21:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.431+0000 7f505fe00640 1 -- 172.21.15.31:0/80427760 learned_addr learned my addr 172.21.15.31:0/80427760 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:21:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.432+0000 7f505ea00640 1 --2- 172.21.15.31:0/80427760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560135ca3690 0x560135cad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.433+0000 7f505fe00640 1 -- 172.21.15.31:0/80427760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560135ca3690 msgr2=0x560135cad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.433+0000 7f505fe00640 1 --2- 172.21.15.31:0/80427760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560135ca3690 0x560135cad4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.433+0000 7f505fe00640 1 -- 172.21.15.31:0/80427760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560135ca2ca0 msgr2=0x560135cacf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.433+0000 7f505fe00640 1 --2- 172.21.15.31:0/80427760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560135ca2ca0 0x560135cacf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.433+0000 7f505fe00640 1 -- 172.21.15.31:0/80427760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601359e6620 con 0x560135ca8800 2024-04-25T01:21:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.433+0000 7f505fe00640 1 --2- 172.21.15.31:0/80427760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca8800 0x560135cb44f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5058007ec0 tx=0x7f505800e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=349b420427a9da6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.433+0000 7f5048c00640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T01:21:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.434+0000 7f5048c00640 1 -- 172.21.15.31:0/80427760 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505802a060 con 0x560135ca8800 2024-04-25T01:21:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.434+0000 7f5048c00640 1 -- 172.21.15.31:0/80427760 <== mon.2 v2:172.21.15.40:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f5058025420 con 0x560135ca8800 2024-04-25T01:21:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.435+0000 7f506905b740 1 -- 172.21.15.31:0/80427760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5050005500 con 0x560135ca8800 2024-04-25T01:21:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.436+0000 7f5048c00640 1 -- 172.21.15.31:0/80427760 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5058025620 con 0x560135ca8800 2024-04-25T01:21:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.436+0000 7f506905b740 10 client.25568 fetch_fsmap learned FSMap version 27 2024-04-25T01:21:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.437+0000 7f506905b740 1 -- 172.21.15.31:0/80427760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560135cc8550 con 0x560135ca8800 2024-04-25T01:21:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.437+0000 7f5048c00640 1 -- 172.21.15.31:0/80427760 <== mon.2 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5050005500 con 0x560135ca8800 2024-04-25T01:21:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.439+0000 7f5048c00640 1 -- 172.21.15.31:0/80427760 <== mon.2 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7f505802d350 con 0x560135ca8800 2024-04-25T01:21:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.439+0000 7f506905b740 10 client.25568 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:21:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.439+0000 7f506905b740 10 client.25568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:21:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.441+0000 7f506905b740 20 client.25568 populate_metadata read hostname 'smithi031' 2024-04-25T01:21:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.441+0000 7f506905b740 1 --2- 172.21.15.31:0/80427760 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560135cc9bf0 0x560135ccc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.441+0000 7f506905b740 4 client.25568 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:21:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.441+0000 7f506905b740 1 -- 172.21.15.31:0/80427760 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560135c12ff0 con 0x560135cc9bf0 2024-04-25T01:21:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.444+0000 7f505f400640 1 --2- 172.21.15.31:0/80427760 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560135cc9bf0 0x560135ccc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.446+0000 7f5048c00640 10 client.25568 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:21:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.446+0000 7f505f400640 1 --2- 172.21.15.31:0/80427760 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560135cc9bf0 0x560135ccc080 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9702bbda0d1fc64 server_cookie=fd4514d38fa3925 in_seq=0 out_seq=0 2024-04-25T01:21:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.448+0000 7f5048c00640 1 -- 172.21.15.31:0/80427760 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560135c12ff0 con 0x560135cc9bf0 2024-04-25T01:21:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.448+0000 7f5048c00640 10 client.25568 handle_command_reply: tid=0 2024-04-25T01:21:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.448+0000 7f506905b740 1 client.25568 shutdown 2024-04-25T01:21:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.454+0000 7f506905b740 1 -- 172.21.15.31:0/80427760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca8800 msgr2=0x560135cb44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.454+0000 7f506905b740 1 --2- 172.21.15.31:0/80427760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca8800 0x560135cb44f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5058007ec0 tx=0x7f505800e040 comp rx=0 tx=0).stop 2024-04-25T01:21:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.454+0000 7f506905b740 1 -- 172.21.15.31:0/80427760 shutdown_connections 2024-04-25T01:21:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.454+0000 7f506905b740 1 --2- 172.21.15.31:0/80427760 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560135cc9bf0 0x560135ccc080 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.454+0000 7f506905b740 1 --2- 172.21.15.31:0/80427760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca8800 0x560135cb44f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.454+0000 7f506905b740 1 --2- 172.21.15.31:0/80427760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560135ca3690 0x560135cad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.454+0000 7f506905b740 1 --2- 172.21.15.31:0/80427760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560135ca2ca0 0x560135cacf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.454+0000 7f506905b740 1 -- 172.21.15.31:0/80427760 >> 172.21.15.31:0/80427760 conn(0x560135c9e310 msgr2=0x560135c9d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.454+0000 7f5048c00640 0 client.25568 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:21:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.455+0000 7f506905b740 1 -- 172.21.15.31:0/80427760 shutdown_connections 2024-04-25T01:21:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.455+0000 7f506905b740 1 -- 172.21.15.31:0/80427760 wait complete. 2024-04-25T01:21:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.455+0000 7f506905b740 20 client.25568 trim_cache size 0 max 16384 2024-04-25T01:21:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.518+0000 7f506905b740 1 Processor -- start 2024-04-25T01:21:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.518+0000 7f506905b740 1 -- start start 2024-04-25T01:21:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.519+0000 7f506905b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560135ca2ca0 0x560135cabd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.519+0000 7f506905b740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca3690 0x560135cb44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.519+0000 7f506905b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560135ca8800 0x560135ca1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.519+0000 7f506905b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f506006de10 con 0x560135ca8800 2024-04-25T01:21:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.519+0000 7f506905b740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f506006b680 con 0x560135ca3690 2024-04-25T01:21:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.519+0000 7f506905b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5060080b10 con 0x560135ca2ca0 2024-04-25T01:21:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.520+0000 7f505fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560135ca8800 0x560135ca1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.520+0000 7f505fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560135ca8800 0x560135ca1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57082/0 (socket says 172.21.15.31:57082) 2024-04-25T01:21:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.520+0000 7f505ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca3690 0x560135cb44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.520+0000 7f505ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca3690 0x560135cb44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:53756/0 (socket says 172.21.15.31:53756) 2024-04-25T01:21:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.520+0000 7f505ea00640 1 -- 172.21.15.31:0/2141071617 learned_addr learned my addr 172.21.15.31:0/2141071617 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:21:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.521+0000 7f505fe00640 1 -- 172.21.15.31:0/2141071617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560135ca2ca0 msgr2=0x560135cabd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.521+0000 7f505fe00640 1 --2- 172.21.15.31:0/2141071617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560135ca2ca0 0x560135cabd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.521+0000 7f505fe00640 1 -- 172.21.15.31:0/2141071617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca3690 msgr2=0x560135cb44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.521+0000 7f505fe00640 1 --2- 172.21.15.31:0/2141071617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca3690 0x560135cb44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.521+0000 7f505fe00640 1 -- 172.21.15.31:0/2141071617 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560135c9c890 con 0x560135ca8800 2024-04-25T01:21:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.522+0000 7f505fe00640 1 --2- 172.21.15.31:0/2141071617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560135ca8800 0x560135ca1220 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f505802d8e0 tx=0x7f5058007450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78d202a664468f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.523+0000 7f504b400640 1 -- 172.21.15.31:0/2141071617 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5058034080 con 0x560135ca8800 2024-04-25T01:21:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.524+0000 7f504b400640 1 -- 172.21.15.31:0/2141071617 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f505802de40 con 0x560135ca8800 2024-04-25T01:21:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.524+0000 7f504b400640 1 -- 172.21.15.31:0/2141071617 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5058025bb0 con 0x560135ca8800 2024-04-25T01:21:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.525+0000 7f506905b740 1 -- 172.21.15.31:0/2141071617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560135ca8800 msgr2=0x560135ca1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.525+0000 7f506905b740 1 --2- 172.21.15.31:0/2141071617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560135ca8800 0x560135ca1220 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f505802d8e0 tx=0x7f5058007450 comp rx=0 tx=0).stop 2024-04-25T01:21:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.526+0000 7f506905b740 1 -- 172.21.15.31:0/2141071617 shutdown_connections 2024-04-25T01:21:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.527+0000 7f506905b740 1 --2- 172.21.15.31:0/2141071617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560135ca8800 0x560135ca1220 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.527+0000 7f506905b740 1 --2- 172.21.15.31:0/2141071617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca3690 0x560135cb44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.527+0000 7f506905b740 1 --2- 172.21.15.31:0/2141071617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560135ca2ca0 0x560135cabd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.527+0000 7f506905b740 1 -- 172.21.15.31:0/2141071617 >> 172.21.15.31:0/2141071617 conn(0x560135c9e310 msgr2=0x560135ccb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.528+0000 7f506905b740 1 -- 172.21.15.31:0/2141071617 shutdown_connections 2024-04-25T01:21:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.528+0000 7f506905b740 1 -- 172.21.15.31:0/2141071617 wait complete. 2024-04-25T01:21:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.528+0000 7f506905b740 1 -- start start 2024-04-25T01:21:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.529+0000 7f506905b740 1 Processor -- start 2024-04-25T01:21:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.533+0000 7f506905b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560135ca2ca0 0x560135cb1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.533+0000 7f506905b740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca3690 0x560135cb21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.533+0000 7f506905b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560135ca8800 0x560135cb3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.535+0000 7f505fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560135ca8800 0x560135cb3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.535+0000 7f505fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560135ca8800 0x560135cb3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:40038/0 (socket says 172.21.15.31:40038) 2024-04-25T01:21:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.535+0000 7f505ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca3690 0x560135cb21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.536+0000 7f505ea00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca3690 0x560135cb21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:53760/0 (socket says 172.21.15.31:53760) 2024-04-25T01:21:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.537+0000 7f505fe00640 1 -- 172.21.15.31:0/966093973 learned_addr learned my addr 172.21.15.31:0/966093973 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:21:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.537+0000 7f505f400640 1 --2- 172.21.15.31:0/966093973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560135ca2ca0 0x560135cb1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.537+0000 7f506905b740 1 -- 172.21.15.31:0/966093973 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5060081180 con 0x560135ca2ca0 2024-04-25T01:21:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.537+0000 7f506905b740 1 -- 172.21.15.31:0/966093973 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f50600825f0 con 0x560135ca3690 2024-04-25T01:21:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.537+0000 7f506905b740 1 -- 172.21.15.31:0/966093973 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5060081850 con 0x560135ca8800 2024-04-25T01:21:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.537+0000 7f506905b740 10 client.0 _ll_register_callbacks cb 0x7f5058009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:21:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.538+0000 7f505fe00640 1 -- 172.21.15.31:0/966093973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca3690 msgr2=0x560135cb21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.538+0000 7f505fe00640 1 --2- 172.21.15.31:0/966093973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca3690 0x560135cb21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.538+0000 7f505fe00640 1 -- 172.21.15.31:0/966093973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560135ca2ca0 msgr2=0x560135cb1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.538+0000 7f505fe00640 1 --2- 172.21.15.31:0/966093973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560135ca2ca0 0x560135cb1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.538+0000 7f505fe00640 1 -- 172.21.15.31:0/966093973 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560135cb37f0 con 0x560135ca8800 2024-04-25T01:21:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.540+0000 7f505ea00640 1 --2- 172.21.15.31:0/966093973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca3690 0x560135cb21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:21:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.540+0000 7f505fe00640 1 --2- 172.21.15.31:0/966093973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560135ca8800 0x560135cb3200 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f505800ce70 tx=0x7f505800c2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f27826d1129bf643 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.540+0000 7f5048c00640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:21:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.541+0000 7f5048c00640 1 -- 172.21.15.31:0/966093973 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5058034070 con 0x560135ca8800 2024-04-25T01:21:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.542+0000 7f5048c00640 1 -- 172.21.15.31:0/966093973 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f5058025420 con 0x560135ca8800 2024-04-25T01:21:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.542+0000 7f506905b740 1 -- 172.21.15.31:0/966093973 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f505802de40 con 0x560135ca8800 2024-04-25T01:21:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.543+0000 7f5048c00640 1 -- 172.21.15.31:0/966093973 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f505802a020 con 0x560135ca8800 2024-04-25T01:21:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.543+0000 7f506905b740 10 client.25594 fetch_fsmap learned FSMap version 27 2024-04-25T01:21:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.543+0000 7f506905b740 1 -- 172.21.15.31:0/966093973 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560135d30320 con 0x560135ca8800 2024-04-25T01:21:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.544+0000 7f5048c00640 1 -- 172.21.15.31:0/966093973 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f505802de40 con 0x560135ca8800 2024-04-25T01:21:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.547+0000 7f5048c00640 1 -- 172.21.15.31:0/966093973 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7f505804b6c0 con 0x560135ca8800 2024-04-25T01:21:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.547+0000 7f506905b740 10 client.25594 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:21:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.547+0000 7f506905b740 10 client.25594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:21:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.547+0000 7f506905b740 20 client.25594 populate_metadata read hostname 'smithi031' 2024-04-25T01:21:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.547+0000 7f506905b740 1 --2- 172.21.15.31:0/966093973 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560135d30cd0 0x560135d330f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.547+0000 7f506905b740 4 client.25594 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:21:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.547+0000 7f506905b740 1 -- 172.21.15.31:0/966093973 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560135c0d340 con 0x560135d30cd0 2024-04-25T01:21:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.548+0000 7f505f400640 1 --2- 172.21.15.31:0/966093973 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560135d30cd0 0x560135d330f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.549+0000 7f505f400640 1 --2- 172.21.15.31:0/966093973 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560135d30cd0 0x560135d330f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adba750f21cf42ba server_cookie=4bf570958d90603a in_seq=0 out_seq=0 2024-04-25T01:21:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:01.549+0000 7f5048c00640 10 client.25594 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:21:01.807 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:01 smithi031 ceph-mon[28053]: pgmap v543: 97 pgs: 97 active+clean; 13 GiB data, 84 GiB used, 988 GiB / 1.0 TiB avail; 132 KiB/s rd, 132 MiB/s wr, 18.77k op/s 2024-04-25T01:21:01.940 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:01 smithi169 ceph-mon[32759]: pgmap v543: 97 pgs: 97 active+clean; 13 GiB data, 84 GiB used, 988 GiB / 1.0 TiB avail; 132 KiB/s rd, 132 MiB/s wr, 18.77k op/s 2024-04-25T01:21:01.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:01 smithi040 ceph-mon[32518]: pgmap v543: 97 pgs: 97 active+clean; 13 GiB data, 84 GiB used, 988 GiB / 1.0 TiB avail; 132 KiB/s rd, 132 MiB/s wr, 18.77k op/s 2024-04-25T01:21:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.047+0000 7f5048c00640 1 -- 172.21.15.31:0/966093973 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560135c0d340 con 0x560135d30cd0 2024-04-25T01:21:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.047+0000 7f5048c00640 10 client.25594 handle_command_reply: tid=0 2024-04-25T01:21:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.048+0000 7f506905b740 1 client.25594 shutdown 2024-04-25T01:21:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.048+0000 7f506905b740 1 -- 172.21.15.31:0/966093973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560135ca8800 msgr2=0x560135cb3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.048+0000 7f506905b740 1 --2- 172.21.15.31:0/966093973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560135ca8800 0x560135cb3200 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f505800ce70 tx=0x7f505800c2f0 comp rx=0 tx=0).stop 2024-04-25T01:21:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.049+0000 7f506905b740 1 -- 172.21.15.31:0/966093973 shutdown_connections 2024-04-25T01:21:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.049+0000 7f506905b740 1 --2- 172.21.15.31:0/966093973 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560135d30cd0 0x560135d330f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.049+0000 7f5048c00640 0 client.25594 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:21:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.049+0000 7f506905b740 1 --2- 172.21.15.31:0/966093973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560135ca8800 0x560135cb3200 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.049+0000 7f506905b740 1 --2- 172.21.15.31:0/966093973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560135ca3690 0x560135cb21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.049+0000 7f506905b740 1 --2- 172.21.15.31:0/966093973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560135ca2ca0 0x560135cb1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.049+0000 7f506905b740 1 -- 172.21.15.31:0/966093973 >> 172.21.15.31:0/966093973 conn(0x560135c9e310 msgr2=0x560135ca1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.050+0000 7f506905b740 1 -- 172.21.15.31:0/966093973 shutdown_connections 2024-04-25T01:21:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.050+0000 7f506905b740 1 -- 172.21.15.31:0/966093973 wait complete. 2024-04-25T01:21:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.051+0000 7f506905b740 20 client.25594 trim_cache size 0 max 16384 2024-04-25T01:21:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:21:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:21:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:21:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:21:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.054+0000 7f5066200640 1 -- 172.21.15.31:0/671317654 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f50440a2d30 msgr2=0x7f50440a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.054+0000 7f5066200640 1 --2- 172.21.15.31:0/671317654 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f50440a2d30 0x7f50440a51f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x560135ca3ef0 tx=0x7f5058010040 comp rx=0 tx=0).stop 2024-04-25T01:21:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.054+0000 7f5066200640 1 -- 172.21.15.31:0/671317654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5060108fc0 msgr2=0x7f506019e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.054+0000 7f5066200640 1 --2- 172.21.15.31:0/671317654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5060108fc0 0x7f506019e5c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f504c031440 tx=0x7f504c032ce0 comp rx=0 tx=0).stop 2024-04-25T01:21:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.054+0000 7f5066200640 1 -- 172.21.15.31:0/671317654 shutdown_connections 2024-04-25T01:21:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.054+0000 7f5066200640 1 --2- 172.21.15.31:0/671317654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5060109980 0x7f50601a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.054+0000 7f5066200640 1 --2- 172.21.15.31:0/671317654 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f50440a2d30 0x7f50440a51f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.054+0000 7f5066200640 1 --2- 172.21.15.31:0/671317654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5060108fc0 0x7f506019e5c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.054+0000 7f5066200640 1 --2- 172.21.15.31:0/671317654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5060107dd0 0x7f506019e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.054+0000 7f5066200640 1 -- 172.21.15.31:0/671317654 >> 172.21.15.31:0/671317654 conn(0x7f5060076d90 msgr2=0x7f506010c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.055+0000 7f5066200640 1 -- 172.21.15.31:0/671317654 shutdown_connections 2024-04-25T01:21:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:02.055+0000 7f5066200640 1 -- 172.21.15.31:0/671317654 wait complete. 2024-04-25T01:21:03.940 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:03 smithi169 ceph-mon[32759]: pgmap v544: 97 pgs: 97 active+clean; 13 GiB data, 85 GiB used, 988 GiB / 1.0 TiB avail; 108 KiB/s rd, 127 MiB/s wr, 18.23k op/s 2024-04-25T01:21:03.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:03 smithi040 ceph-mon[32518]: pgmap v544: 97 pgs: 97 active+clean; 13 GiB data, 85 GiB used, 988 GiB / 1.0 TiB avail; 108 KiB/s rd, 127 MiB/s wr, 18.23k op/s 2024-04-25T01:21:04.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:03 smithi031 ceph-mon[28053]: pgmap v544: 97 pgs: 97 active+clean; 13 GiB data, 85 GiB used, 988 GiB / 1.0 TiB avail; 108 KiB/s rd, 127 MiB/s wr, 18.23k op/s 2024-04-25T01:21:05.940 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:05 smithi169 ceph-mon[32759]: pgmap v545: 97 pgs: 97 active+clean; 13 GiB data, 85 GiB used, 987 GiB / 1.0 TiB avail; 94 KiB/s rd, 122 MiB/s wr, 17.43k op/s 2024-04-25T01:21:05.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:05 smithi040 ceph-mon[32518]: pgmap v545: 97 pgs: 97 active+clean; 13 GiB data, 85 GiB used, 987 GiB / 1.0 TiB avail; 94 KiB/s rd, 122 MiB/s wr, 17.43k op/s 2024-04-25T01:21:06.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:05 smithi031 ceph-mon[28053]: pgmap v545: 97 pgs: 97 active+clean; 13 GiB data, 85 GiB used, 987 GiB / 1.0 TiB avail; 94 KiB/s rd, 122 MiB/s wr, 17.43k op/s 2024-04-25T01:21:07.940 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:07 smithi169 ceph-mon[32759]: pgmap v546: 97 pgs: 97 active+clean; 13 GiB data, 87 GiB used, 986 GiB / 1.0 TiB avail; 136 KiB/s rd, 144 MiB/s wr, 20.15k op/s 2024-04-25T01:21:07.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:07 smithi040 ceph-mon[32518]: pgmap v546: 97 pgs: 97 active+clean; 13 GiB data, 87 GiB used, 986 GiB / 1.0 TiB avail; 136 KiB/s rd, 144 MiB/s wr, 20.15k op/s 2024-04-25T01:21:08.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:07 smithi031 ceph-mon[28053]: pgmap v546: 97 pgs: 97 active+clean; 13 GiB data, 87 GiB used, 986 GiB / 1.0 TiB avail; 136 KiB/s rd, 144 MiB/s wr, 20.15k op/s 2024-04-25T01:21:09.940 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:09 smithi169 ceph-mon[32759]: pgmap v547: 97 pgs: 97 active+clean; 13 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 117 KiB/s rd, 119 MiB/s wr, 16.40k op/s 2024-04-25T01:21:09.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:09 smithi040 ceph-mon[32518]: pgmap v547: 97 pgs: 97 active+clean; 13 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 117 KiB/s rd, 119 MiB/s wr, 16.40k op/s 2024-04-25T01:21:10.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:09 smithi031 ceph-mon[28053]: pgmap v547: 97 pgs: 97 active+clean; 13 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 117 KiB/s rd, 119 MiB/s wr, 16.40k op/s 2024-04-25T01:21:11.941 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:11 smithi169 ceph-mon[32759]: pgmap v548: 97 pgs: 97 active+clean; 13 GiB data, 87 GiB used, 986 GiB / 1.0 TiB avail; 123 KiB/s rd, 124 MiB/s wr, 16.99k op/s 2024-04-25T01:21:11.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:11 smithi040 ceph-mon[32518]: pgmap v548: 97 pgs: 97 active+clean; 13 GiB data, 87 GiB used, 986 GiB / 1.0 TiB avail; 123 KiB/s rd, 124 MiB/s wr, 16.99k op/s 2024-04-25T01:21:12.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:11 smithi031 ceph-mon[28053]: pgmap v548: 97 pgs: 97 active+clean; 13 GiB data, 87 GiB used, 986 GiB / 1.0 TiB avail; 123 KiB/s rd, 124 MiB/s wr, 16.99k op/s 2024-04-25T01:21:13.940 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:13 smithi169 ceph-mon[32759]: pgmap v549: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 111 KiB/s rd, 118 MiB/s wr, 15.97k op/s 2024-04-25T01:21:13.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:13 smithi040 ceph-mon[32518]: pgmap v549: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 111 KiB/s rd, 118 MiB/s wr, 15.97k op/s 2024-04-25T01:21:14.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:13 smithi031 ceph-mon[28053]: pgmap v549: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 111 KiB/s rd, 118 MiB/s wr, 15.97k op/s 2024-04-25T01:21:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.371+0000 7fca23e00640 1 -- 172.21.15.31:0/1934581598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca1c104970 msgr2=0x7fca1c106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.371+0000 7fca23e00640 1 --2- 172.21.15.31:0/1934581598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca1c104970 0x7fca1c106d60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fca08009900 tx=0x7fca0802a420 comp rx=0 tx=0).stop 2024-04-25T01:21:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.372+0000 7fca23e00640 1 -- 172.21.15.31:0/1934581598 shutdown_connections 2024-04-25T01:21:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.372+0000 7fca23e00640 1 --2- 172.21.15.31:0/1934581598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca1c1072a0 0x7fca1c1096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.372+0000 7fca23e00640 1 --2- 172.21.15.31:0/1934581598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca1c104970 0x7fca1c106d60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.372+0000 7fca23e00640 1 --2- 172.21.15.31:0/1934581598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca1c102040 0x7fca1c104430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.372+0000 7fca23e00640 1 -- 172.21.15.31:0/1934581598 >> 172.21.15.31:0/1934581598 conn(0x7fca1c0fba30 msgr2=0x7fca1c0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.372+0000 7fca23e00640 1 -- 172.21.15.31:0/1934581598 shutdown_connections 2024-04-25T01:21:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.372+0000 7fca23e00640 1 -- 172.21.15.31:0/1934581598 wait complete. 2024-04-25T01:21:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.376+0000 7fca23e00640 1 Processor -- start 2024-04-25T01:21:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.377+0000 7fca23e00640 1 -- start start 2024-04-25T01:21:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.377+0000 7fca23e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca1c102040 0x7fca1c1956e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.377+0000 7fca23e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca1c104970 0x7fca1c195c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.377+0000 7fca23e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca1c1072a0 0x7fca1c1a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.378+0000 7fca21400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca1c102040 0x7fca1c1956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.378+0000 7fca21400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca1c102040 0x7fca1c1956e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:46354/0 (socket says 172.21.15.31:46354) 2024-04-25T01:21:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.379+0000 7fca21e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca1c1072a0 0x7fca1c1a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.379+0000 7fca21e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca1c1072a0 0x7fca1c1a5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:52318/0 (socket says 172.21.15.31:52318) 2024-04-25T01:21:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.379+0000 7fca20a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca1c104970 0x7fca1c195c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.379+0000 7fca23e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c10d300 con 0x7fca1c104970 2024-04-25T01:21:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.379+0000 7fca21400640 1 -- 172.21.15.31:0/2487133107 learned_addr learned my addr 172.21.15.31:0/2487133107 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:21:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.379+0000 7fca23e00640 1 -- 172.21.15.31:0/2487133107 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c10d180 con 0x7fca1c1072a0 2024-04-25T01:21:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.379+0000 7fca23e00640 1 -- 172.21.15.31:0/2487133107 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c10d480 con 0x7fca1c102040 2024-04-25T01:21:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.380+0000 7fca21400640 1 -- 172.21.15.31:0/2487133107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca1c1072a0 msgr2=0x7fca1c1a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.380+0000 7fca21400640 1 --2- 172.21.15.31:0/2487133107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca1c1072a0 0x7fca1c1a5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.380+0000 7fca21400640 1 -- 172.21.15.31:0/2487133107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca1c104970 msgr2=0x7fca1c195c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.380+0000 7fca21400640 1 --2- 172.21.15.31:0/2487133107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca1c104970 0x7fca1c195c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.380+0000 7fca21400640 1 -- 172.21.15.31:0/2487133107 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca1c1a5b70 con 0x7fca1c102040 2024-04-25T01:21:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.381+0000 7fca21400640 1 --2- 172.21.15.31:0/2487133107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca1c102040 0x7fca1c1956e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fca1000c920 tx=0x7fca1000cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.382+0000 7fca1a000640 1 -- 172.21.15.31:0/2487133107 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca10007c70 con 0x7fca1c102040 2024-04-25T01:21:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.382+0000 7fca23e00640 1 -- 172.21.15.31:0/2487133107 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca1c1a5e60 con 0x7fca1c102040 2024-04-25T01:21:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.382+0000 7fca23e00640 1 -- 172.21.15.31:0/2487133107 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca1c1a6350 con 0x7fca1c102040 2024-04-25T01:21:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.385+0000 7fca1a000640 1 -- 172.21.15.31:0/2487133107 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca10007e10 con 0x7fca1c102040 2024-04-25T01:21:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.385+0000 7fca1a000640 1 -- 172.21.15.31:0/2487133107 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca10018890 con 0x7fca1c102040 2024-04-25T01:21:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.392+0000 7fca1a000640 1 -- 172.21.15.31:0/2487133107 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fca10018b30 con 0x7fca1c102040 2024-04-25T01:21:15.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.396+0000 7fca1a000640 1 --2- 172.21.15.31:0/2487133107 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fca040a2c00 0x7fca040a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.397+0000 7fca26d70740 1 Processor -- start 2024-04-25T01:21:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.397+0000 7fca1a000640 1 -- 172.21.15.31:0/2487133107 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fca100d0140 con 0x7fca1c102040 2024-04-25T01:21:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.398+0000 7fca26d70740 1 -- start start 2024-04-25T01:21:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.399+0000 7fca20a00640 1 --2- 172.21.15.31:0/2487133107 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fca040a2c00 0x7fca040a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.400+0000 7fca26d70740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e79a77e50 0x563e79a78270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.400+0000 7fca26d70740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563e79a78840 0x563e79a7d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.400+0000 7fca26d70740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a7d9b0 0x563e79a7fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.401+0000 7fca20a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563e79a78840 0x563e79a7d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.401+0000 7fca21e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a7d9b0 0x563e79a7fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.401+0000 7fca21e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a7d9b0 0x563e79a7fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:52328/0 (socket says 172.21.15.31:52328) 2024-04-25T01:21:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.401+0000 7fca26d70740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563e799daab0 con 0x563e79a77e50 2024-04-25T01:21:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.401+0000 7fca26d70740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563e79983640 con 0x563e79a7d9b0 2024-04-25T01:21:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.401+0000 7fca26d70740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563e799d6a70 con 0x563e79a78840 2024-04-25T01:21:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.401+0000 7fca21e00640 1 -- 172.21.15.31:0/3938969748 learned_addr learned my addr 172.21.15.31:0/3938969748 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:21:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.402+0000 7fca21e00640 1 -- 172.21.15.31:0/3938969748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563e79a78840 msgr2=0x563e79a7d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.402+0000 7fca21e00640 1 --2- 172.21.15.31:0/3938969748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563e79a78840 0x563e79a7d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.402+0000 7fca21e00640 1 -- 172.21.15.31:0/3938969748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e79a77e50 msgr2=0x563e79a78270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.402+0000 7fca21e00640 1 --2- 172.21.15.31:0/3938969748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e79a77e50 0x563e79a78270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.402+0000 7fca21e00640 1 -- 172.21.15.31:0/3938969748 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e796ef210 con 0x563e79a7d9b0 2024-04-25T01:21:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.402+0000 7fca21e00640 1 --2- 172.21.15.31:0/3938969748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a7d9b0 0x563e79a7fe40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fca14008d50 tx=0x7fca14008e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc04b94e0433ecb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.403+0000 7fca20a00640 1 --2- 172.21.15.31:0/2487133107 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fca040a2c00 0x7fca040a50c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fca0802aa10 tx=0x7fca08004490 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.404+0000 7fca0e600640 1 -- 172.21.15.31:0/3938969748 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1400ee50 con 0x563e79a7d9b0 2024-04-25T01:21:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.404+0000 7fca0e600640 1 -- 172.21.15.31:0/3938969748 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca1400ee50 con 0x563e79a7d9b0 2024-04-25T01:21:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.405+0000 7fca26d70740 1 -- 172.21.15.31:0/3938969748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a7d9b0 msgr2=0x563e79a7fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.405+0000 7fca26d70740 1 --2- 172.21.15.31:0/3938969748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a7d9b0 0x563e79a7fe40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fca14008d50 tx=0x7fca14008e40 comp rx=0 tx=0).stop 2024-04-25T01:21:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.405+0000 7fca26d70740 1 -- 172.21.15.31:0/3938969748 shutdown_connections 2024-04-25T01:21:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.405+0000 7fca26d70740 1 --2- 172.21.15.31:0/3938969748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a7d9b0 0x563e79a7fe40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.405+0000 7fca26d70740 1 --2- 172.21.15.31:0/3938969748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563e79a78840 0x563e79a7d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.405+0000 7fca26d70740 1 --2- 172.21.15.31:0/3938969748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e79a77e50 0x563e79a78270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.405+0000 7fca26d70740 1 -- 172.21.15.31:0/3938969748 >> 172.21.15.31:0/3938969748 conn(0x563e79a734c0 msgr2=0x563e79a738c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.407+0000 7fca26d70740 1 -- 172.21.15.31:0/3938969748 shutdown_connections 2024-04-25T01:21:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.407+0000 7fca26d70740 1 -- 172.21.15.31:0/3938969748 wait complete. 2024-04-25T01:21:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.407+0000 7fca26d70740 1 -- start start 2024-04-25T01:21:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.408+0000 7fca26d70740 1 Processor -- start 2024-04-25T01:21:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.410+0000 7fca26d70740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a77e50 0x563e79a820c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.410+0000 7fca26d70740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e79a78840 0x563e79a82600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.410+0000 7fca26d70740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563e79a7d9b0 0x563e79a89640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.411+0000 7fca20a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e79a78840 0x563e79a82600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.411+0000 7fca20a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e79a78840 0x563e79a82600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53816/0 (socket says 172.21.15.31:53816) 2024-04-25T01:21:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.411+0000 7fca21e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563e79a7d9b0 0x563e79a89640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.411+0000 7fca21400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a77e50 0x563e79a820c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.411+0000 7fca21400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a77e50 0x563e79a820c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:52336/0 (socket says 172.21.15.31:52336) 2024-04-25T01:21:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.411+0000 7fca21e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563e79a7d9b0 0x563e79a89640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:46370/0 (socket says 172.21.15.31:46370) 2024-04-25T01:21:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.411+0000 7fca20a00640 1 -- 172.21.15.31:0/2496874318 learned_addr learned my addr 172.21.15.31:0/2496874318 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:21:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.411+0000 7fca26d70740 1 -- 172.21.15.31:0/2496874318 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563e79a80a00 con 0x563e79a78840 2024-04-25T01:21:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.411+0000 7fca26d70740 1 -- 172.21.15.31:0/2496874318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x563e79996990 con 0x563e79a77e50 2024-04-25T01:21:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.411+0000 7fca26d70740 1 -- 172.21.15.31:0/2496874318 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563e79a80b80 con 0x563e79a7d9b0 2024-04-25T01:21:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.412+0000 7fca26d70740 10 client.0 _ll_register_callbacks cb 0x563e79998950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:21:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.412+0000 7fca21e00640 1 -- 172.21.15.31:0/2496874318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a77e50 msgr2=0x563e79a820c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.412+0000 7fca21e00640 1 --2- 172.21.15.31:0/2496874318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a77e50 0x563e79a820c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.412+0000 7fca21e00640 1 -- 172.21.15.31:0/2496874318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e79a78840 msgr2=0x563e79a82600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.412+0000 7fca21e00640 1 --2- 172.21.15.31:0/2496874318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e79a78840 0x563e79a82600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.412+0000 7fca21e00640 1 -- 172.21.15.31:0/2496874318 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e797bb620 con 0x563e79a7d9b0 2024-04-25T01:21:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.412+0000 7fca20a00640 1 --2- 172.21.15.31:0/2496874318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e79a78840 0x563e79a82600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-25T01:21:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.413+0000 7fca21e00640 1 --2- 172.21.15.31:0/2496874318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563e79a7d9b0 0x563e79a89640 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fca14008d20 tx=0x7fca14008a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45a047a8a5013e39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.413+0000 7fca02a00640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:21:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.413+0000 7fca26d70740 1 -- 172.21.15.31:0/2496874318 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca1400ee50 con 0x563e79a7d9b0 2024-04-25T01:21:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.413+0000 7fca02a00640 1 -- 172.21.15.31:0/2496874318 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca14026060 con 0x563e79a7d9b0 2024-04-25T01:21:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.413+0000 7fca02a00640 1 -- 172.21.15.31:0/2496874318 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fca14018680 con 0x563e79a7d9b0 2024-04-25T01:21:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.414+0000 7fca02a00640 1 -- 172.21.15.31:0/2496874318 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fca1400fe10 con 0x563e79a7d9b0 2024-04-25T01:21:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.416+0000 7fca26d70740 10 client.25606 fetch_fsmap learned FSMap version 27 2024-04-25T01:21:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.416+0000 7fca26d70740 1 -- 172.21.15.31:0/2496874318 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({fsmap=27}) -- 0x563e79a8c5e0 con 0x563e79a7d9b0 2024-04-25T01:21:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.416+0000 7fca02a00640 1 -- 172.21.15.31:0/2496874318 <== mon.1 v2:172.21.15.169:3300/0 4 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7fca1400a070 con 0x563e79a7d9b0 2024-04-25T01:21:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.417+0000 7fca26d70740 10 client.25606 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:21:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.417+0000 7fca26d70740 10 client.25606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:21:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.417+0000 7fca26d70740 20 client.25606 populate_metadata read hostname 'smithi031' 2024-04-25T01:21:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.417+0000 7fca26d70740 1 --2- 172.21.15.31:0/2496874318 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x563e79a8d930 0x563e79a8fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.417+0000 7fca26d70740 4 client.25606 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:21:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.417+0000 7fca26d70740 1 -- 172.21.15.31:0/2496874318 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e799e7ff0 con 0x563e79a8d930 2024-04-25T01:21:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.417+0000 7fca21400640 1 --2- 172.21.15.31:0/2496874318 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x563e79a8d930 0x563e79a8fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.418+0000 7fca02a00640 10 client.25606 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:21:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.418+0000 7fca21400640 1 --2- 172.21.15.31:0/2496874318 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x563e79a8d930 0x563e79a8fd20 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d537c3172affd03 server_cookie=3996de195943fe3e in_seq=0 out_seq=0 2024-04-25T01:21:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.419+0000 7fca02a00640 1 -- 172.21.15.31:0/2496874318 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e799e7ff0 con 0x563e79a8d930 2024-04-25T01:21:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.420+0000 7fca02a00640 10 client.25606 handle_command_reply: tid=0 2024-04-25T01:21:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.420+0000 7fca26d70740 1 client.25606 shutdown 2024-04-25T01:21:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.420+0000 7fca26d70740 1 -- 172.21.15.31:0/2496874318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563e79a7d9b0 msgr2=0x563e79a89640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.420+0000 7fca26d70740 1 --2- 172.21.15.31:0/2496874318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563e79a7d9b0 0x563e79a89640 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fca14008d20 tx=0x7fca14008a90 comp rx=0 tx=0).stop 2024-04-25T01:21:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.420+0000 7fca26d70740 1 -- 172.21.15.31:0/2496874318 shutdown_connections 2024-04-25T01:21:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.420+0000 7fca26d70740 1 --2- 172.21.15.31:0/2496874318 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x563e79a8d930 0x563e79a8fd20 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.421+0000 7fca26d70740 1 --2- 172.21.15.31:0/2496874318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563e79a7d9b0 0x563e79a89640 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.421+0000 7fca26d70740 1 --2- 172.21.15.31:0/2496874318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e79a78840 0x563e79a82600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.421+0000 7fca26d70740 1 --2- 172.21.15.31:0/2496874318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a77e50 0x563e79a820c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.421+0000 7fca26d70740 1 -- 172.21.15.31:0/2496874318 >> 172.21.15.31:0/2496874318 conn(0x563e79a734c0 msgr2=0x563e79a72650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.421+0000 7fca02a00640 0 client.25606 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:21:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.421+0000 7fca26d70740 1 -- 172.21.15.31:0/2496874318 shutdown_connections 2024-04-25T01:21:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.422+0000 7fca26d70740 1 -- 172.21.15.31:0/2496874318 wait complete. 2024-04-25T01:21:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.422+0000 7fca26d70740 20 client.25606 trim_cache size 0 max 16384 2024-04-25T01:21:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.495+0000 7fca26d70740 1 Processor -- start 2024-04-25T01:21:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.495+0000 7fca26d70740 1 -- start start 2024-04-25T01:21:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.496+0000 7fca26d70740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e79a77e50 0x563e79a7ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.496+0000 7fca26d70740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a78840 0x563e79a89600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.496+0000 7fca26d70740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563e79a7d9b0 0x563e79a763d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.496+0000 7fca26d70740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1b1800 con 0x563e79a77e50 2024-04-25T01:21:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.496+0000 7fca26d70740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1af070 con 0x563e79a78840 2024-04-25T01:21:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.496+0000 7fca26d70740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1acec0 con 0x563e79a7d9b0 2024-04-25T01:21:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.497+0000 7fca20a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a78840 0x563e79a89600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.497+0000 7fca20a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a78840 0x563e79a89600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:52350/0 (socket says 172.21.15.31:52350) 2024-04-25T01:21:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.497+0000 7fca20a00640 1 -- 172.21.15.31:0/2097749281 learned_addr learned my addr 172.21.15.31:0/2097749281 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:21:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.497+0000 7fca21e00640 1 --2- 172.21.15.31:0/2097749281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563e79a7d9b0 0x563e79a763d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.498+0000 7fca21400640 1 --2- 172.21.15.31:0/2097749281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e79a77e50 0x563e79a7ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.498+0000 7fca21e00640 1 -- 172.21.15.31:0/2097749281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a78840 msgr2=0x563e79a89600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.498+0000 7fca21e00640 1 --2- 172.21.15.31:0/2097749281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a78840 0x563e79a89600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.498+0000 7fca21e00640 1 -- 172.21.15.31:0/2097749281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e79a77e50 msgr2=0x563e79a7ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.499+0000 7fca21e00640 1 --2- 172.21.15.31:0/2097749281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e79a77e50 0x563e79a7ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.499+0000 7fca21e00640 1 -- 172.21.15.31:0/2097749281 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e79a6e9f0 con 0x563e79a7d9b0 2024-04-25T01:21:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.499+0000 7fca21400640 1 --2- 172.21.15.31:0/2097749281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e79a77e50 0x563e79a7ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:21:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.500+0000 7fca21e00640 1 --2- 172.21.15.31:0/2097749281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563e79a7d9b0 0x563e79a763d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fca14008d20 tx=0x7fca1401d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4954dfff21298e87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.501+0000 7fca0dc00640 1 -- 172.21.15.31:0/2097749281 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca14039070 con 0x563e79a7d9b0 2024-04-25T01:21:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.501+0000 7fca0dc00640 1 -- 172.21.15.31:0/2097749281 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca1400ee50 con 0x563e79a7d9b0 2024-04-25T01:21:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.501+0000 7fca0dc00640 1 -- 172.21.15.31:0/2097749281 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca14039070 con 0x563e79a7d9b0 2024-04-25T01:21:15.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.503+0000 7fca26d70740 1 -- 172.21.15.31:0/2097749281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563e79a7d9b0 msgr2=0x563e79a763d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:15.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.503+0000 7fca26d70740 1 --2- 172.21.15.31:0/2097749281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563e79a7d9b0 0x563e79a763d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fca14008d20 tx=0x7fca1401d000 comp rx=0 tx=0).stop 2024-04-25T01:21:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.504+0000 7fca26d70740 1 -- 172.21.15.31:0/2097749281 shutdown_connections 2024-04-25T01:21:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.504+0000 7fca26d70740 1 --2- 172.21.15.31:0/2097749281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563e79a7d9b0 0x563e79a763d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.504+0000 7fca26d70740 1 --2- 172.21.15.31:0/2097749281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a78840 0x563e79a89600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.504+0000 7fca26d70740 1 --2- 172.21.15.31:0/2097749281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e79a77e50 0x563e79a7ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.504+0000 7fca26d70740 1 -- 172.21.15.31:0/2097749281 >> 172.21.15.31:0/2097749281 conn(0x563e79a734c0 msgr2=0x563e79a79990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.504+0000 7fca26d70740 1 -- 172.21.15.31:0/2097749281 shutdown_connections 2024-04-25T01:21:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.504+0000 7fca26d70740 1 -- 172.21.15.31:0/2097749281 wait complete. 2024-04-25T01:21:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.505+0000 7fca26d70740 1 -- start start 2024-04-25T01:21:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.506+0000 7fca26d70740 1 Processor -- start 2024-04-25T01:21:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.507+0000 7fca26d70740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e79a77e50 0x563e79a85440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.507+0000 7fca26d70740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a78840 0x563e79a85980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.507+0000 7fca26d70740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563e79a7d9b0 0x563e79a889a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.508+0000 7fca26d70740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1ad530 con 0x563e79a77e50 2024-04-25T01:21:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.508+0000 7fca26d70740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1ae9a0 con 0x563e79a78840 2024-04-25T01:21:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.508+0000 7fca21400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e79a77e50 0x563e79a85440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.508+0000 7fca26d70740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1adc00 con 0x563e79a7d9b0 2024-04-25T01:21:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.508+0000 7fca26d70740 10 client.0 _ll_register_callbacks cb 0x7fca14008ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:21:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.508+0000 7fca20a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a78840 0x563e79a85980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.508+0000 7fca20a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a78840 0x563e79a85980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:52364/0 (socket says 172.21.15.31:52364) 2024-04-25T01:21:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.508+0000 7fca20a00640 1 -- 172.21.15.31:0/3746865003 learned_addr learned my addr 172.21.15.31:0/3746865003 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:21:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.509+0000 7fca21e00640 1 --2- 172.21.15.31:0/3746865003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563e79a7d9b0 0x563e79a889a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.509+0000 7fca21400640 1 -- 172.21.15.31:0/3746865003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563e79a7d9b0 msgr2=0x563e79a889a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.509+0000 7fca21400640 1 --2- 172.21.15.31:0/3746865003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563e79a7d9b0 0x563e79a889a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.509+0000 7fca21400640 1 -- 172.21.15.31:0/3746865003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a78840 msgr2=0x563e79a85980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.509+0000 7fca21400640 1 --2- 172.21.15.31:0/3746865003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a78840 0x563e79a85980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.509+0000 7fca21400640 1 -- 172.21.15.31:0/3746865003 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e79a88f90 con 0x563e79a77e50 2024-04-25T01:21:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.510+0000 7fca20a00640 1 --2- 172.21.15.31:0/3746865003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a78840 0x563e79a85980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:21:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.510+0000 7fca21400640 1 --2- 172.21.15.31:0/3746865003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e79a77e50 0x563e79a85440 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fca100cdd20 tx=0x7fca1000a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd989455620570e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.510+0000 7fca02a00640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T01:21:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.512+0000 7fca02a00640 1 -- 172.21.15.31:0/3746865003 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca100b9020 con 0x563e79a77e50 2024-04-25T01:21:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.512+0000 7fca02a00640 1 -- 172.21.15.31:0/3746865003 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fca1001fda0 con 0x563e79a77e50 2024-04-25T01:21:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.512+0000 7fca26d70740 1 -- 172.21.15.31:0/3746865003 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca1400ee50 con 0x563e79a77e50 2024-04-25T01:21:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.512+0000 7fca02a00640 1 -- 172.21.15.31:0/3746865003 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fca100a56a0 con 0x563e79a77e50 2024-04-25T01:21:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.514+0000 7fca26d70740 10 client.15351 fetch_fsmap learned FSMap version 27 2024-04-25T01:21:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.514+0000 7fca26d70740 1 -- 172.21.15.31:0/3746865003 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563e79b05460 con 0x563e79a77e50 2024-04-25T01:21:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.514+0000 7fca02a00640 1 -- 172.21.15.31:0/3746865003 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca1400ee50 con 0x563e79a77e50 2024-04-25T01:21:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.514+0000 7fca02a00640 1 -- 172.21.15.31:0/3746865003 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7fca100a5210 con 0x563e79a77e50 2024-04-25T01:21:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.515+0000 7fca26d70740 10 client.15351 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:21:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.515+0000 7fca26d70740 10 client.15351 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:21:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.515+0000 7fca26d70740 20 client.15351 populate_metadata read hostname 'smithi031' 2024-04-25T01:21:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.515+0000 7fca26d70740 1 --2- 172.21.15.31:0/3746865003 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x563e79b05db0 0x563e79b081a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.516+0000 7fca26d70740 4 client.15351 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:21:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.516+0000 7fca26d70740 1 -- 172.21.15.31:0/3746865003 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563e799e2340 con 0x563e79b05db0 2024-04-25T01:21:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.516+0000 7fca20a00640 1 --2- 172.21.15.31:0/3746865003 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x563e79b05db0 0x563e79b081a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.518+0000 7fca20a00640 1 --2- 172.21.15.31:0/3746865003 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x563e79b05db0 0x563e79b081a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8b829c0829d53fe server_cookie=c322bf3e2feb5a92 in_seq=0 out_seq=0 2024-04-25T01:21:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:15.518+0000 7fca02a00640 10 client.15351 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:21:15.940 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:15 smithi169 ceph-mon[32759]: pgmap v550: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 98 KiB/s rd, 113 MiB/s wr, 15.12k op/s 2024-04-25T01:21:15.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:15 smithi040 ceph-mon[32518]: pgmap v550: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 98 KiB/s rd, 113 MiB/s wr, 15.12k op/s 2024-04-25T01:21:16.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:15 smithi031 ceph-mon[28053]: pgmap v550: 97 pgs: 97 active+clean; 13 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 98 KiB/s rd, 113 MiB/s wr, 15.12k op/s 2024-04-25T01:21:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.369+0000 7fca02a00640 1 -- 172.21.15.31:0/3746865003 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563e799e2340 con 0x563e79b05db0 2024-04-25T01:21:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.369+0000 7fca02a00640 10 client.15351 handle_command_reply: tid=0 2024-04-25T01:21:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.369+0000 7fca26d70740 1 client.15351 shutdown 2024-04-25T01:21:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.369+0000 7fca26d70740 1 -- 172.21.15.31:0/3746865003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e79a77e50 msgr2=0x563e79a85440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.369+0000 7fca26d70740 1 --2- 172.21.15.31:0/3746865003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e79a77e50 0x563e79a85440 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fca100cdd20 tx=0x7fca1000a5e0 comp rx=0 tx=0).stop 2024-04-25T01:21:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.370+0000 7fca26d70740 1 -- 172.21.15.31:0/3746865003 shutdown_connections 2024-04-25T01:21:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.370+0000 7fca26d70740 1 --2- 172.21.15.31:0/3746865003 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x563e79b05db0 0x563e79b081a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.370+0000 7fca02a00640 0 client.15351 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:21:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.370+0000 7fca26d70740 1 --2- 172.21.15.31:0/3746865003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563e79a7d9b0 0x563e79a889a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:16.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.370+0000 7fca26d70740 1 --2- 172.21.15.31:0/3746865003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563e79a78840 0x563e79a85980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:16.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.370+0000 7fca26d70740 1 --2- 172.21.15.31:0/3746865003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e79a77e50 0x563e79a85440 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:16.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.370+0000 7fca26d70740 1 -- 172.21.15.31:0/3746865003 >> 172.21.15.31:0/3746865003 conn(0x563e79a734c0 msgr2=0x563e79a763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:16.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.371+0000 7fca26d70740 1 -- 172.21.15.31:0/3746865003 shutdown_connections 2024-04-25T01:21:16.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.371+0000 7fca26d70740 1 -- 172.21.15.31:0/3746865003 wait complete. 2024-04-25T01:21:16.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.371+0000 7fca26d70740 20 client.15351 trim_cache size 0 max 16384 2024-04-25T01:21:16.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:21:16.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:21:16.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:21:16.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:21:16.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.373+0000 7fca23e00640 1 -- 172.21.15.31:0/2487133107 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fca040a2c00 msgr2=0x7fca040a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:16.354 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.373+0000 7fca23e00640 1 --2- 172.21.15.31:0/2487133107 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fca040a2c00 0x7fca040a50c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fca0802aa10 tx=0x7fca08004490 comp rx=0 tx=0).stop 2024-04-25T01:21:16.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.373+0000 7fca23e00640 1 -- 172.21.15.31:0/2487133107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca1c102040 msgr2=0x7fca1c1956e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:16.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.373+0000 7fca23e00640 1 --2- 172.21.15.31:0/2487133107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca1c102040 0x7fca1c1956e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fca1000c920 tx=0x7fca1000cde0 comp rx=0 tx=0).stop 2024-04-25T01:21:16.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.373+0000 7fca23e00640 1 -- 172.21.15.31:0/2487133107 shutdown_connections 2024-04-25T01:21:16.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.373+0000 7fca23e00640 1 --2- 172.21.15.31:0/2487133107 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fca040a2c00 0x7fca040a50c0 secure :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fca0802aa10 tx=0x7fca08004490 comp rx=0 tx=0).stop 2024-04-25T01:21:16.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.373+0000 7fca23e00640 1 --2- 172.21.15.31:0/2487133107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca1c1072a0 0x7fca1c1a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:16.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.373+0000 7fca23e00640 1 --2- 172.21.15.31:0/2487133107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca1c104970 0x7fca1c195c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:16.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.373+0000 7fca23e00640 1 --2- 172.21.15.31:0/2487133107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca1c102040 0x7fca1c1956e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:16.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.373+0000 7fca23e00640 1 -- 172.21.15.31:0/2487133107 >> 172.21.15.31:0/2487133107 conn(0x7fca1c0fba30 msgr2=0x7fca1c101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:16.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.373+0000 7fca23e00640 1 -- 172.21.15.31:0/2487133107 shutdown_connections 2024-04-25T01:21:16.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:16.373+0000 7fca23e00640 1 -- 172.21.15.31:0/2487133107 wait complete. 2024-04-25T01:21:16.940 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:16 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:21:16.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:16 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:21:17.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:16 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:21:17.940 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:17 smithi169 ceph-mon[32759]: pgmap v551: 97 pgs: 97 active+clean; 13 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 109 KiB/s rd, 135 MiB/s wr, 18.16k op/s 2024-04-25T01:21:17.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:17 smithi040 ceph-mon[32518]: pgmap v551: 97 pgs: 97 active+clean; 13 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 109 KiB/s rd, 135 MiB/s wr, 18.16k op/s 2024-04-25T01:21:18.003 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:17 smithi031 ceph-mon[28053]: pgmap v551: 97 pgs: 97 active+clean; 13 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 109 KiB/s rd, 135 MiB/s wr, 18.16k op/s 2024-04-25T01:21:19.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:21:19.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:21:19.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:18 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:21:19.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:18 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:21:19.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:18 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:21:19.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:18 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:21:19.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:18 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:21:19.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:18 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:21:19.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:18 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:21:20.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:19 smithi031 ceph-mon[28053]: pgmap v552: 97 pgs: 97 active+clean; 13 GiB data, 90 GiB used, 983 GiB / 1.0 TiB avail; 72 KiB/s rd, 115 MiB/s wr, 15.68k op/s 2024-04-25T01:21:20.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:19 smithi169 ceph-mon[32759]: pgmap v552: 97 pgs: 97 active+clean; 13 GiB data, 90 GiB used, 983 GiB / 1.0 TiB avail; 72 KiB/s rd, 115 MiB/s wr, 15.68k op/s 2024-04-25T01:21:20.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:19 smithi040 ceph-mon[32518]: pgmap v552: 97 pgs: 97 active+clean; 13 GiB data, 90 GiB used, 983 GiB / 1.0 TiB avail; 72 KiB/s rd, 115 MiB/s wr, 15.68k op/s 2024-04-25T01:21:22.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:21 smithi031 ceph-mon[28053]: pgmap v553: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 63 KiB/s rd, 118 MiB/s wr, 16.37k op/s 2024-04-25T01:21:22.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:21 smithi169 ceph-mon[32759]: pgmap v553: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 63 KiB/s rd, 118 MiB/s wr, 16.37k op/s 2024-04-25T01:21:22.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:21 smithi040 ceph-mon[32518]: pgmap v553: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 63 KiB/s rd, 118 MiB/s wr, 16.37k op/s 2024-04-25T01:21:24.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:23 smithi031 ceph-mon[28053]: pgmap v554: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 34 KiB/s rd, 108 MiB/s wr, 15.28k op/s 2024-04-25T01:21:24.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:23 smithi169 ceph-mon[32759]: pgmap v554: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 34 KiB/s rd, 108 MiB/s wr, 15.28k op/s 2024-04-25T01:21:24.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:23 smithi040 ceph-mon[32518]: pgmap v554: 97 pgs: 97 active+clean; 13 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 34 KiB/s rd, 108 MiB/s wr, 15.28k op/s 2024-04-25T01:21:25.991 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:25 smithi169 ceph-mon[32759]: pgmap v555: 97 pgs: 97 active+clean; 13 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 30 KiB/s rd, 107 MiB/s wr, 15.45k op/s 2024-04-25T01:21:26.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:25 smithi031 ceph-mon[28053]: pgmap v555: 97 pgs: 97 active+clean; 13 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 30 KiB/s rd, 107 MiB/s wr, 15.45k op/s 2024-04-25T01:21:26.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:25 smithi040 ceph-mon[32518]: pgmap v555: 97 pgs: 97 active+clean; 13 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 30 KiB/s rd, 107 MiB/s wr, 15.45k op/s 2024-04-25T01:21:28.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:27 smithi031 ceph-mon[28053]: pgmap v556: 97 pgs: 97 active+clean; 13 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 43 KiB/s rd, 124 MiB/s wr, 17.78k op/s 2024-04-25T01:21:28.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:27 smithi169 ceph-mon[32759]: pgmap v556: 97 pgs: 97 active+clean; 13 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 43 KiB/s rd, 124 MiB/s wr, 17.78k op/s 2024-04-25T01:21:28.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:27 smithi040 ceph-mon[32518]: pgmap v556: 97 pgs: 97 active+clean; 13 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 43 KiB/s rd, 124 MiB/s wr, 17.78k op/s 2024-04-25T01:21:29.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.591+0000 7fec6c200640 1 -- 172.21.15.31:0/931602240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec64108740 msgr2=0x7fec6410ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:29.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.591+0000 7fec6c200640 1 --2- 172.21.15.31:0/931602240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec64108740 0x7fec6410ab70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fec5c009be0 tx=0x7fec5c02a6c0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.591+0000 7fec6c200640 1 -- 172.21.15.31:0/931602240 shutdown_connections 2024-04-25T01:21:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.591+0000 7fec6c200640 1 --2- 172.21.15.31:0/931602240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec6410da60 0x7fec6410dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.591+0000 7fec6c200640 1 --2- 172.21.15.31:0/931602240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6410b0b0 0x7fec6410d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.591+0000 7fec6c200640 1 --2- 172.21.15.31:0/931602240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec64108740 0x7fec6410ab70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.591+0000 7fec6c200640 1 -- 172.21.15.31:0/931602240 >> 172.21.15.31:0/931602240 conn(0x7fec6406e7b0 msgr2=0x7fec64070bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.591+0000 7fec6c200640 1 -- 172.21.15.31:0/931602240 shutdown_connections 2024-04-25T01:21:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.591+0000 7fec6c200640 1 -- 172.21.15.31:0/931602240 wait complete. 2024-04-25T01:21:29.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.593+0000 7fec6c200640 1 Processor -- start 2024-04-25T01:21:29.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.593+0000 7fec6c200640 1 -- start start 2024-04-25T01:21:29.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.593+0000 7fec6c200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec64108740 0x7fec641a6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:29.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.593+0000 7fec6c200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec6410b0b0 0x7fec641a6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:29.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.593+0000 7fec6c200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6410da60 0x7fec641add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:29.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.593+0000 7fec6c200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fec6411c1f0 con 0x7fec64108740 2024-04-25T01:21:29.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.593+0000 7fec6c200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fec6411c070 con 0x7fec6410da60 2024-04-25T01:21:29.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.593+0000 7fec6c200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fec6411c370 con 0x7fec6410b0b0 2024-04-25T01:21:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.594+0000 7fec68e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec6410b0b0 0x7fec641a6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.595+0000 7fec6a200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6410da60 0x7fec641add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.595+0000 7fec6a200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6410da60 0x7fec641add10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:42988/0 (socket says 172.21.15.31:42988) 2024-04-25T01:21:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.595+0000 7fec6a200640 1 -- 172.21.15.31:0/3057109236 learned_addr learned my addr 172.21.15.31:0/3057109236 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:21:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.595+0000 7fec69800640 1 --2- 172.21.15.31:0/3057109236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec64108740 0x7fec641a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.595+0000 7fec6a200640 1 -- 172.21.15.31:0/3057109236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec6410b0b0 msgr2=0x7fec641a6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.595+0000 7fec6a200640 1 --2- 172.21.15.31:0/3057109236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec6410b0b0 0x7fec641a6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.595+0000 7fec6a200640 1 -- 172.21.15.31:0/3057109236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec64108740 msgr2=0x7fec641a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.595+0000 7fec6a200640 1 --2- 172.21.15.31:0/3057109236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec64108740 0x7fec641a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.595+0000 7fec6a200640 1 -- 172.21.15.31:0/3057109236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec641ae380 con 0x7fec6410da60 2024-04-25T01:21:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.596+0000 7fec6a200640 1 --2- 172.21.15.31:0/3057109236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6410da60 0x7fec641add10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fec5800c9b0 tx=0x7fec5800ce70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.596+0000 7fec62000640 1 -- 172.21.15.31:0/3057109236 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58007c70 con 0x7fec6410da60 2024-04-25T01:21:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.597+0000 7fec62000640 1 -- 172.21.15.31:0/3057109236 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec58007e10 con 0x7fec6410da60 2024-04-25T01:21:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.597+0000 7fec62000640 1 -- 172.21.15.31:0/3057109236 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5800f6b0 con 0x7fec6410da60 2024-04-25T01:21:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.597+0000 7fec6c200640 1 -- 172.21.15.31:0/3057109236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec641ae670 con 0x7fec6410da60 2024-04-25T01:21:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.597+0000 7fec6c200640 1 -- 172.21.15.31:0/3057109236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec641aeb70 con 0x7fec6410da60 2024-04-25T01:21:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.601+0000 7fec62000640 1 -- 172.21.15.31:0/3057109236 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fec58003ee0 con 0x7fec6410da60 2024-04-25T01:21:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.604+0000 7fec6ef76740 1 Processor -- start 2024-04-25T01:21:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.605+0000 7fec62000640 1 --2- 172.21.15.31:0/3057109236 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fec4c0a2c30 0x7fec4c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.605+0000 7fec62000640 1 -- 172.21.15.31:0/3057109236 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fec580ceb00 con 0x7fec6410da60 2024-04-25T01:21:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.606+0000 7fec6ef76740 1 -- start start 2024-04-25T01:21:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.606+0000 7fec6ef76740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564399d93ca0 0x564399d940c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.606+0000 7fec6ef76740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564399d94690 0x564399d99190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.606+0000 7fec6ef76740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564399d99800 0x564399d9bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.606+0000 7fec6ef76740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564399cf6ab0 con 0x564399d93ca0 2024-04-25T01:21:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.606+0000 7fec6ef76740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564399c9f640 con 0x564399d99800 2024-04-25T01:21:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.606+0000 7fec6ef76740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564399cf2a70 con 0x564399d94690 2024-04-25T01:21:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.607+0000 7fec69800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564399d93ca0 0x564399d940c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.607+0000 7fec6a200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564399d99800 0x564399d9bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.607+0000 7fec6a200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564399d99800 0x564399d9bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:42998/0 (socket says 172.21.15.31:42998) 2024-04-25T01:21:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.607+0000 7fec6a200640 1 -- 172.21.15.31:0/1446938905 learned_addr learned my addr 172.21.15.31:0/1446938905 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:21:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.608+0000 7fec68e00640 1 --2- 172.21.15.31:0/3057109236 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fec4c0a2c30 0x7fec4c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.608+0000 7fec6a200640 1 -- 172.21.15.31:0/1446938905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564399d94690 msgr2=0x564399d99190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T01:21:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.608+0000 7fec6a200640 1 --2- 172.21.15.31:0/1446938905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564399d94690 0x564399d99190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.608+0000 7fec6a200640 1 -- 172.21.15.31:0/1446938905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564399d93ca0 msgr2=0x564399d940c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.609+0000 7fec6a200640 1 --2- 172.21.15.31:0/1446938905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564399d93ca0 0x564399d940c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.609+0000 7fec6a200640 1 -- 172.21.15.31:0/1446938905 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564399a0b210 con 0x564399d99800 2024-04-25T01:21:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.609+0000 7fec6a200640 1 --2- 172.21.15.31:0/1446938905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564399d99800 0x564399d9bc90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fec580a3b30 tx=0x7fec58005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13659b9a19ff1e10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.610+0000 7fec56a00640 1 -- 172.21.15.31:0/1446938905 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5801d040 con 0x564399d99800 2024-04-25T01:21:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.610+0000 7fec56a00640 1 -- 172.21.15.31:0/1446938905 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec580a4060 con 0x564399d99800 2024-04-25T01:21:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.611+0000 7fec6ef76740 1 -- 172.21.15.31:0/1446938905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564399d99800 msgr2=0x564399d9bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.611+0000 7fec68e00640 1 --2- 172.21.15.31:0/3057109236 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fec4c0a2c30 0x7fec4c0a50f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fec641a7c70 tx=0x7fec50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.611+0000 7fec56a00640 1 -- 172.21.15.31:0/1446938905 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec580041d0 con 0x564399d99800 2024-04-25T01:21:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.611+0000 7fec6ef76740 1 --2- 172.21.15.31:0/1446938905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564399d99800 0x564399d9bc90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fec580a3b30 tx=0x7fec58005800 comp rx=0 tx=0).stop 2024-04-25T01:21:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.612+0000 7fec6ef76740 1 -- 172.21.15.31:0/1446938905 shutdown_connections 2024-04-25T01:21:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.612+0000 7fec6ef76740 1 --2- 172.21.15.31:0/1446938905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564399d99800 0x564399d9bc90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.612+0000 7fec6ef76740 1 --2- 172.21.15.31:0/1446938905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564399d94690 0x564399d99190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.612+0000 7fec6ef76740 1 --2- 172.21.15.31:0/1446938905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564399d93ca0 0x564399d940c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.612+0000 7fec6ef76740 1 -- 172.21.15.31:0/1446938905 >> 172.21.15.31:0/1446938905 conn(0x564399d8f310 msgr2=0x564399d8f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.612+0000 7fec6ef76740 1 -- 172.21.15.31:0/1446938905 shutdown_connections 2024-04-25T01:21:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.612+0000 7fec6ef76740 1 -- 172.21.15.31:0/1446938905 wait complete. 2024-04-25T01:21:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.613+0000 7fec6ef76740 1 -- start start 2024-04-25T01:21:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.613+0000 7fec6ef76740 1 Processor -- start 2024-04-25T01:21:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.614+0000 7fec6ef76740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564399d93ca0 0x564399d9ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.614+0000 7fec6ef76740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564399d94690 0x564399d9e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.614+0000 7fec6ef76740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564399d99800 0x564399da5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.614+0000 7fec6ef76740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564399d9c7a0 con 0x564399d99800 2024-04-25T01:21:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.614+0000 7fec6ef76740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x564399cb2990 con 0x564399d94690 2024-04-25T01:21:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.614+0000 7fec6ef76740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564399d9c920 con 0x564399d93ca0 2024-04-25T01:21:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.614+0000 7fec6ef76740 10 client.0 _ll_register_callbacks cb 0x564399cb4950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:21:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.614+0000 7fec69800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564399d93ca0 0x564399d9ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.614+0000 7fec69800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564399d93ca0 0x564399d9ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:46988/0 (socket says 172.21.15.31:46988) 2024-04-25T01:21:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.614+0000 7fec69800640 1 -- 172.21.15.31:0/1422273792 learned_addr learned my addr 172.21.15.31:0/1422273792 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:21:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.615+0000 7fec68e00640 1 --2- 172.21.15.31:0/1422273792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564399d94690 0x564399d9e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.615+0000 7fec69800640 1 -- 172.21.15.31:0/1422273792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564399d94690 msgr2=0x564399d9e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.615+0000 7fec69800640 1 --2- 172.21.15.31:0/1422273792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564399d94690 0x564399d9e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.615+0000 7fec69800640 1 -- 172.21.15.31:0/1422273792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564399d99800 msgr2=0x564399da5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.615+0000 7fec69800640 1 --2- 172.21.15.31:0/1422273792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564399d99800 0x564399da5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.615+0000 7fec69800640 1 -- 172.21.15.31:0/1422273792 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564399ad7620 con 0x564399d93ca0 2024-04-25T01:21:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.616+0000 7fec4b400640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:21:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.616+0000 7fec69800640 1 --2- 172.21.15.31:0/1422273792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564399d93ca0 0x564399d9ded0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fec5c000f80 tx=0x7fec5c009280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82924cfb9bb144ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.616+0000 7fec4b400640 1 -- 172.21.15.31:0/1422273792 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c042070 con 0x564399d93ca0 2024-04-25T01:21:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.616+0000 7fec4b400640 1 -- 172.21.15.31:0/1422273792 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fec5c0305b0 con 0x564399d93ca0 2024-04-25T01:21:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.616+0000 7fec6ef76740 1 -- 172.21.15.31:0/1422273792 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec580a4060 con 0x564399d93ca0 2024-04-25T01:21:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.617+0000 7fec4b400640 1 -- 172.21.15.31:0/1422273792 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fec5c04c050 con 0x564399d93ca0 2024-04-25T01:21:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.617+0000 7fec6ef76740 10 client.25621 fetch_fsmap learned FSMap version 27 2024-04-25T01:21:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.617+0000 7fec6ef76740 1 -- 172.21.15.31:0/1422273792 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564399da8290 con 0x564399d93ca0 2024-04-25T01:21:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.617+0000 7fec4b400640 1 -- 172.21.15.31:0/1422273792 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec5c003e50 con 0x564399d93ca0 2024-04-25T01:21:29.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.619+0000 7fec4b400640 1 -- 172.21.15.31:0/1422273792 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7fec5c03c040 con 0x564399d93ca0 2024-04-25T01:21:29.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.619+0000 7fec6ef76740 10 client.25621 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:21:29.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.619+0000 7fec6ef76740 10 client.25621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:21:29.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.619+0000 7fec6ef76740 20 client.25621 populate_metadata read hostname 'smithi031' 2024-04-25T01:21:29.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.619+0000 7fec6ef76740 1 --2- 172.21.15.31:0/1422273792 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x564399da9e60 0x564399dac300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:29.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.619+0000 7fec6ef76740 4 client.25621 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:21:29.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.619+0000 7fec6ef76740 1 -- 172.21.15.31:0/1422273792 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fec580a4060 con 0x564399da9e60 2024-04-25T01:21:29.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.619+0000 7fec68e00640 1 --2- 172.21.15.31:0/1422273792 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x564399da9e60 0x564399dac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.620+0000 7fec4b400640 10 client.25621 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:21:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.620+0000 7fec68e00640 1 --2- 172.21.15.31:0/1422273792 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x564399da9e60 0x564399dac300 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e99436dedcefbb31 server_cookie=fe2745a99003f7fa in_seq=0 out_seq=0 2024-04-25T01:21:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.621+0000 7fec4b400640 1 -- 172.21.15.31:0/1422273792 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fec580a4060 con 0x564399da9e60 2024-04-25T01:21:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.621+0000 7fec4b400640 10 client.25621 handle_command_reply: tid=0 2024-04-25T01:21:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.621+0000 7fec6ef76740 1 client.25621 shutdown 2024-04-25T01:21:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.621+0000 7fec6ef76740 1 -- 172.21.15.31:0/1422273792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564399d93ca0 msgr2=0x564399d9ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.621+0000 7fec6ef76740 1 --2- 172.21.15.31:0/1422273792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564399d93ca0 0x564399d9ded0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fec5c000f80 tx=0x7fec5c009280 comp rx=0 tx=0).stop 2024-04-25T01:21:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.621+0000 7fec6ef76740 1 -- 172.21.15.31:0/1422273792 shutdown_connections 2024-04-25T01:21:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.621+0000 7fec6ef76740 1 --2- 172.21.15.31:0/1422273792 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x564399da9e60 0x564399dac300 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.622+0000 7fec4b400640 0 client.25621 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:21:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.622+0000 7fec6ef76740 1 --2- 172.21.15.31:0/1422273792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564399d99800 0x564399da5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.622+0000 7fec6ef76740 1 --2- 172.21.15.31:0/1422273792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564399d94690 0x564399d9e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.622+0000 7fec6ef76740 1 --2- 172.21.15.31:0/1422273792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564399d93ca0 0x564399d9ded0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.622+0000 7fec6ef76740 1 -- 172.21.15.31:0/1422273792 >> 172.21.15.31:0/1422273792 conn(0x564399d8f310 msgr2=0x564399d8e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.622+0000 7fec6ef76740 1 -- 172.21.15.31:0/1422273792 shutdown_connections 2024-04-25T01:21:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.622+0000 7fec6ef76740 1 -- 172.21.15.31:0/1422273792 wait complete. 2024-04-25T01:21:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.622+0000 7fec6ef76740 20 client.25621 trim_cache size 0 max 16384 2024-04-25T01:21:29.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.658+0000 7fec6ef76740 1 Processor -- start 2024-04-25T01:21:29.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.658+0000 7fec6ef76740 1 -- start start 2024-04-25T01:21:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.659+0000 7fec6ef76740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564399d93ca0 0x564399d98ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.659+0000 7fec6ef76740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564399d94690 0x564399d8e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.659+0000 7fec6ef76740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564399d99800 0x564399d9d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.659+0000 7fec6ef76740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fec641ba2d0 con 0x564399d94690 2024-04-25T01:21:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.659+0000 7fec6ef76740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fec641b7b40 con 0x564399d99800 2024-04-25T01:21:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.659+0000 7fec6ef76740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fec641b5950 con 0x564399d93ca0 2024-04-25T01:21:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.659+0000 7fec69800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564399d93ca0 0x564399d98ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.659+0000 7fec69800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564399d93ca0 0x564399d98ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:46992/0 (socket says 172.21.15.31:46992) 2024-04-25T01:21:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.659+0000 7fec69800640 1 -- 172.21.15.31:0/571519467 learned_addr learned my addr 172.21.15.31:0/571519467 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:21:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.660+0000 7fec69800640 1 -- 172.21.15.31:0/571519467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564399d99800 msgr2=0x564399d9d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.660+0000 7fec69800640 1 --2- 172.21.15.31:0/571519467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564399d99800 0x564399d9d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.660+0000 7fec69800640 1 -- 172.21.15.31:0/571519467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564399d94690 msgr2=0x564399d8e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.660+0000 7fec69800640 1 --2- 172.21.15.31:0/571519467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564399d94690 0x564399d8e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.660+0000 7fec69800640 1 -- 172.21.15.31:0/571519467 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564399d98b40 con 0x564399d93ca0 2024-04-25T01:21:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.660+0000 7fec69800640 1 --2- 172.21.15.31:0/571519467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564399d93ca0 0x564399d98ec0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fec5c02acb0 tx=0x7fec5c0045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a66e4c138cd2b00b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.661+0000 7fec56000640 1 -- 172.21.15.31:0/571519467 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c04c050 con 0x564399d93ca0 2024-04-25T01:21:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.661+0000 7fec56000640 1 -- 172.21.15.31:0/571519467 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec5c003e50 con 0x564399d93ca0 2024-04-25T01:21:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.661+0000 7fec56000640 1 -- 172.21.15.31:0/571519467 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c042070 con 0x564399d93ca0 2024-04-25T01:21:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.665+0000 7fec6ef76740 1 -- 172.21.15.31:0/571519467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564399d93ca0 msgr2=0x564399d98ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.665+0000 7fec6ef76740 1 --2- 172.21.15.31:0/571519467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564399d93ca0 0x564399d98ec0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fec5c02acb0 tx=0x7fec5c0045b0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.665+0000 7fec6ef76740 1 -- 172.21.15.31:0/571519467 shutdown_connections 2024-04-25T01:21:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.665+0000 7fec6ef76740 1 --2- 172.21.15.31:0/571519467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564399d99800 0x564399d9d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.665+0000 7fec6ef76740 1 --2- 172.21.15.31:0/571519467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564399d94690 0x564399d8e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.665+0000 7fec6ef76740 1 --2- 172.21.15.31:0/571519467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564399d93ca0 0x564399d98ec0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.665+0000 7fec6ef76740 1 -- 172.21.15.31:0/571519467 >> 172.21.15.31:0/571519467 conn(0x564399d8f310 msgr2=0x564399d89fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.666+0000 7fec6ef76740 1 -- 172.21.15.31:0/571519467 shutdown_connections 2024-04-25T01:21:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.666+0000 7fec6ef76740 1 -- 172.21.15.31:0/571519467 wait complete. 2024-04-25T01:21:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.667+0000 7fec6ef76740 1 -- start start 2024-04-25T01:21:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.667+0000 7fec6ef76740 1 Processor -- start 2024-04-25T01:21:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.668+0000 7fec6ef76740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564399d93ca0 0x564399da2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.668+0000 7fec6ef76740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564399d94690 0x564399da2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.669+0000 7fec6ef76740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564399d99800 0x564399da3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.669+0000 7fec6ef76740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fec641b6000 con 0x564399d94690 2024-04-25T01:21:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.669+0000 7fec6ef76740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fec641b7470 con 0x564399d99800 2024-04-25T01:21:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.669+0000 7fec6ef76740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fec641b66d0 con 0x564399d93ca0 2024-04-25T01:21:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.669+0000 7fec69800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564399d93ca0 0x564399da2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.669+0000 7fec69800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564399d93ca0 0x564399da2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:47008/0 (socket says 172.21.15.31:47008) 2024-04-25T01:21:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.669+0000 7fec69800640 1 -- 172.21.15.31:0/3204170195 learned_addr learned my addr 172.21.15.31:0/3204170195 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:21:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.669+0000 7fec6ef76740 10 client.0 _ll_register_callbacks cb 0x7fec5c034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:21:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.669+0000 7fec6a200640 1 --2- 172.21.15.31:0/3204170195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564399d99800 0x564399da3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.670+0000 7fec69800640 1 -- 172.21.15.31:0/3204170195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564399d99800 msgr2=0x564399da3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.670+0000 7fec69800640 1 --2- 172.21.15.31:0/3204170195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564399d99800 0x564399da3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.670+0000 7fec69800640 1 -- 172.21.15.31:0/3204170195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564399d94690 msgr2=0x564399da2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.670+0000 7fec69800640 1 --2- 172.21.15.31:0/3204170195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564399d94690 0x564399da2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.670+0000 7fec69800640 1 -- 172.21.15.31:0/3204170195 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564399da42c0 con 0x564399d93ca0 2024-04-25T01:21:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.670+0000 7fec69800640 1 --2- 172.21.15.31:0/3204170195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564399d93ca0 0x564399da2780 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fec5c03c6d0 tx=0x7fec5c009ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31f57fa39148464e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.670+0000 7fec4b400640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:21:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.670+0000 7fec4b400640 1 -- 172.21.15.31:0/3204170195 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c04c050 con 0x564399d93ca0 2024-04-25T01:21:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.671+0000 7fec4b400640 1 -- 172.21.15.31:0/3204170195 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fec5c0305b0 con 0x564399d93ca0 2024-04-25T01:21:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.671+0000 7fec6ef76740 1 -- 172.21.15.31:0/3204170195 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec5c003e50 con 0x564399d93ca0 2024-04-25T01:21:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.671+0000 7fec4b400640 1 -- 172.21.15.31:0/3204170195 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fec5c042040 con 0x564399d93ca0 2024-04-25T01:21:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.672+0000 7fec6ef76740 10 client.25627 fetch_fsmap learned FSMap version 27 2024-04-25T01:21:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.672+0000 7fec6ef76740 1 -- 172.21.15.31:0/3204170195 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564399e10480 con 0x564399d93ca0 2024-04-25T01:21:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.672+0000 7fec4b400640 1 -- 172.21.15.31:0/3204170195 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec5c003e50 con 0x564399d93ca0 2024-04-25T01:21:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.672+0000 7fec4b400640 1 -- 172.21.15.31:0/3204170195 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7fec5c03c040 con 0x564399d93ca0 2024-04-25T01:21:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.673+0000 7fec6ef76740 10 client.25627 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:21:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.673+0000 7fec6ef76740 10 client.25627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:21:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.673+0000 7fec6ef76740 20 client.25627 populate_metadata read hostname 'smithi031' 2024-04-25T01:21:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.673+0000 7fec6ef76740 1 --2- 172.21.15.31:0/3204170195 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x564399da9e60 0x564399e12c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.673+0000 7fec6ef76740 4 client.25627 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:21:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.673+0000 7fec6ef76740 1 -- 172.21.15.31:0/3204170195 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564399d03ff0 con 0x564399da9e60 2024-04-25T01:21:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.674+0000 7fec68e00640 1 --2- 172.21.15.31:0/3204170195 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x564399da9e60 0x564399e12c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.674+0000 7fec68e00640 1 --2- 172.21.15.31:0/3204170195 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x564399da9e60 0x564399e12c30 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3159e42d43bff393 server_cookie=329aa360424e529a in_seq=0 out_seq=0 2024-04-25T01:21:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.675+0000 7fec4b400640 10 client.25627 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:21:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.714+0000 7fec4b400640 1 -- 172.21.15.31:0/3204170195 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564399d03ff0 con 0x564399da9e60 2024-04-25T01:21:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.714+0000 7fec4b400640 10 client.25627 handle_command_reply: tid=0 2024-04-25T01:21:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.714+0000 7fec6ef76740 1 client.25627 shutdown 2024-04-25T01:21:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.715+0000 7fec6ef76740 1 -- 172.21.15.31:0/3204170195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564399d93ca0 msgr2=0x564399da2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.715+0000 7fec6ef76740 1 --2- 172.21.15.31:0/3204170195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564399d93ca0 0x564399da2780 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fec5c03c6d0 tx=0x7fec5c009ad0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.715+0000 7fec6ef76740 1 -- 172.21.15.31:0/3204170195 shutdown_connections 2024-04-25T01:21:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.715+0000 7fec6ef76740 1 --2- 172.21.15.31:0/3204170195 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x564399da9e60 0x564399e12c30 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.715+0000 7fec6ef76740 1 --2- 172.21.15.31:0/3204170195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564399d99800 0x564399da3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.715+0000 7fec6ef76740 1 --2- 172.21.15.31:0/3204170195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564399d94690 0x564399da2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.715+0000 7fec6ef76740 1 --2- 172.21.15.31:0/3204170195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564399d93ca0 0x564399da2780 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.715+0000 7fec6ef76740 1 -- 172.21.15.31:0/3204170195 >> 172.21.15.31:0/3204170195 conn(0x564399d8f310 msgr2=0x564399d8dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.715+0000 7fec4b400640 0 client.25627 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:21:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.716+0000 7fec6ef76740 1 -- 172.21.15.31:0/3204170195 shutdown_connections 2024-04-25T01:21:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.719+0000 7fec6ef76740 1 -- 172.21.15.31:0/3204170195 wait complete. 2024-04-25T01:21:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.719+0000 7fec6ef76740 20 client.25627 trim_cache size 0 max 16384 2024-04-25T01:21:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:21:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:21:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:21:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:21:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.720+0000 7fec6c200640 1 -- 172.21.15.31:0/3057109236 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fec4c0a2c30 msgr2=0x7fec4c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.720+0000 7fec6c200640 1 --2- 172.21.15.31:0/3057109236 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fec4c0a2c30 0x7fec4c0a50f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fec641a7c70 tx=0x7fec50009260 comp rx=0 tx=0).stop 2024-04-25T01:21:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.721+0000 7fec6c200640 1 -- 172.21.15.31:0/3057109236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6410da60 msgr2=0x7fec641add10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.721+0000 7fec6c200640 1 --2- 172.21.15.31:0/3057109236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6410da60 0x7fec641add10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fec5800c9b0 tx=0x7fec5800ce70 comp rx=0 tx=0).stop 2024-04-25T01:21:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.721+0000 7fec6c200640 1 -- 172.21.15.31:0/3057109236 shutdown_connections 2024-04-25T01:21:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.721+0000 7fec6c200640 1 --2- 172.21.15.31:0/3057109236 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fec4c0a2c30 0x7fec4c0a50f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.721+0000 7fec6c200640 1 --2- 172.21.15.31:0/3057109236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec6410da60 0x7fec641add10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.721+0000 7fec6c200640 1 --2- 172.21.15.31:0/3057109236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec6410b0b0 0x7fec641a6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.721+0000 7fec6c200640 1 --2- 172.21.15.31:0/3057109236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec64108740 0x7fec641a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.721+0000 7fec6c200640 1 -- 172.21.15.31:0/3057109236 >> 172.21.15.31:0/3057109236 conn(0x7fec6406e7b0 msgr2=0x7fec6410d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.721+0000 7fec6c200640 1 -- 172.21.15.31:0/3057109236 shutdown_connections 2024-04-25T01:21:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:29.721+0000 7fec6c200640 1 -- 172.21.15.31:0/3057109236 wait complete. 2024-04-25T01:21:30.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:29 smithi031 ceph-mon[28053]: pgmap v557: 97 pgs: 97 active+clean; 13 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 35 KiB/s rd, 113 MiB/s wr, 16.26k op/s 2024-04-25T01:21:30.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:29 smithi169 ceph-mon[32759]: pgmap v557: 97 pgs: 97 active+clean; 13 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 35 KiB/s rd, 113 MiB/s wr, 16.26k op/s 2024-04-25T01:21:30.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:29 smithi040 ceph-mon[32518]: pgmap v557: 97 pgs: 97 active+clean; 13 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 35 KiB/s rd, 113 MiB/s wr, 16.26k op/s 2024-04-25T01:21:32.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:31 smithi169 ceph-mon[32759]: pgmap v558: 97 pgs: 97 active+clean; 13 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 42 KiB/s rd, 115 MiB/s wr, 16.65k op/s 2024-04-25T01:21:32.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:31 smithi040 ceph-mon[32518]: pgmap v558: 97 pgs: 97 active+clean; 13 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 42 KiB/s rd, 115 MiB/s wr, 16.65k op/s 2024-04-25T01:21:32.252 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:31 smithi031 ceph-mon[28053]: pgmap v558: 97 pgs: 97 active+clean; 13 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 42 KiB/s rd, 115 MiB/s wr, 16.65k op/s 2024-04-25T01:21:34.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:33 smithi169 ceph-mon[32759]: pgmap v559: 97 pgs: 97 active+clean; 13 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 35 KiB/s rd, 115 MiB/s wr, 16.71k op/s 2024-04-25T01:21:34.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:33 smithi040 ceph-mon[32518]: pgmap v559: 97 pgs: 97 active+clean; 13 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 35 KiB/s rd, 115 MiB/s wr, 16.71k op/s 2024-04-25T01:21:34.252 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:33 smithi031 ceph-mon[28053]: pgmap v559: 97 pgs: 97 active+clean; 13 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 35 KiB/s rd, 115 MiB/s wr, 16.71k op/s 2024-04-25T01:21:36.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:35 smithi169 ceph-mon[32759]: pgmap v560: 97 pgs: 97 active+clean; 13 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 43 KiB/s rd, 115 MiB/s wr, 17.02k op/s 2024-04-25T01:21:36.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:35 smithi040 ceph-mon[32518]: pgmap v560: 97 pgs: 97 active+clean; 13 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 43 KiB/s rd, 115 MiB/s wr, 17.02k op/s 2024-04-25T01:21:36.251 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:35 smithi031 ceph-mon[28053]: pgmap v560: 97 pgs: 97 active+clean; 13 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 43 KiB/s rd, 115 MiB/s wr, 17.02k op/s 2024-04-25T01:21:38.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:37 smithi169 ceph-mon[32759]: pgmap v561: 97 pgs: 97 active+clean; 13 GiB data, 93 GiB used, 979 GiB / 1.0 TiB avail; 62 KiB/s rd, 128 MiB/s wr, 19.12k op/s 2024-04-25T01:21:38.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:37 smithi040 ceph-mon[32518]: pgmap v561: 97 pgs: 97 active+clean; 13 GiB data, 93 GiB used, 979 GiB / 1.0 TiB avail; 62 KiB/s rd, 128 MiB/s wr, 19.12k op/s 2024-04-25T01:21:38.251 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:37 smithi031 ceph-mon[28053]: pgmap v561: 97 pgs: 97 active+clean; 13 GiB data, 93 GiB used, 979 GiB / 1.0 TiB avail; 62 KiB/s rd, 128 MiB/s wr, 19.12k op/s 2024-04-25T01:21:40.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:39 smithi169 ceph-mon[32759]: pgmap v562: 97 pgs: 97 active+clean; 13 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 73 KiB/s rd, 110 MiB/s wr, 16.69k op/s 2024-04-25T01:21:40.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:39 smithi040 ceph-mon[32518]: pgmap v562: 97 pgs: 97 active+clean; 13 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 73 KiB/s rd, 110 MiB/s wr, 16.69k op/s 2024-04-25T01:21:40.253 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:39 smithi031 ceph-mon[28053]: pgmap v562: 97 pgs: 97 active+clean; 13 GiB data, 94 GiB used, 979 GiB / 1.0 TiB avail; 73 KiB/s rd, 110 MiB/s wr, 16.69k op/s 2024-04-25T01:21:42.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:41 smithi169 ceph-mon[32759]: pgmap v563: 97 pgs: 97 active+clean; 13 GiB data, 95 GiB used, 978 GiB / 1.0 TiB avail; 120 KiB/s rd, 113 MiB/s wr, 17.40k op/s 2024-04-25T01:21:42.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:41 smithi040 ceph-mon[32518]: pgmap v563: 97 pgs: 97 active+clean; 13 GiB data, 95 GiB used, 978 GiB / 1.0 TiB avail; 120 KiB/s rd, 113 MiB/s wr, 17.40k op/s 2024-04-25T01:21:42.251 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:41 smithi031 ceph-mon[28053]: pgmap v563: 97 pgs: 97 active+clean; 13 GiB data, 95 GiB used, 978 GiB / 1.0 TiB avail; 120 KiB/s rd, 113 MiB/s wr, 17.40k op/s 2024-04-25T01:21:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.941+0000 7f8252600640 1 -- 172.21.15.31:0/266562593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f824c10dc70 msgr2=0x7f824c110130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.941+0000 7f8252600640 1 --2- 172.21.15.31:0/266562593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f824c10dc70 0x7f824c110130 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f824400cc90 tx=0x7f824400a320 comp rx=0 tx=0).stop 2024-04-25T01:21:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.942+0000 7f8252600640 1 -- 172.21.15.31:0/266562593 shutdown_connections 2024-04-25T01:21:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.942+0000 7f8252600640 1 --2- 172.21.15.31:0/266562593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f824c10dc70 0x7f824c110130 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.942+0000 7f8252600640 1 --2- 172.21.15.31:0/266562593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f824c10b190 0x7f824c10d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.942+0000 7f8252600640 1 --2- 172.21.15.31:0/266562593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f824c1087d0 0x7f824c10abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.942+0000 7f8252600640 1 -- 172.21.15.31:0/266562593 >> 172.21.15.31:0/266562593 conn(0x7f824c06e7b0 msgr2=0x7f824c070bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.942+0000 7f8252600640 1 -- 172.21.15.31:0/266562593 shutdown_connections 2024-04-25T01:21:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.942+0000 7f8252600640 1 -- 172.21.15.31:0/266562593 wait complete. 2024-04-25T01:21:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.945+0000 7f8252600640 1 Processor -- start 2024-04-25T01:21:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.945+0000 7f8252600640 1 -- start start 2024-04-25T01:21:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.945+0000 7f8252600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f824c1087d0 0x7f824c1a6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.946+0000 7f8252600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f824c10b190 0x7f824c1a6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.946+0000 7f8252600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f824c10dc70 0x7f824c1adc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.946+0000 7f824a000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f824c10b190 0x7f824c1a6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.947+0000 7f824a000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f824c10b190 0x7f824c1a6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:40886/0 (socket says 172.21.15.31:40886) 2024-04-25T01:21:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.947+0000 7f824a000640 1 -- 172.21.15.31:0/2509250064 learned_addr learned my addr 172.21.15.31:0/2509250064 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:21:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.947+0000 7f8252600640 1 -- 172.21.15.31:0/2509250064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1138d0 con 0x7f824c1087d0 2024-04-25T01:21:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.947+0000 7f8252600640 1 -- 172.21.15.31:0/2509250064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f824c113750 con 0x7f824c10b190 2024-04-25T01:21:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.947+0000 7f8252600640 1 -- 172.21.15.31:0/2509250064 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f824c113a50 con 0x7f824c10dc70 2024-04-25T01:21:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.947+0000 7f824b400640 1 --2- 172.21.15.31:0/2509250064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f824c10dc70 0x7f824c1adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.947+0000 7f824a000640 1 -- 172.21.15.31:0/2509250064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f824c10dc70 msgr2=0x7f824c1adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.947+0000 7f824a000640 1 --2- 172.21.15.31:0/2509250064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f824c10dc70 0x7f824c1adc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.947+0000 7f824a000640 1 -- 172.21.15.31:0/2509250064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f824c1087d0 msgr2=0x7f824c1a6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.947+0000 7f824a000640 1 --2- 172.21.15.31:0/2509250064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f824c1087d0 0x7f824c1a6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.947+0000 7f824a000640 1 -- 172.21.15.31:0/2509250064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824c1ae270 con 0x7f824c10b190 2024-04-25T01:21:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.948+0000 7f824a000640 1 --2- 172.21.15.31:0/2509250064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f824c10b190 0x7f824c1a6b90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f82380078d0 tx=0x7f8238007d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.948+0000 7f823fe00640 1 -- 172.21.15.31:0/2509250064 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823800dab0 con 0x7f824c10b190 2024-04-25T01:21:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.948+0000 7f8252600640 1 -- 172.21.15.31:0/2509250064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f824c1ae560 con 0x7f824c10b190 2024-04-25T01:21:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.948+0000 7f8252600640 1 -- 172.21.15.31:0/2509250064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f824c1aeb80 con 0x7f824c10b190 2024-04-25T01:21:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.949+0000 7f823fe00640 1 -- 172.21.15.31:0/2509250064 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f823800dc50 con 0x7f824c10b190 2024-04-25T01:21:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.949+0000 7f823fe00640 1 -- 172.21.15.31:0/2509250064 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823800dab0 con 0x7f824c10b190 2024-04-25T01:21:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.957+0000 7f823fe00640 1 -- 172.21.15.31:0/2509250064 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f82380173f0 con 0x7f824c10b190 2024-04-25T01:21:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.959+0000 7f82555b7740 1 Processor -- start 2024-04-25T01:21:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.959+0000 7f82555b7740 1 -- start start 2024-04-25T01:21:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.960+0000 7f82555b7740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556cc03ae50 0x5556cc03b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.960+0000 7f82555b7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556cc03b840 0x5556cc040340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.960+0000 7f82555b7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556cc0409b0 0x5556cc042e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.960+0000 7f82555b7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5556cbf9dab0 con 0x5556cc03b840 2024-04-25T01:21:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.960+0000 7f82555b7740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5556cbf46640 con 0x5556cc03ae50 2024-04-25T01:21:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.960+0000 7f82555b7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5556cbf99a70 con 0x5556cc0409b0 2024-04-25T01:21:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.961+0000 7f824b400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556cc03ae50 0x5556cc03b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.961+0000 7f824b400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556cc03ae50 0x5556cc03b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:40900/0 (socket says 172.21.15.31:40900) 2024-04-25T01:21:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.961+0000 7f824b400640 1 -- 172.21.15.31:0/2931232344 learned_addr learned my addr 172.21.15.31:0/2931232344 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:21:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.962+0000 7f824a000640 1 --2- 172.21.15.31:0/2931232344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556cc0409b0 0x5556cc042e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.962+0000 7f824b400640 1 -- 172.21.15.31:0/2931232344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556cc0409b0 msgr2=0x5556cc042e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.962+0000 7f824b400640 1 --2- 172.21.15.31:0/2931232344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556cc0409b0 0x5556cc042e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.962+0000 7f824b400640 1 -- 172.21.15.31:0/2931232344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556cc03b840 msgr2=0x5556cc040340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.962+0000 7f824b400640 1 --2- 172.21.15.31:0/2931232344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556cc03b840 0x5556cc040340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.962+0000 7f824b400640 1 -- 172.21.15.31:0/2931232344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556cbcb2210 con 0x5556cc03ae50 2024-04-25T01:21:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.963+0000 7f823fe00640 1 --2- 172.21.15.31:0/2509250064 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f82340a2f40 0x7f82340a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.964+0000 7f823fe00640 1 -- 172.21.15.31:0/2509250064 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f82380ce740 con 0x7f824c10b190 2024-04-25T01:21:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.964+0000 7f824aa00640 1 --2- 172.21.15.31:0/2509250064 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f82340a2f40 0x7f82340a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.965+0000 7f824aa00640 1 --2- 172.21.15.31:0/2509250064 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f82340a2f40 0x7f82340a5400 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x5556cc03bc40 tx=0x7f8240006d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.966+0000 7f824b400640 1 --2- 172.21.15.31:0/2931232344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556cc03ae50 0x5556cc03b270 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f824400cc60 tx=0x7f8244001400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c51bea4442f5de54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.966+0000 7f823cc00640 1 -- 172.21.15.31:0/2931232344 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824400c270 con 0x5556cc03ae50 2024-04-25T01:21:42.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.967+0000 7f823cc00640 1 -- 172.21.15.31:0/2931232344 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8244041070 con 0x5556cc03ae50 2024-04-25T01:21:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.967+0000 7f823cc00640 1 -- 172.21.15.31:0/2931232344 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8244033a10 con 0x5556cc03ae50 2024-04-25T01:21:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.967+0000 7f82555b7740 1 -- 172.21.15.31:0/2931232344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556cc03ae50 msgr2=0x5556cc03b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.967+0000 7f82555b7740 1 --2- 172.21.15.31:0/2931232344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556cc03ae50 0x5556cc03b270 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f824400cc60 tx=0x7f8244001400 comp rx=0 tx=0).stop 2024-04-25T01:21:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.968+0000 7f82555b7740 1 -- 172.21.15.31:0/2931232344 shutdown_connections 2024-04-25T01:21:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.968+0000 7f82555b7740 1 --2- 172.21.15.31:0/2931232344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556cc0409b0 0x5556cc042e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.968+0000 7f82555b7740 1 --2- 172.21.15.31:0/2931232344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556cc03b840 0x5556cc040340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.968+0000 7f82555b7740 1 --2- 172.21.15.31:0/2931232344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556cc03ae50 0x5556cc03b270 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.968+0000 7f82555b7740 1 -- 172.21.15.31:0/2931232344 >> 172.21.15.31:0/2931232344 conn(0x5556cc0364c0 msgr2=0x5556cc0368c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.968+0000 7f82555b7740 1 -- 172.21.15.31:0/2931232344 shutdown_connections 2024-04-25T01:21:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.968+0000 7f82555b7740 1 -- 172.21.15.31:0/2931232344 wait complete. 2024-04-25T01:21:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.969+0000 7f82555b7740 1 -- start start 2024-04-25T01:21:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.969+0000 7f82555b7740 1 Processor -- start 2024-04-25T01:21:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.970+0000 7f82555b7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556cc03ae50 0x5556cc045030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.970+0000 7f82555b7740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556cc03b840 0x5556cc045570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.970+0000 7f82555b7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556cc0409b0 0x5556cc04c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.971+0000 7f824aa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556cc03ae50 0x5556cc045030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.971+0000 7f824aa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556cc03ae50 0x5556cc045030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:44956/0 (socket says 172.21.15.31:44956) 2024-04-25T01:21:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.971+0000 7f82555b7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5556cc043900 con 0x5556cc0409b0 2024-04-25T01:21:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.971+0000 7f82555b7740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5556cbf59990 con 0x5556cc03b840 2024-04-25T01:21:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.971+0000 7f82555b7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5556cc043a80 con 0x5556cc03ae50 2024-04-25T01:21:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.971+0000 7f82555b7740 10 client.0 _ll_register_callbacks cb 0x5556cbf5b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:21:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.971+0000 7f824a000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556cc03b840 0x5556cc045570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.971+0000 7f824a000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556cc03b840 0x5556cc045570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:40906/0 (socket says 172.21.15.31:40906) 2024-04-25T01:21:42.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.971+0000 7f824a000640 1 -- 172.21.15.31:0/3638269339 learned_addr learned my addr 172.21.15.31:0/3638269339 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:21:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.972+0000 7f824aa00640 1 -- 172.21.15.31:0/3638269339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556cc03b840 msgr2=0x5556cc045570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.972+0000 7f824aa00640 1 --2- 172.21.15.31:0/3638269339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556cc03b840 0x5556cc045570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.972+0000 7f824aa00640 1 -- 172.21.15.31:0/3638269339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556cc0409b0 msgr2=0x5556cc04c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.972+0000 7f824aa00640 1 --2- 172.21.15.31:0/3638269339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556cc0409b0 0x5556cc04c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:42.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.972+0000 7f824aa00640 1 -- 172.21.15.31:0/3638269339 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556cbd7e620 con 0x5556cc03ae50 2024-04-25T01:21:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.972+0000 7f824aa00640 1 --2- 172.21.15.31:0/3638269339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556cc03ae50 0x5556cc045030 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8240004810 tx=0x7f8240004840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4cbc084066bf8cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.972+0000 7f8231600640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:21:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.973+0000 7f8231600640 1 -- 172.21.15.31:0/3638269339 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240023060 con 0x5556cc03ae50 2024-04-25T01:21:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.973+0000 7f8231600640 1 -- 172.21.15.31:0/3638269339 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f82400150d0 con 0x5556cc03ae50 2024-04-25T01:21:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.974+0000 7f82555b7740 1 -- 172.21.15.31:0/3638269339 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8244041070 con 0x5556cc03ae50 2024-04-25T01:21:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.975+0000 7f8231600640 1 -- 172.21.15.31:0/3638269339 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f82400152d0 con 0x5556cc03ae50 2024-04-25T01:21:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.975+0000 7f82555b7740 10 client.25639 fetch_fsmap learned FSMap version 27 2024-04-25T01:21:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.975+0000 7f82555b7740 1 -- 172.21.15.31:0/3638269339 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5556cc04f600 con 0x5556cc03ae50 2024-04-25T01:21:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.979+0000 7f8231600640 1 -- 172.21.15.31:0/3638269339 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8244041070 con 0x5556cc03ae50 2024-04-25T01:21:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.979+0000 7f8231600640 1 -- 172.21.15.31:0/3638269339 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7f82400124c0 con 0x5556cc03ae50 2024-04-25T01:21:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.981+0000 7f82555b7740 10 client.25639 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:21:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.981+0000 7f82555b7740 10 client.25639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:21:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.981+0000 7f82555b7740 20 client.25639 populate_metadata read hostname 'smithi031' 2024-04-25T01:21:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.981+0000 7f82555b7740 1 --2- 172.21.15.31:0/3638269339 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5556cc050cf0 0x5556cc0530e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.981+0000 7f82555b7740 4 client.25639 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:21:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.981+0000 7f82555b7740 1 -- 172.21.15.31:0/3638269339 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556cbfaaff0 con 0x5556cc050cf0 2024-04-25T01:21:42.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.982+0000 7f824a000640 1 --2- 172.21.15.31:0/3638269339 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5556cc050cf0 0x5556cc0530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.988+0000 7f824a000640 1 --2- 172.21.15.31:0/3638269339 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5556cc050cf0 0x5556cc0530e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e001104c4ba951e0 server_cookie=8bc91795cf34aca4 in_seq=0 out_seq=0 2024-04-25T01:21:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.989+0000 7f8231600640 10 client.25639 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:21:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.992+0000 7f8231600640 1 -- 172.21.15.31:0/3638269339 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5556cbfaaff0 con 0x5556cc050cf0 2024-04-25T01:21:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.992+0000 7f8231600640 10 client.25639 handle_command_reply: tid=0 2024-04-25T01:21:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.993+0000 7f82555b7740 1 client.25639 shutdown 2024-04-25T01:21:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.997+0000 7f82555b7740 1 -- 172.21.15.31:0/3638269339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556cc03ae50 msgr2=0x5556cc045030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:42.997+0000 7f82555b7740 1 --2- 172.21.15.31:0/3638269339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556cc03ae50 0x5556cc045030 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8240004810 tx=0x7f8240004840 comp rx=0 tx=0).stop 2024-04-25T01:21:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.000+0000 7f82555b7740 1 -- 172.21.15.31:0/3638269339 shutdown_connections 2024-04-25T01:21:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.000+0000 7f82555b7740 1 --2- 172.21.15.31:0/3638269339 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5556cc050cf0 0x5556cc0530e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.000+0000 7f82555b7740 1 --2- 172.21.15.31:0/3638269339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556cc0409b0 0x5556cc04c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.000+0000 7f82555b7740 1 --2- 172.21.15.31:0/3638269339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556cc03b840 0x5556cc045570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.000+0000 7f82555b7740 1 --2- 172.21.15.31:0/3638269339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556cc03ae50 0x5556cc045030 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.000+0000 7f82555b7740 1 -- 172.21.15.31:0/3638269339 >> 172.21.15.31:0/3638269339 conn(0x5556cc0364c0 msgr2=0x5556cc035650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.001+0000 7f8231600640 0 client.25639 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:21:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.004+0000 7f82555b7740 1 -- 172.21.15.31:0/3638269339 shutdown_connections 2024-04-25T01:21:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.006+0000 7f82555b7740 1 -- 172.21.15.31:0/3638269339 wait complete. 2024-04-25T01:21:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.007+0000 7f82555b7740 20 client.25639 trim_cache size 0 max 16384 2024-04-25T01:21:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.044+0000 7f82555b7740 1 Processor -- start 2024-04-25T01:21:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.044+0000 7f82555b7740 1 -- start start 2024-04-25T01:21:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.045+0000 7f82555b7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556cc03ae50 0x5556cc039d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.045+0000 7f82555b7740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556cc03b840 0x5556cc04c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.045+0000 7f82555b7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556cc0409b0 0x5556cc044c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.046+0000 7f824aa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556cc03ae50 0x5556cc039d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.046+0000 7f82555b7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1ba150 con 0x5556cc03ae50 2024-04-25T01:21:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.046+0000 7f82555b7740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1b79c0 con 0x5556cc03b840 2024-04-25T01:21:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.046+0000 7f82555b7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1b5810 con 0x5556cc0409b0 2024-04-25T01:21:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.047+0000 7f824b400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556cc0409b0 0x5556cc044c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.047+0000 7f824aa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556cc03ae50 0x5556cc039d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54264/0 (socket says 172.21.15.31:54264) 2024-04-25T01:21:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.047+0000 7f824aa00640 1 -- 172.21.15.31:0/225452034 learned_addr learned my addr 172.21.15.31:0/225452034 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:21:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.047+0000 7f824a000640 1 --2- 172.21.15.31:0/225452034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556cc03b840 0x5556cc04c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.048+0000 7f824aa00640 1 -- 172.21.15.31:0/225452034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556cc0409b0 msgr2=0x5556cc044c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.048+0000 7f824aa00640 1 --2- 172.21.15.31:0/225452034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556cc0409b0 0x5556cc044c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.048+0000 7f824aa00640 1 -- 172.21.15.31:0/225452034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556cc03b840 msgr2=0x5556cc04c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.048+0000 7f824aa00640 1 --2- 172.21.15.31:0/225452034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556cc03b840 0x5556cc04c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.048+0000 7f824aa00640 1 -- 172.21.15.31:0/225452034 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556cc053450 con 0x5556cc03ae50 2024-04-25T01:21:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.050+0000 7f824aa00640 1 --2- 172.21.15.31:0/225452034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556cc03ae50 0x5556cc039d40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8240015940 tx=0x7f824000bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be87aede81ca742 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.052+0000 7f8233e00640 1 -- 172.21.15.31:0/225452034 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824002d080 con 0x5556cc03ae50 2024-04-25T01:21:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.052+0000 7f8233e00640 1 -- 172.21.15.31:0/225452034 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8240012b20 con 0x5556cc03ae50 2024-04-25T01:21:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.052+0000 7f8233e00640 1 -- 172.21.15.31:0/225452034 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240032bd0 con 0x5556cc03ae50 2024-04-25T01:21:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.053+0000 7f82555b7740 1 -- 172.21.15.31:0/225452034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556cc03ae50 msgr2=0x5556cc039d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.053+0000 7f82555b7740 1 --2- 172.21.15.31:0/225452034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556cc03ae50 0x5556cc039d40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8240015940 tx=0x7f824000bf20 comp rx=0 tx=0).stop 2024-04-25T01:21:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.057+0000 7f82555b7740 1 -- 172.21.15.31:0/225452034 shutdown_connections 2024-04-25T01:21:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.057+0000 7f82555b7740 1 --2- 172.21.15.31:0/225452034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556cc0409b0 0x5556cc044c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.058+0000 7f82555b7740 1 --2- 172.21.15.31:0/225452034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556cc03b840 0x5556cc04c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.058+0000 7f82555b7740 1 --2- 172.21.15.31:0/225452034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556cc03ae50 0x5556cc039d40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.058+0000 7f82555b7740 1 -- 172.21.15.31:0/225452034 >> 172.21.15.31:0/225452034 conn(0x5556cc0364c0 msgr2=0x5556cc040090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.059+0000 7f82555b7740 1 -- 172.21.15.31:0/225452034 shutdown_connections 2024-04-25T01:21:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.059+0000 7f82555b7740 1 -- 172.21.15.31:0/225452034 wait complete. 2024-04-25T01:21:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.059+0000 7f82555b7740 1 -- start start 2024-04-25T01:21:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.060+0000 7f82555b7740 1 Processor -- start 2024-04-25T01:21:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.061+0000 7f82555b7740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556cc03ae50 0x5556cc049f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.061+0000 7f82555b7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556cc03b840 0x5556cc04a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.061+0000 7f82555b7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556cc0409b0 0x5556cc04b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.061+0000 7f82555b7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1b5e80 con 0x5556cc03b840 2024-04-25T01:21:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.061+0000 7f82555b7740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1b72f0 con 0x5556cc03ae50 2024-04-25T01:21:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.061+0000 7f82555b7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1b6550 con 0x5556cc0409b0 2024-04-25T01:21:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.061+0000 7f82555b7740 10 client.0 _ll_register_callbacks cb 0x7f8240009050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:21:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.063+0000 7f824aa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556cc03ae50 0x5556cc049f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.064+0000 7f824aa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556cc03ae50 0x5556cc049f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:40924/0 (socket says 172.21.15.31:40924) 2024-04-25T01:21:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.064+0000 7f824b400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556cc0409b0 0x5556cc04b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.064+0000 7f824b400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556cc0409b0 0x5556cc04b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:44984/0 (socket says 172.21.15.31:44984) 2024-04-25T01:21:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.064+0000 7f824b400640 1 -- 172.21.15.31:0/502602154 learned_addr learned my addr 172.21.15.31:0/502602154 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:21:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.065+0000 7f824b400640 1 -- 172.21.15.31:0/502602154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556cc03ae50 msgr2=0x5556cc049f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.065+0000 7f824b400640 1 --2- 172.21.15.31:0/502602154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556cc03ae50 0x5556cc049f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.065+0000 7f824b400640 1 -- 172.21.15.31:0/502602154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556cc03b840 msgr2=0x5556cc04a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.065+0000 7f824b400640 1 --2- 172.21.15.31:0/502602154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556cc03b840 0x5556cc04a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.065+0000 7f824b400640 1 -- 172.21.15.31:0/502602154 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556cc04ba80 con 0x5556cc0409b0 2024-04-25T01:21:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.066+0000 7f824b400640 1 --2- 172.21.15.31:0/502602154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556cc0409b0 0x5556cc04b490 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f82440334d0 tx=0x7f824400c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e581f54458166f35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.066+0000 7f8231600640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:21:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.067+0000 7f8231600640 1 -- 172.21.15.31:0/502602154 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824400a490 con 0x5556cc0409b0 2024-04-25T01:21:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.068+0000 7f8231600640 1 -- 172.21.15.31:0/502602154 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f824400a630 con 0x5556cc0409b0 2024-04-25T01:21:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.069+0000 7f82555b7740 1 -- 172.21.15.31:0/502602154 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8240012b20 con 0x5556cc0409b0 2024-04-25T01:21:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.070+0000 7f8231600640 1 -- 172.21.15.31:0/502602154 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f824400a830 con 0x5556cc0409b0 2024-04-25T01:21:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.070+0000 7f82555b7740 10 client.25645 fetch_fsmap learned FSMap version 27 2024-04-25T01:21:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.070+0000 7f82555b7740 1 -- 172.21.15.31:0/502602154 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5556cc0c8530 con 0x5556cc0409b0 2024-04-25T01:21:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.070+0000 7f8231600640 1 -- 172.21.15.31:0/502602154 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8240012b20 con 0x5556cc0409b0 2024-04-25T01:21:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.071+0000 7f8231600640 1 -- 172.21.15.31:0/502602154 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7f824404f430 con 0x5556cc0409b0 2024-04-25T01:21:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.071+0000 7f82555b7740 10 client.25645 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:21:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.071+0000 7f82555b7740 10 client.25645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:21:43.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.071+0000 7f82555b7740 20 client.25645 populate_metadata read hostname 'smithi031' 2024-04-25T01:21:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.071+0000 7f82555b7740 1 --2- 172.21.15.31:0/502602154 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5556cc0c8ee0 0x5556cc0cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.072+0000 7f82555b7740 4 client.25645 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:21:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.072+0000 7f82555b7740 1 -- 172.21.15.31:0/502602154 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5556cbfa5340 con 0x5556cc0c8ee0 2024-04-25T01:21:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.072+0000 7f824aa00640 1 --2- 172.21.15.31:0/502602154 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5556cc0c8ee0 0x5556cc0cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.073+0000 7f8231600640 10 client.25645 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:21:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.073+0000 7f824aa00640 1 --2- 172.21.15.31:0/502602154 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5556cc0c8ee0 0x5556cc0cb2d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e594be7dfa7d480d server_cookie=14e34a7658cf043 in_seq=0 out_seq=0 2024-04-25T01:21:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.725+0000 7f8231600640 1 -- 172.21.15.31:0/502602154 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5556cbfa5340 con 0x5556cc0c8ee0 2024-04-25T01:21:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.725+0000 7f8231600640 10 client.25645 handle_command_reply: tid=0 2024-04-25T01:21:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.725+0000 7f82555b7740 1 client.25645 shutdown 2024-04-25T01:21:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.726+0000 7f82555b7740 1 -- 172.21.15.31:0/502602154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556cc0409b0 msgr2=0x5556cc04b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.726+0000 7f82555b7740 1 --2- 172.21.15.31:0/502602154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556cc0409b0 0x5556cc04b490 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f82440334d0 tx=0x7f824400c420 comp rx=0 tx=0).stop 2024-04-25T01:21:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.726+0000 7f82555b7740 1 -- 172.21.15.31:0/502602154 shutdown_connections 2024-04-25T01:21:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.726+0000 7f82555b7740 1 --2- 172.21.15.31:0/502602154 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5556cc0c8ee0 0x5556cc0cb2d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.726+0000 7f82555b7740 1 --2- 172.21.15.31:0/502602154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556cc0409b0 0x5556cc04b490 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.726+0000 7f82555b7740 1 --2- 172.21.15.31:0/502602154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556cc03b840 0x5556cc04a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.726+0000 7f82555b7740 1 --2- 172.21.15.31:0/502602154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556cc03ae50 0x5556cc049f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.726+0000 7f82555b7740 1 -- 172.21.15.31:0/502602154 >> 172.21.15.31:0/502602154 conn(0x5556cc0364c0 msgr2=0x5556cc044c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.727+0000 7f8231600640 0 client.25645 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:21:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.727+0000 7f82555b7740 1 -- 172.21.15.31:0/502602154 shutdown_connections 2024-04-25T01:21:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.727+0000 7f82555b7740 1 -- 172.21.15.31:0/502602154 wait complete. 2024-04-25T01:21:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.727+0000 7f82555b7740 20 client.25645 trim_cache size 0 max 16384 2024-04-25T01:21:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:21:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:21:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:21:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:21:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.729+0000 7f8252600640 1 -- 172.21.15.31:0/2509250064 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f82340a2f40 msgr2=0x7f82340a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.729+0000 7f8252600640 1 --2- 172.21.15.31:0/2509250064 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f82340a2f40 0x7f82340a5400 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x5556cc03bc40 tx=0x7f8240006d00 comp rx=0 tx=0).stop 2024-04-25T01:21:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.729+0000 7f8252600640 1 -- 172.21.15.31:0/2509250064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f824c10b190 msgr2=0x7f824c1a6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.729+0000 7f8252600640 1 --2- 172.21.15.31:0/2509250064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f824c10b190 0x7f824c1a6b90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f82380078d0 tx=0x7f8238007d90 comp rx=0 tx=0).stop 2024-04-25T01:21:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.730+0000 7f8252600640 1 -- 172.21.15.31:0/2509250064 shutdown_connections 2024-04-25T01:21:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.730+0000 7f8252600640 1 --2- 172.21.15.31:0/2509250064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f824c10dc70 0x7f824c1adc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.730+0000 7f8252600640 1 --2- 172.21.15.31:0/2509250064 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f82340a2f40 0x7f82340a5400 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.730+0000 7f8252600640 1 --2- 172.21.15.31:0/2509250064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f824c10b190 0x7f824c1a6b90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.730+0000 7f8252600640 1 --2- 172.21.15.31:0/2509250064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f824c1087d0 0x7f824c1a6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.730+0000 7f8252600640 1 -- 172.21.15.31:0/2509250064 >> 172.21.15.31:0/2509250064 conn(0x7f824c06e7b0 msgr2=0x7f824c10a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.730+0000 7f8252600640 1 -- 172.21.15.31:0/2509250064 shutdown_connections 2024-04-25T01:21:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:43.730+0000 7f8252600640 1 -- 172.21.15.31:0/2509250064 wait complete. 2024-04-25T01:21:44.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:43 smithi169 ceph-mon[32759]: pgmap v564: 97 pgs: 97 active+clean; 13 GiB data, 95 GiB used, 978 GiB / 1.0 TiB avail; 107 KiB/s rd, 107 MiB/s wr, 16.38k op/s 2024-04-25T01:21:44.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:43 smithi040 ceph-mon[32518]: pgmap v564: 97 pgs: 97 active+clean; 13 GiB data, 95 GiB used, 978 GiB / 1.0 TiB avail; 107 KiB/s rd, 107 MiB/s wr, 16.38k op/s 2024-04-25T01:21:44.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:43 smithi031 ceph-mon[28053]: pgmap v564: 97 pgs: 97 active+clean; 13 GiB data, 95 GiB used, 978 GiB / 1.0 TiB avail; 107 KiB/s rd, 107 MiB/s wr, 16.38k op/s 2024-04-25T01:21:46.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:45 smithi169 ceph-mon[32759]: pgmap v565: 97 pgs: 97 active+clean; 13 GiB data, 96 GiB used, 977 GiB / 1.0 TiB avail; 119 KiB/s rd, 113 MiB/s wr, 17.34k op/s 2024-04-25T01:21:46.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:45 smithi040 ceph-mon[32518]: pgmap v565: 97 pgs: 97 active+clean; 13 GiB data, 96 GiB used, 977 GiB / 1.0 TiB avail; 119 KiB/s rd, 113 MiB/s wr, 17.34k op/s 2024-04-25T01:21:46.254 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:45 smithi031 ceph-mon[28053]: pgmap v565: 97 pgs: 97 active+clean; 13 GiB data, 96 GiB used, 977 GiB / 1.0 TiB avail; 119 KiB/s rd, 113 MiB/s wr, 17.34k op/s 2024-04-25T01:21:48.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:47 smithi169 ceph-mon[32759]: pgmap v566: 97 pgs: 97 active+clean; 13 GiB data, 97 GiB used, 976 GiB / 1.0 TiB avail; 132 KiB/s rd, 124 MiB/s wr, 18.97k op/s 2024-04-25T01:21:48.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:47 smithi040 ceph-mon[32518]: pgmap v566: 97 pgs: 97 active+clean; 13 GiB data, 97 GiB used, 976 GiB / 1.0 TiB avail; 132 KiB/s rd, 124 MiB/s wr, 18.97k op/s 2024-04-25T01:21:48.250 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:47 smithi031 ceph-mon[28053]: pgmap v566: 97 pgs: 97 active+clean; 13 GiB data, 97 GiB used, 976 GiB / 1.0 TiB avail; 132 KiB/s rd, 124 MiB/s wr, 18.97k op/s 2024-04-25T01:21:50.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:49 smithi169 ceph-mon[32759]: pgmap v567: 97 pgs: 97 active+clean; 13 GiB data, 97 GiB used, 976 GiB / 1.0 TiB avail; 107 KiB/s rd, 98 MiB/s wr, 14.83k op/s 2024-04-25T01:21:50.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:49 smithi040 ceph-mon[32518]: pgmap v567: 97 pgs: 97 active+clean; 13 GiB data, 97 GiB used, 976 GiB / 1.0 TiB avail; 107 KiB/s rd, 98 MiB/s wr, 14.83k op/s 2024-04-25T01:21:50.251 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:49 smithi031 ceph-mon[28053]: pgmap v567: 97 pgs: 97 active+clean; 13 GiB data, 97 GiB used, 976 GiB / 1.0 TiB avail; 107 KiB/s rd, 98 MiB/s wr, 14.83k op/s 2024-04-25T01:21:52.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:51 smithi169 ceph-mon[32759]: pgmap v568: 97 pgs: 97 active+clean; 13 GiB data, 98 GiB used, 975 GiB / 1.0 TiB avail; 119 KiB/s rd, 126 MiB/s wr, 18.60k op/s 2024-04-25T01:21:52.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:51 smithi040 ceph-mon[32518]: pgmap v568: 97 pgs: 97 active+clean; 13 GiB data, 98 GiB used, 975 GiB / 1.0 TiB avail; 119 KiB/s rd, 126 MiB/s wr, 18.60k op/s 2024-04-25T01:21:52.252 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:51 smithi031 ceph-mon[28053]: pgmap v568: 97 pgs: 97 active+clean; 13 GiB data, 98 GiB used, 975 GiB / 1.0 TiB avail; 119 KiB/s rd, 126 MiB/s wr, 18.60k op/s 2024-04-25T01:21:54.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:53 smithi169 ceph-mon[32759]: pgmap v569: 97 pgs: 97 active+clean; 13 GiB data, 99 GiB used, 974 GiB / 1.0 TiB avail; 68 KiB/s rd, 110 MiB/s wr, 16.08k op/s 2024-04-25T01:21:54.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:53 smithi040 ceph-mon[32518]: pgmap v569: 97 pgs: 97 active+clean; 13 GiB data, 99 GiB used, 974 GiB / 1.0 TiB avail; 68 KiB/s rd, 110 MiB/s wr, 16.08k op/s 2024-04-25T01:21:54.251 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:53 smithi031 ceph-mon[28053]: pgmap v569: 97 pgs: 97 active+clean; 13 GiB data, 99 GiB used, 974 GiB / 1.0 TiB avail; 68 KiB/s rd, 110 MiB/s wr, 16.08k op/s 2024-04-25T01:21:56.252 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:55 smithi031 ceph-mon[28053]: pgmap v570: 97 pgs: 97 active+clean; 13 GiB data, 99 GiB used, 974 GiB / 1.0 TiB avail; 69 KiB/s rd, 109 MiB/s wr, 15.96k op/s 2024-04-25T01:21:56.440 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:55 smithi169 ceph-mon[32759]: pgmap v570: 97 pgs: 97 active+clean; 13 GiB data, 99 GiB used, 974 GiB / 1.0 TiB avail; 69 KiB/s rd, 109 MiB/s wr, 15.96k op/s 2024-04-25T01:21:56.443 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:55 smithi040 ceph-mon[32518]: pgmap v570: 97 pgs: 97 active+clean; 13 GiB data, 99 GiB used, 974 GiB / 1.0 TiB avail; 69 KiB/s rd, 109 MiB/s wr, 15.96k op/s 2024-04-25T01:21:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.976+0000 7eff9a000640 1 -- 172.21.15.31:0/1523237644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff94107540 msgr2=0x7eff94109930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.976+0000 7eff9a000640 1 --2- 172.21.15.31:0/1523237644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff94107540 0x7eff94109930 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7eff8c009ca0 tx=0x7eff8c02a8c0 comp rx=0 tx=0).stop 2024-04-25T01:21:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.977+0000 7eff9a000640 1 -- 172.21.15.31:0/1523237644 shutdown_connections 2024-04-25T01:21:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.977+0000 7eff9a000640 1 --2- 172.21.15.31:0/1523237644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff94107540 0x7eff94109930 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.977+0000 7eff9a000640 1 --2- 172.21.15.31:0/1523237644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff94104c10 0x7eff94107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.977+0000 7eff9a000640 1 --2- 172.21.15.31:0/1523237644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff941022e0 0x7eff941046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.977+0000 7eff9a000640 1 -- 172.21.15.31:0/1523237644 >> 172.21.15.31:0/1523237644 conn(0x7eff940fbc50 msgr2=0x7eff940fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.977+0000 7eff9a000640 1 -- 172.21.15.31:0/1523237644 shutdown_connections 2024-04-25T01:21:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.977+0000 7eff9a000640 1 -- 172.21.15.31:0/1523237644 wait complete. 2024-04-25T01:21:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.979+0000 7eff9a000640 1 Processor -- start 2024-04-25T01:21:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.980+0000 7eff9a000640 1 -- start start 2024-04-25T01:21:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.981+0000 7eff9a000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff941022e0 0x7eff94101c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.981+0000 7eff9a000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff94104c10 0x7eff94100290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.981+0000 7eff9a000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff94107540 0x7eff941007d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.981+0000 7eff9a000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7eff9410d4b0 con 0x7eff941022e0 2024-04-25T01:21:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.981+0000 7eff9a000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7eff9410d330 con 0x7eff94104c10 2024-04-25T01:21:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.981+0000 7eff9a000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7eff9410d630 con 0x7eff94107540 2024-04-25T01:21:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.981+0000 7eff93e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff94107540 0x7eff941007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.981+0000 7eff93e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff94107540 0x7eff941007d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:45766/0 (socket says 172.21.15.31:45766) 2024-04-25T01:21:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.981+0000 7eff93e00640 1 -- 172.21.15.31:0/4286799799 learned_addr learned my addr 172.21.15.31:0/4286799799 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:21:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.981+0000 7eff92a00640 1 --2- 172.21.15.31:0/4286799799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff94104c10 0x7eff94100290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.982+0000 7eff93e00640 1 -- 172.21.15.31:0/4286799799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff94104c10 msgr2=0x7eff94100290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.982+0000 7eff93e00640 1 --2- 172.21.15.31:0/4286799799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff94104c10 0x7eff94100290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.982+0000 7eff93e00640 1 -- 172.21.15.31:0/4286799799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff941022e0 msgr2=0x7eff94101c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.982+0000 7eff93e00640 1 --2- 172.21.15.31:0/4286799799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff941022e0 0x7eff94101c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.982+0000 7eff93e00640 1 -- 172.21.15.31:0/4286799799 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff94101060 con 0x7eff94107540 2024-04-25T01:21:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.983+0000 7eff93e00640 1 --2- 172.21.15.31:0/4286799799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff94107540 0x7eff941007d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7eff8c031680 tx=0x7eff8c0316b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.984+0000 7eff8be00640 1 -- 172.21.15.31:0/4286799799 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c051070 con 0x7eff94107540 2024-04-25T01:21:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.984+0000 7eff8be00640 1 -- 172.21.15.31:0/4286799799 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff8c003e50 con 0x7eff94107540 2024-04-25T01:21:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.984+0000 7eff8be00640 1 -- 172.21.15.31:0/4286799799 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c051070 con 0x7eff94107540 2024-04-25T01:21:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.984+0000 7eff9a000640 1 -- 172.21.15.31:0/4286799799 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff941976c0 con 0x7eff94107540 2024-04-25T01:21:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.984+0000 7eff9a000640 1 -- 172.21.15.31:0/4286799799 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff94197b80 con 0x7eff94107540 2024-04-25T01:21:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.992+0000 7eff8be00640 1 -- 172.21.15.31:0/4286799799 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7eff8c054070 con 0x7eff94107540 2024-04-25T01:21:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.995+0000 7eff9cd90740 1 Processor -- start 2024-04-25T01:21:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.997+0000 7eff9cd90740 1 -- start start 2024-04-25T01:21:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.997+0000 7eff9cd90740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626e0bbbca0 0x5626e0bbc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.997+0000 7eff9cd90740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e0bbc690 0x5626e0bc1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.997+0000 7eff8be00640 1 --2- 172.21.15.31:0/4286799799 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7eff780a2f10 0x7eff780a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.998+0000 7eff8be00640 1 -- 172.21.15.31:0/4286799799 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7eff8c0f7750 con 0x7eff94107540 2024-04-25T01:21:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:56.999+0000 7eff92a00640 1 --2- 172.21.15.31:0/4286799799 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7eff780a2f10 0x7eff780a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.001+0000 7eff9cd90740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626e0bc1800 0x5626e0bc3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.001+0000 7eff93400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e0bbc690 0x5626e0bc1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.001+0000 7eff93400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e0bbc690 0x5626e0bc1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:45772/0 (socket says 172.21.15.31:45772) 2024-04-25T01:21:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.001+0000 7eff93400640 1 -- 172.21.15.31:0/332659241 learned_addr learned my addr 172.21.15.31:0/332659241 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:21:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.001+0000 7eff9cd90740 1 -- 172.21.15.31:0/332659241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5626e0b1eab0 con 0x5626e0bbbca0 2024-04-25T01:21:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.002+0000 7eff92a00640 1 --2- 172.21.15.31:0/332659241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626e0bbbca0 0x5626e0bbc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.002+0000 7eff9cd90740 1 -- 172.21.15.31:0/332659241 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5626e0ac7640 con 0x5626e0bc1800 2024-04-25T01:21:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.002+0000 7eff9cd90740 1 -- 172.21.15.31:0/332659241 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5626e0b1aa70 con 0x5626e0bbc690 2024-04-25T01:21:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.002+0000 7eff93400640 1 -- 172.21.15.31:0/332659241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626e0bc1800 msgr2=0x5626e0bc3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.002+0000 7eff93400640 1 --2- 172.21.15.31:0/332659241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626e0bc1800 0x5626e0bc3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.002+0000 7eff93400640 1 -- 172.21.15.31:0/332659241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626e0bbbca0 msgr2=0x5626e0bbc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.002+0000 7eff93400640 1 --2- 172.21.15.31:0/332659241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626e0bbbca0 0x5626e0bbc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.002+0000 7eff93400640 1 -- 172.21.15.31:0/332659241 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626e0833210 con 0x5626e0bbc690 2024-04-25T01:21:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.003+0000 7eff92a00640 1 --2- 172.21.15.31:0/4286799799 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7eff780a2f10 0x7eff780a53d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7eff80006e10 tx=0x7eff80006d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.003+0000 7eff93400640 1 --2- 172.21.15.31:0/332659241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e0bbc690 0x5626e0bc1190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7eff84009c40 tx=0x7eff8400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eb4ae07f54a201d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.004+0000 7eff7fe00640 1 -- 172.21.15.31:0/332659241 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff84005020 con 0x5626e0bbc690 2024-04-25T01:21:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.004+0000 7eff7fe00640 1 -- 172.21.15.31:0/332659241 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff840047a0 con 0x5626e0bbc690 2024-04-25T01:21:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.004+0000 7eff7fe00640 1 -- 172.21.15.31:0/332659241 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff84010720 con 0x5626e0bbc690 2024-04-25T01:21:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.006+0000 7eff9cd90740 1 -- 172.21.15.31:0/332659241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e0bbc690 msgr2=0x5626e0bc1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.007+0000 7eff9cd90740 1 --2- 172.21.15.31:0/332659241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e0bbc690 0x5626e0bc1190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7eff84009c40 tx=0x7eff8400e720 comp rx=0 tx=0).stop 2024-04-25T01:21:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.007+0000 7eff9cd90740 1 -- 172.21.15.31:0/332659241 shutdown_connections 2024-04-25T01:21:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.007+0000 7eff9cd90740 1 --2- 172.21.15.31:0/332659241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626e0bc1800 0x5626e0bc3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.007+0000 7eff9cd90740 1 --2- 172.21.15.31:0/332659241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e0bbc690 0x5626e0bc1190 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.008+0000 7eff9cd90740 1 --2- 172.21.15.31:0/332659241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626e0bbbca0 0x5626e0bbc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.008+0000 7eff9cd90740 1 -- 172.21.15.31:0/332659241 >> 172.21.15.31:0/332659241 conn(0x5626e0bb7310 msgr2=0x5626e0bb7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.008+0000 7eff9cd90740 1 -- 172.21.15.31:0/332659241 shutdown_connections 2024-04-25T01:21:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.008+0000 7eff9cd90740 1 -- 172.21.15.31:0/332659241 wait complete. 2024-04-25T01:21:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.009+0000 7eff9cd90740 1 -- start start 2024-04-25T01:21:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.009+0000 7eff9cd90740 1 Processor -- start 2024-04-25T01:21:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.011+0000 7eff9cd90740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626e0bbbca0 0x5626e0bc5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.011+0000 7eff9cd90740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626e0bbc690 0x5626e0bc64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.011+0000 7eff93400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626e0bbbca0 0x5626e0bc5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.012+0000 7eff93400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626e0bbbca0 0x5626e0bc5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:41056/0 (socket says 172.21.15.31:41056) 2024-04-25T01:21:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.012+0000 7eff92a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626e0bbc690 0x5626e0bc64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.012+0000 7eff93400640 1 -- 172.21.15.31:0/2336084612 learned_addr learned my addr 172.21.15.31:0/2336084612 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:21:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.012+0000 7eff9cd90740 1 --2- 172.21.15.31:0/2336084612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e0bc1800 0x5626e0bcd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.012+0000 7eff93e00640 1 --2- 172.21.15.31:0/2336084612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e0bc1800 0x5626e0bcd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.012+0000 7eff9cd90740 1 -- 172.21.15.31:0/2336084612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5626e0bc4800 con 0x5626e0bbc690 2024-04-25T01:21:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.012+0000 7eff9cd90740 1 -- 172.21.15.31:0/2336084612 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5626e0ada990 con 0x5626e0bbbca0 2024-04-25T01:21:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.013+0000 7eff9cd90740 1 -- 172.21.15.31:0/2336084612 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5626e0bc4980 con 0x5626e0bc1800 2024-04-25T01:21:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.013+0000 7eff93e00640 1 -- 172.21.15.31:0/2336084612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626e0bbbca0 msgr2=0x5626e0bc5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.013+0000 7eff93e00640 1 --2- 172.21.15.31:0/2336084612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626e0bbbca0 0x5626e0bc5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.014+0000 7eff93e00640 1 -- 172.21.15.31:0/2336084612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626e0bbc690 msgr2=0x5626e0bc64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.014+0000 7eff93e00640 1 --2- 172.21.15.31:0/2336084612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626e0bbc690 0x5626e0bc64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.014+0000 7eff93e00640 1 -- 172.21.15.31:0/2336084612 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626e08ff620 con 0x5626e0bc1800 2024-04-25T01:21:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.014+0000 7eff9cd90740 10 client.0 _ll_register_callbacks cb 0x5626e0adc950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:21:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.014+0000 7eff9cd90740 1 -- 172.21.15.31:0/2336084612 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff840047a0 con 0x5626e0bc1800 2024-04-25T01:21:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.015+0000 7eff93e00640 1 --2- 172.21.15.31:0/2336084612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e0bc1800 0x5626e0bcd4e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0f5330 tx=0x7eff8c0bfcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=360c92840e9ca0e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.015+0000 7eff7cc00640 10 client.25657 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:21:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.015+0000 7eff7cc00640 1 -- 172.21.15.31:0/2336084612 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c0c4740 con 0x5626e0bc1800 2024-04-25T01:21:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.016+0000 7eff7cc00640 1 -- 172.21.15.31:0/2336084612 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7eff8c031720 con 0x5626e0bc1800 2024-04-25T01:21:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.016+0000 7eff7cc00640 1 -- 172.21.15.31:0/2336084612 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7eff8c0c4040 con 0x5626e0bc1800 2024-04-25T01:21:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.016+0000 7eff9cd90740 10 client.25657 fetch_fsmap learned FSMap version 27 2024-04-25T01:21:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.016+0000 7eff9cd90740 1 -- 172.21.15.31:0/2336084612 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({fsmap=27}) -- 0x5626e0bcffa0 con 0x5626e0bc1800 2024-04-25T01:21:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.017+0000 7eff7cc00640 1 -- 172.21.15.31:0/2336084612 <== mon.1 v2:172.21.15.169:3300/0 4 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7eff8c004b40 con 0x5626e0bc1800 2024-04-25T01:21:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.017+0000 7eff9cd90740 10 client.25657 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:21:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.017+0000 7eff9cd90740 10 client.25657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:21:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.017+0000 7eff9cd90740 20 client.25657 populate_metadata read hostname 'smithi031' 2024-04-25T01:21:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.017+0000 7eff9cd90740 1 --2- 172.21.15.31:0/2336084612 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5626e0bd1860 0x5626e0bd3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.017+0000 7eff9cd90740 4 client.25657 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:21:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.017+0000 7eff9cd90740 1 -- 172.21.15.31:0/2336084612 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626e0b2bff0 con 0x5626e0bd1860 2024-04-25T01:21:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.018+0000 7eff93400640 1 --2- 172.21.15.31:0/2336084612 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5626e0bd1860 0x5626e0bd3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.019+0000 7eff93400640 1 --2- 172.21.15.31:0/2336084612 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5626e0bd1860 0x5626e0bd3c50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a93e7338e1063445 server_cookie=db643304b115bffa in_seq=0 out_seq=0 2024-04-25T01:21:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.019+0000 7eff7cc00640 10 client.25657 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:21:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.020+0000 7eff7cc00640 1 -- 172.21.15.31:0/2336084612 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5626e0b2bff0 con 0x5626e0bd1860 2024-04-25T01:21:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.020+0000 7eff7cc00640 10 client.25657 handle_command_reply: tid=0 2024-04-25T01:21:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.021+0000 7eff9cd90740 1 client.25657 shutdown 2024-04-25T01:21:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.026+0000 7eff9cd90740 1 -- 172.21.15.31:0/2336084612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e0bc1800 msgr2=0x5626e0bcd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.027+0000 7eff9cd90740 1 --2- 172.21.15.31:0/2336084612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e0bc1800 0x5626e0bcd4e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0f5330 tx=0x7eff8c0bfcb0 comp rx=0 tx=0).stop 2024-04-25T01:21:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.027+0000 7eff9cd90740 1 -- 172.21.15.31:0/2336084612 shutdown_connections 2024-04-25T01:21:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.027+0000 7eff9cd90740 1 --2- 172.21.15.31:0/2336084612 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5626e0bd1860 0x5626e0bd3c50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.027+0000 7eff9cd90740 1 --2- 172.21.15.31:0/2336084612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e0bc1800 0x5626e0bcd4e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.027+0000 7eff9cd90740 1 --2- 172.21.15.31:0/2336084612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626e0bbc690 0x5626e0bc64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.027+0000 7eff9cd90740 1 --2- 172.21.15.31:0/2336084612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626e0bbbca0 0x5626e0bc5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.027+0000 7eff9cd90740 1 -- 172.21.15.31:0/2336084612 >> 172.21.15.31:0/2336084612 conn(0x5626e0bb7310 msgr2=0x5626e0bb6480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.028+0000 7eff7cc00640 0 client.25657 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:21:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.028+0000 7eff9cd90740 1 -- 172.21.15.31:0/2336084612 shutdown_connections 2024-04-25T01:21:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.028+0000 7eff9cd90740 1 -- 172.21.15.31:0/2336084612 wait complete. 2024-04-25T01:21:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.029+0000 7eff9cd90740 20 client.25657 trim_cache size 0 max 16384 2024-04-25T01:21:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.164+0000 7eff9cd90740 1 Processor -- start 2024-04-25T01:21:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.164+0000 7eff9cd90740 1 -- start start 2024-04-25T01:21:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.165+0000 7eff9cd90740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626e0bbbca0 0x5626e0bba300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.165+0000 7eff9cd90740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e0bbc690 0x5626e0bcd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.165+0000 7eff9cd90740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626e0bc1800 0x5626e0bba840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.165+0000 7eff9cd90740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7eff941a9190 con 0x5626e0bbbca0 2024-04-25T01:21:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.165+0000 7eff9cd90740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7eff941a6a00 con 0x5626e0bc1800 2024-04-25T01:21:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.165+0000 7eff9cd90740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7eff941a4850 con 0x5626e0bbc690 2024-04-25T01:21:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.166+0000 7eff93e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626e0bc1800 0x5626e0bba840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.166+0000 7eff93e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626e0bc1800 0x5626e0bba840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:41060/0 (socket says 172.21.15.31:41060) 2024-04-25T01:21:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.166+0000 7eff93e00640 1 -- 172.21.15.31:0/1413695406 learned_addr learned my addr 172.21.15.31:0/1413695406 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:21:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.166+0000 7eff92a00640 1 --2- 172.21.15.31:0/1413695406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e0bbc690 0x5626e0bcd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.167+0000 7eff93e00640 1 -- 172.21.15.31:0/1413695406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e0bbc690 msgr2=0x5626e0bcd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.167+0000 7eff93e00640 1 --2- 172.21.15.31:0/1413695406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e0bbc690 0x5626e0bcd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.167+0000 7eff93e00640 1 -- 172.21.15.31:0/1413695406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626e0bbbca0 msgr2=0x5626e0bba300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.167+0000 7eff93e00640 1 --2- 172.21.15.31:0/1413695406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626e0bbbca0 0x5626e0bba300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.167+0000 7eff93e00640 1 -- 172.21.15.31:0/1413695406 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626e0bb5890 con 0x5626e0bc1800 2024-04-25T01:21:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.170+0000 7eff93e00640 1 --2- 172.21.15.31:0/1413695406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626e0bc1800 0x5626e0bba840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7eff8c04e000 tx=0x7eff8c0e4000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f58b5ee64bdd3022 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.170+0000 7eff7f400640 1 -- 172.21.15.31:0/1413695406 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c0c4740 con 0x5626e0bc1800 2024-04-25T01:21:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.170+0000 7eff7f400640 1 -- 172.21.15.31:0/1413695406 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff840047a0 con 0x5626e0bc1800 2024-04-25T01:21:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.170+0000 7eff7f400640 1 -- 172.21.15.31:0/1413695406 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c0bebb0 con 0x5626e0bc1800 2024-04-25T01:21:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.171+0000 7eff9cd90740 1 -- 172.21.15.31:0/1413695406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626e0bc1800 msgr2=0x5626e0bba840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.171+0000 7eff9cd90740 1 --2- 172.21.15.31:0/1413695406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626e0bc1800 0x5626e0bba840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7eff8c04e000 tx=0x7eff8c0e4000 comp rx=0 tx=0).stop 2024-04-25T01:21:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.173+0000 7eff9cd90740 1 -- 172.21.15.31:0/1413695406 shutdown_connections 2024-04-25T01:21:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.173+0000 7eff9cd90740 1 --2- 172.21.15.31:0/1413695406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626e0bc1800 0x5626e0bba840 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.173+0000 7eff9cd90740 1 --2- 172.21.15.31:0/1413695406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e0bbc690 0x5626e0bcd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.173+0000 7eff9cd90740 1 --2- 172.21.15.31:0/1413695406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626e0bbbca0 0x5626e0bba300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.173+0000 7eff9cd90740 1 -- 172.21.15.31:0/1413695406 >> 172.21.15.31:0/1413695406 conn(0x5626e0bb7310 msgr2=0x5626e0bbd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.174+0000 7eff9cd90740 1 -- 172.21.15.31:0/1413695406 shutdown_connections 2024-04-25T01:21:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.174+0000 7eff9cd90740 1 -- 172.21.15.31:0/1413695406 wait complete. 2024-04-25T01:21:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.174+0000 7eff9cd90740 1 -- start start 2024-04-25T01:21:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.175+0000 7eff9cd90740 1 Processor -- start 2024-04-25T01:21:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.177+0000 7eff9cd90740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626e0bbbca0 0x5626e0bcaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.178+0000 7eff9cd90740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e0bbc690 0x5626e0bcb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.181+0000 7eff9cd90740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626e0bc1800 0x5626e0bcc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.181+0000 7eff9cd90740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7eff941a4ec0 con 0x5626e0bc1800 2024-04-25T01:21:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.181+0000 7eff9cd90740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7eff941a6330 con 0x5626e0bbbca0 2024-04-25T01:21:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.181+0000 7eff9cd90740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7eff941a5590 con 0x5626e0bbc690 2024-04-25T01:21:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.181+0000 7eff9cd90740 10 client.0 _ll_register_callbacks cb 0x7eff8c041ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:21:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.178+0000 7eff93400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626e0bbbca0 0x5626e0bcaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.182+0000 7eff93400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626e0bbbca0 0x5626e0bcaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:41076/0 (socket says 172.21.15.31:41076) 2024-04-25T01:21:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.182+0000 7eff93400640 1 -- 172.21.15.31:0/2534492603 learned_addr learned my addr 172.21.15.31:0/2534492603 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:21:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.182+0000 7eff92a00640 1 --2- 172.21.15.31:0/2534492603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e0bbc690 0x5626e0bcb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.182+0000 7eff92a00640 1 -- 172.21.15.31:0/2534492603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626e0bbbca0 msgr2=0x5626e0bcaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.182+0000 7eff92a00640 1 --2- 172.21.15.31:0/2534492603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626e0bbbca0 0x5626e0bcaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.182+0000 7eff92a00640 1 -- 172.21.15.31:0/2534492603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626e0bc1800 msgr2=0x5626e0bcc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:21:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.183+0000 7eff92a00640 1 --2- 172.21.15.31:0/2534492603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626e0bc1800 0x5626e0bcc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.183+0000 7eff92a00640 1 -- 172.21.15.31:0/2534492603 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626e0bcc7e0 con 0x5626e0bbc690 2024-04-25T01:21:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.184+0000 7eff92a00640 1 --2- 172.21.15.31:0/2534492603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e0bbc690 0x5626e0bcb1e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x5626e0bbcef0 tx=0x7eff80009110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e865af973642c72a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:21:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.184+0000 7eff7cc00640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:21:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.184+0000 7eff7cc00640 1 -- 172.21.15.31:0/2534492603 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8003f060 con 0x5626e0bbc690 2024-04-25T01:21:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.184+0000 7eff9cd90740 1 -- 172.21.15.31:0/2534492603 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff840047a0 con 0x5626e0bbc690 2024-04-25T01:21:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.185+0000 7eff7cc00640 1 -- 172.21.15.31:0/2534492603 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7eff80032790 con 0x5626e0bbc690 2024-04-25T01:21:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.185+0000 7eff7cc00640 1 -- 172.21.15.31:0/2534492603 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7eff800329d0 con 0x5626e0bbc690 2024-04-25T01:21:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.185+0000 7eff9cd90740 10 client.25663 fetch_fsmap learned FSMap version 27 2024-04-25T01:21:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.185+0000 7eff9cd90740 1 -- 172.21.15.31:0/2534492603 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5626e0c37f60 con 0x5626e0bbc690 2024-04-25T01:21:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.185+0000 7eff7cc00640 1 -- 172.21.15.31:0/2534492603 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff840047a0 con 0x5626e0bbc690 2024-04-25T01:21:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.185+0000 7eff7cc00640 1 -- 172.21.15.31:0/2534492603 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7eff80039030 con 0x5626e0bbc690 2024-04-25T01:21:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.186+0000 7eff9cd90740 10 client.25663 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T01:21:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.186+0000 7eff9cd90740 10 client.25663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:21:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.186+0000 7eff9cd90740 20 client.25663 populate_metadata read hostname 'smithi031' 2024-04-25T01:21:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.186+0000 7eff9cd90740 1 --2- 172.21.15.31:0/2534492603 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5626e0c388c0 0x5626e0c3ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:21:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.186+0000 7eff9cd90740 4 client.25663 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:21:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.186+0000 7eff9cd90740 1 -- 172.21.15.31:0/2534492603 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5626e0b26340 con 0x5626e0c388c0 2024-04-25T01:21:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.186+0000 7eff93400640 1 --2- 172.21.15.31:0/2534492603 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5626e0c388c0 0x5626e0c3ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:21:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.187+0000 7eff93400640 1 --2- 172.21.15.31:0/2534492603 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5626e0c388c0 0x5626e0c3ace0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e47947dd1ad89960 server_cookie=ac25f3412e75518f in_seq=0 out_seq=0 2024-04-25T01:21:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:57.187+0000 7eff7cc00640 10 client.25663 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:21:58.250 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:57 smithi031 ceph-mon[28053]: pgmap v571: 97 pgs: 97 active+clean; 13 GiB data, 99 GiB used, 974 GiB / 1.0 TiB avail; 104 KiB/s rd, 128 MiB/s wr, 18.30k op/s 2024-04-25T01:21:58.440 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:57 smithi169 ceph-mon[32759]: pgmap v571: 97 pgs: 97 active+clean; 13 GiB data, 99 GiB used, 974 GiB / 1.0 TiB avail; 104 KiB/s rd, 128 MiB/s wr, 18.30k op/s 2024-04-25T01:21:58.443 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:57 smithi040 ceph-mon[32518]: pgmap v571: 97 pgs: 97 active+clean; 13 GiB data, 99 GiB used, 974 GiB / 1.0 TiB avail; 104 KiB/s rd, 128 MiB/s wr, 18.30k op/s 2024-04-25T01:21:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.652+0000 7eff7cc00640 1 -- 172.21.15.31:0/2534492603 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5626e0b26340 con 0x5626e0c388c0 2024-04-25T01:21:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.652+0000 7eff7cc00640 10 client.25663 handle_command_reply: tid=0 2024-04-25T01:21:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.652+0000 7eff9cd90740 1 client.25663 shutdown 2024-04-25T01:21:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.653+0000 7eff9cd90740 1 -- 172.21.15.31:0/2534492603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e0bbc690 msgr2=0x5626e0bcb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.653+0000 7eff9cd90740 1 --2- 172.21.15.31:0/2534492603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e0bbc690 0x5626e0bcb1e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x5626e0bbcef0 tx=0x7eff80009110 comp rx=0 tx=0).stop 2024-04-25T01:21:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.653+0000 7eff9cd90740 1 -- 172.21.15.31:0/2534492603 shutdown_connections 2024-04-25T01:21:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.653+0000 7eff9cd90740 1 --2- 172.21.15.31:0/2534492603 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5626e0c388c0 0x5626e0c3ace0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.653+0000 7eff9cd90740 1 --2- 172.21.15.31:0/2534492603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5626e0bc1800 0x5626e0bcc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.653+0000 7eff9cd90740 1 --2- 172.21.15.31:0/2534492603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e0bbc690 0x5626e0bcb1e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.653+0000 7eff9cd90740 1 --2- 172.21.15.31:0/2534492603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626e0bbbca0 0x5626e0bcaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.653+0000 7eff9cd90740 1 -- 172.21.15.31:0/2534492603 >> 172.21.15.31:0/2534492603 conn(0x5626e0bb7310 msgr2=0x5626e0bba300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.653+0000 7eff7cc00640 0 client.25663 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:21:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.654+0000 7eff9cd90740 1 -- 172.21.15.31:0/2534492603 shutdown_connections 2024-04-25T01:21:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.654+0000 7eff9cd90740 1 -- 172.21.15.31:0/2534492603 wait complete. 2024-04-25T01:21:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.654+0000 7eff9cd90740 20 client.25663 trim_cache size 0 max 16384 2024-04-25T01:21:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:21:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:21:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:21:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:21:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.657+0000 7eff9a000640 1 -- 172.21.15.31:0/4286799799 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7eff780a2f10 msgr2=0x7eff780a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.657+0000 7eff9a000640 1 --2- 172.21.15.31:0/4286799799 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7eff780a2f10 0x7eff780a53d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7eff80006e10 tx=0x7eff80006d60 comp rx=0 tx=0).stop 2024-04-25T01:21:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.658+0000 7eff9a000640 1 -- 172.21.15.31:0/4286799799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff94107540 msgr2=0x7eff941007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:21:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.658+0000 7eff9a000640 1 --2- 172.21.15.31:0/4286799799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff94107540 0x7eff941007d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7eff8c031680 tx=0x7eff8c0316b0 comp rx=0 tx=0).stop 2024-04-25T01:21:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.658+0000 7eff9a000640 1 -- 172.21.15.31:0/4286799799 shutdown_connections 2024-04-25T01:21:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.658+0000 7eff9a000640 1 --2- 172.21.15.31:0/4286799799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff94107540 0x7eff941007d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.658+0000 7eff9a000640 1 --2- 172.21.15.31:0/4286799799 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7eff780a2f10 0x7eff780a53d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.658+0000 7eff9a000640 1 --2- 172.21.15.31:0/4286799799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff94104c10 0x7eff94100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.658+0000 7eff9a000640 1 --2- 172.21.15.31:0/4286799799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff941022e0 0x7eff94101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:21:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.658+0000 7eff9a000640 1 -- 172.21.15.31:0/4286799799 >> 172.21.15.31:0/4286799799 conn(0x7eff940fbc50 msgr2=0x7eff94102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:21:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.659+0000 7eff9a000640 1 -- 172.21.15.31:0/4286799799 shutdown_connections 2024-04-25T01:21:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:21:58.659+0000 7eff9a000640 1 -- 172.21.15.31:0/4286799799 wait complete. 2024-04-25T01:22:00.252 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:21:59 smithi031 ceph-mon[28053]: pgmap v572: 97 pgs: 97 active+clean; 13 GiB data, 99 GiB used, 974 GiB / 1.0 TiB avail; 83 KiB/s rd, 104 MiB/s wr, 14.48k op/s 2024-04-25T01:22:00.440 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:21:59 smithi169 ceph-mon[32759]: pgmap v572: 97 pgs: 97 active+clean; 13 GiB data, 99 GiB used, 974 GiB / 1.0 TiB avail; 83 KiB/s rd, 104 MiB/s wr, 14.48k op/s 2024-04-25T01:22:00.443 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:21:59 smithi040 ceph-mon[32518]: pgmap v572: 97 pgs: 97 active+clean; 13 GiB data, 99 GiB used, 974 GiB / 1.0 TiB avail; 83 KiB/s rd, 104 MiB/s wr, 14.48k op/s 2024-04-25T01:22:02.440 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:02 smithi169 ceph-mon[32759]: pgmap v573: 97 pgs: 97 active+clean; 13 GiB data, 100 GiB used, 973 GiB / 1.0 TiB avail; 96 KiB/s rd, 127 MiB/s wr, 17.65k op/s 2024-04-25T01:22:02.443 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:02 smithi040 ceph-mon[32518]: pgmap v573: 97 pgs: 97 active+clean; 13 GiB data, 100 GiB used, 973 GiB / 1.0 TiB avail; 96 KiB/s rd, 127 MiB/s wr, 17.65k op/s 2024-04-25T01:22:02.503 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:02 smithi031 ceph-mon[28053]: pgmap v573: 97 pgs: 97 active+clean; 13 GiB data, 100 GiB used, 973 GiB / 1.0 TiB avail; 96 KiB/s rd, 127 MiB/s wr, 17.65k op/s 2024-04-25T01:22:04.440 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:04 smithi169 ceph-mon[32759]: pgmap v574: 97 pgs: 97 active+clean; 13 GiB data, 100 GiB used, 972 GiB / 1.0 TiB avail; 60 KiB/s rd, 106 MiB/s wr, 14.78k op/s 2024-04-25T01:22:04.443 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:04 smithi040 ceph-mon[32518]: pgmap v574: 97 pgs: 97 active+clean; 13 GiB data, 100 GiB used, 972 GiB / 1.0 TiB avail; 60 KiB/s rd, 106 MiB/s wr, 14.78k op/s 2024-04-25T01:22:04.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:04 smithi031 ceph-mon[28053]: pgmap v574: 97 pgs: 97 active+clean; 13 GiB data, 100 GiB used, 972 GiB / 1.0 TiB avail; 60 KiB/s rd, 106 MiB/s wr, 14.78k op/s 2024-04-25T01:22:06.440 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:06 smithi169 ceph-mon[32759]: pgmap v575: 97 pgs: 97 active+clean; 13 GiB data, 101 GiB used, 972 GiB / 1.0 TiB avail; 60 KiB/s rd, 101 MiB/s wr, 14.13k op/s 2024-04-25T01:22:06.443 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:06 smithi040 ceph-mon[32518]: pgmap v575: 97 pgs: 97 active+clean; 13 GiB data, 101 GiB used, 972 GiB / 1.0 TiB avail; 60 KiB/s rd, 101 MiB/s wr, 14.13k op/s 2024-04-25T01:22:06.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:06 smithi031 ceph-mon[28053]: pgmap v575: 97 pgs: 97 active+clean; 13 GiB data, 101 GiB used, 972 GiB / 1.0 TiB avail; 60 KiB/s rd, 101 MiB/s wr, 14.13k op/s 2024-04-25T01:22:08.440 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:08 smithi169 ceph-mon[32759]: pgmap v576: 97 pgs: 97 active+clean; 13 GiB data, 102 GiB used, 971 GiB / 1.0 TiB avail; 76 KiB/s rd, 128 MiB/s wr, 18.38k op/s 2024-04-25T01:22:08.443 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:08 smithi040 ceph-mon[32518]: pgmap v576: 97 pgs: 97 active+clean; 13 GiB data, 102 GiB used, 971 GiB / 1.0 TiB avail; 76 KiB/s rd, 128 MiB/s wr, 18.38k op/s 2024-04-25T01:22:08.500 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:08 smithi031 ceph-mon[28053]: pgmap v576: 97 pgs: 97 active+clean; 13 GiB data, 102 GiB used, 971 GiB / 1.0 TiB avail; 76 KiB/s rd, 128 MiB/s wr, 18.38k op/s 2024-04-25T01:22:10.439 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:10 smithi169 ceph-mon[32759]: pgmap v577: 97 pgs: 97 active+clean; 13 GiB data, 103 GiB used, 970 GiB / 1.0 TiB avail; 30 KiB/s rd, 99 MiB/s wr, 14.41k op/s 2024-04-25T01:22:10.443 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:10 smithi040 ceph-mon[32518]: pgmap v577: 97 pgs: 97 active+clean; 13 GiB data, 103 GiB used, 970 GiB / 1.0 TiB avail; 30 KiB/s rd, 99 MiB/s wr, 14.41k op/s 2024-04-25T01:22:10.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:10 smithi031 ceph-mon[28053]: pgmap v577: 97 pgs: 97 active+clean; 13 GiB data, 103 GiB used, 970 GiB / 1.0 TiB avail; 30 KiB/s rd, 99 MiB/s wr, 14.41k op/s 2024-04-25T01:22:11.440 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:11 smithi169 ceph-mon[32759]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:22:11.443 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:11 smithi040 ceph-mon[32518]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:22:11.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:11 smithi031 ceph-mon[28053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:22:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.876+0000 7ff0aca00640 1 -- 172.21.15.31:0/1550374598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0a8068110 msgr2=0x7ff0a8100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.876+0000 7ff0aca00640 1 --2- 172.21.15.31:0/1550374598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0a8068110 0x7ff0a8100bc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff09800aa10 tx=0x7ff09802b730 comp rx=0 tx=0).stop 2024-04-25T01:22:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.877+0000 7ff0aca00640 1 -- 172.21.15.31:0/1550374598 shutdown_connections 2024-04-25T01:22:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.877+0000 7ff0aca00640 1 --2- 172.21.15.31:0/1550374598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0a8105e00 0x7ff0a81081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.877+0000 7ff0aca00640 1 --2- 172.21.15.31:0/1550374598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0a8101100 0x7ff0a8101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.877+0000 7ff0aca00640 1 --2- 172.21.15.31:0/1550374598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0a8068110 0x7ff0a8100bc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.877+0000 7ff0aca00640 1 -- 172.21.15.31:0/1550374598 >> 172.21.15.31:0/1550374598 conn(0x7ff0a80faf60 msgr2=0x7ff0a80fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:22:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.877+0000 7ff0aca00640 1 -- 172.21.15.31:0/1550374598 shutdown_connections 2024-04-25T01:22:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.877+0000 7ff0aca00640 1 -- 172.21.15.31:0/1550374598 wait complete. 2024-04-25T01:22:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.879+0000 7ff0aca00640 1 Processor -- start 2024-04-25T01:22:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.879+0000 7ff0aca00640 1 -- start start 2024-04-25T01:22:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.880+0000 7ff0aca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0a8068110 0x7ff0a8199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.880+0000 7ff0aca00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0a8101100 0x7ff0a819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.880+0000 7ff0aca00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0a8105e00 0x7ff0a81a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.881+0000 7ff0a5400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0a8101100 0x7ff0a819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.881+0000 7ff0a5400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0a8101100 0x7ff0a819a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:33536/0 (socket says 172.21.15.31:33536) 2024-04-25T01:22:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.881+0000 7ff0a5400640 1 -- 172.21.15.31:0/1813200381 learned_addr learned my addr 172.21.15.31:0/1813200381 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:22:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.881+0000 7ff0aca00640 1 -- 172.21.15.31:0/1813200381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a810ad90 con 0x7ff0a8105e00 2024-04-25T01:22:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.881+0000 7ff0aca00640 1 -- 172.21.15.31:0/1813200381 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a810ac10 con 0x7ff0a8101100 2024-04-25T01:22:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.881+0000 7ff0aca00640 1 -- 172.21.15.31:0/1813200381 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a810af10 con 0x7ff0a8068110 2024-04-25T01:22:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.881+0000 7ff0a5e00640 1 --2- 172.21.15.31:0/1813200381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0a8068110 0x7ff0a8199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.881+0000 7ff0a6800640 1 --2- 172.21.15.31:0/1813200381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0a8105e00 0x7ff0a81a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.882+0000 7ff0a5400640 1 -- 172.21.15.31:0/1813200381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0a8068110 msgr2=0x7ff0a8199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.882+0000 7ff0a5400640 1 --2- 172.21.15.31:0/1813200381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0a8068110 0x7ff0a8199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.882+0000 7ff0a5400640 1 -- 172.21.15.31:0/1813200381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0a8105e00 msgr2=0x7ff0a81a5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.882+0000 7ff0a5400640 1 --2- 172.21.15.31:0/1813200381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0a8105e00 0x7ff0a81a5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.882+0000 7ff0a5400640 1 -- 172.21.15.31:0/1813200381 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a8110f10 con 0x7ff0a8101100 2024-04-25T01:22:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.883+0000 7ff0a5400640 1 --2- 172.21.15.31:0/1813200381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0a8101100 0x7ff0a819a1c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff09400e9c0 tx=0x7ff09400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:22:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.884+0000 7ff09ea00640 1 -- 172.21.15.31:0/1813200381 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09400cd80 con 0x7ff0a8101100 2024-04-25T01:22:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.884+0000 7ff09ea00640 1 -- 172.21.15.31:0/1813200381 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0940035f0 con 0x7ff0a8101100 2024-04-25T01:22:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.884+0000 7ff09ea00640 1 -- 172.21.15.31:0/1813200381 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094010690 con 0x7ff0a8101100 2024-04-25T01:22:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.884+0000 7ff0aca00640 1 -- 172.21.15.31:0/1813200381 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a81a5ff0 con 0x7ff0a8101100 2024-04-25T01:22:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.884+0000 7ff0aca00640 1 -- 172.21.15.31:0/1813200381 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a81a64e0 con 0x7ff0a8101100 2024-04-25T01:22:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.891+0000 7ff0af983740 1 Processor -- start 2024-04-25T01:22:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.895+0000 7ff0af983740 1 -- start start 2024-04-25T01:22:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.895+0000 7ff0af983740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d3231eca0 0x562d3231f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.895+0000 7ff0af983740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d3231f690 0x562d32324190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.895+0000 7ff0af983740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d32324800 0x562d32326c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.895+0000 7ff0af983740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562d32281ab0 con 0x562d3231f690 2024-04-25T01:22:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.895+0000 7ff0af983740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562d3222a640 con 0x562d32324800 2024-04-25T01:22:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.895+0000 7ff0af983740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562d3227da70 con 0x562d3231eca0 2024-04-25T01:22:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.896+0000 7ff09ea00640 1 -- 172.21.15.31:0/1813200381 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff0940037a0 con 0x7ff0a8101100 2024-04-25T01:22:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.898+0000 7ff0a6800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d3231eca0 0x562d3231f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.898+0000 7ff0a6800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d3231eca0 0x562d3231f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:51214/0 (socket says 172.21.15.31:51214) 2024-04-25T01:22:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.898+0000 7ff0a6800640 1 -- 172.21.15.31:0/691445735 learned_addr learned my addr 172.21.15.31:0/691445735 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:22:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.900+0000 7ff09ea00640 1 --2- 172.21.15.31:0/1813200381 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff08c0a2cb0 0x7ff08c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.902+0000 7ff0a5e00640 1 --2- 172.21.15.31:0/691445735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d3231f690 0x562d32324190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.902+0000 7ff0a5400640 1 --2- 172.21.15.31:0/691445735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d32324800 0x562d32326c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.902+0000 7ff0a6800640 1 -- 172.21.15.31:0/691445735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d32324800 msgr2=0x562d32326c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.902+0000 7ff0a6800640 1 --2- 172.21.15.31:0/691445735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d32324800 0x562d32326c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.902+0000 7ff0a6800640 1 -- 172.21.15.31:0/691445735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d3231f690 msgr2=0x562d32324190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.902+0000 7ff0a6800640 1 --2- 172.21.15.31:0/691445735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d3231f690 0x562d32324190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.902+0000 7ff0a6800640 1 -- 172.21.15.31:0/691445735 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d31f96210 con 0x562d3231eca0 2024-04-25T01:22:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.903+0000 7ff0a5e00640 1 --2- 172.21.15.31:0/691445735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d3231f690 0x562d32324190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:22:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.903+0000 7ff0a6800640 1 --2- 172.21.15.31:0/1813200381 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff08c0a2cb0 0x7ff08c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.903+0000 7ff09ea00640 1 -- 172.21.15.31:0/1813200381 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7ff094099c20 con 0x7ff0a8101100 2024-04-25T01:22:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.903+0000 7ff0a6800640 1 --2- 172.21.15.31:0/691445735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d3231eca0 0x562d3231f0c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000a5d0 tx=0x7ff0a000aa90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6d7ee0f193fea33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:22:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.905+0000 7ff0a6800640 1 --2- 172.21.15.31:0/1813200381 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff08c0a2cb0 0x7ff08c0a5170 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000af90 tx=0x7ff0a0019000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:22:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.906+0000 7ff093000640 1 -- 172.21.15.31:0/691445735 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a001a040 con 0x562d3231eca0 2024-04-25T01:22:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.906+0000 7ff093000640 1 -- 172.21.15.31:0/691445735 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a0023070 con 0x562d3231eca0 2024-04-25T01:22:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.906+0000 7ff093000640 1 -- 172.21.15.31:0/691445735 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a001eb80 con 0x562d3231eca0 2024-04-25T01:22:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.907+0000 7ff0af983740 1 -- 172.21.15.31:0/691445735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d3231eca0 msgr2=0x562d3231f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.907+0000 7ff0af983740 1 --2- 172.21.15.31:0/691445735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d3231eca0 0x562d3231f0c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000a5d0 tx=0x7ff0a000aa90 comp rx=0 tx=0).stop 2024-04-25T01:22:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.907+0000 7ff0af983740 1 -- 172.21.15.31:0/691445735 shutdown_connections 2024-04-25T01:22:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.907+0000 7ff0af983740 1 --2- 172.21.15.31:0/691445735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d32324800 0x562d32326c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.907+0000 7ff0af983740 1 --2- 172.21.15.31:0/691445735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d3231f690 0x562d32324190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.907+0000 7ff0af983740 1 --2- 172.21.15.31:0/691445735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d3231eca0 0x562d3231f0c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.907+0000 7ff0af983740 1 -- 172.21.15.31:0/691445735 >> 172.21.15.31:0/691445735 conn(0x562d3231a310 msgr2=0x562d3231a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:22:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.908+0000 7ff0af983740 1 -- 172.21.15.31:0/691445735 shutdown_connections 2024-04-25T01:22:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.908+0000 7ff0af983740 1 -- 172.21.15.31:0/691445735 wait complete. 2024-04-25T01:22:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.909+0000 7ff0af983740 1 -- start start 2024-04-25T01:22:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.909+0000 7ff0af983740 1 Processor -- start 2024-04-25T01:22:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.913+0000 7ff0af983740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d3231eca0 0x562d32339ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.913+0000 7ff0af983740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d3231f690 0x562d3233a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.913+0000 7ff0af983740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d32324800 0x562d32341420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.914+0000 7ff0a5e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d3231eca0 0x562d32339ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.914+0000 7ff0a5e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d3231eca0 0x562d32339ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:33556/0 (socket says 172.21.15.31:33556) 2024-04-25T01:22:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.914+0000 7ff0a5e00640 1 -- 172.21.15.31:0/3858130594 learned_addr learned my addr 172.21.15.31:0/3858130594 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:22:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.914+0000 7ff0af983740 1 -- 172.21.15.31:0/3858130594 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562d32338760 con 0x562d3231f690 2024-04-25T01:22:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.914+0000 7ff0af983740 1 -- 172.21.15.31:0/3858130594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x562d3223d990 con 0x562d3231eca0 2024-04-25T01:22:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.914+0000 7ff0af983740 1 -- 172.21.15.31:0/3858130594 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562d323388e0 con 0x562d32324800 2024-04-25T01:22:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.914+0000 7ff0a6800640 1 --2- 172.21.15.31:0/3858130594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d32324800 0x562d32341420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.915+0000 7ff0a6800640 1 -- 172.21.15.31:0/3858130594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d3231eca0 msgr2=0x562d32339ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.915+0000 7ff0a6800640 1 --2- 172.21.15.31:0/3858130594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d3231eca0 0x562d32339ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.915+0000 7ff0a6800640 1 -- 172.21.15.31:0/3858130594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d3231f690 msgr2=0x562d3233a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.915+0000 7ff0a6800640 1 --2- 172.21.15.31:0/3858130594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d3231f690 0x562d3233a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.915+0000 7ff0a6800640 1 -- 172.21.15.31:0/3858130594 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d32062620 con 0x562d32324800 2024-04-25T01:22:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.915+0000 7ff0a5e00640 1 --2- 172.21.15.31:0/3858130594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d3231eca0 0x562d32339ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:22:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.916+0000 7ff0af983740 10 client.0 _ll_register_callbacks cb 0x562d3223f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:22:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.917+0000 7ff0af983740 1 -- 172.21.15.31:0/3858130594 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0a0023070 con 0x562d32324800 2024-04-25T01:22:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.917+0000 7ff0a6800640 1 --2- 172.21.15.31:0/3858130594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d32324800 0x562d32341420 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0004a80 tx=0x7ff0a001fbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cf1c50a35b7c44a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:22:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.918+0000 7ff087400640 10 client.25675 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:22:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.919+0000 7ff087400640 1 -- 172.21.15.31:0/3858130594 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0036050 con 0x562d32324800 2024-04-25T01:22:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.919+0000 7ff087400640 1 -- 172.21.15.31:0/3858130594 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7ff0a0016a40 con 0x562d32324800 2024-04-25T01:22:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.919+0000 7ff087400640 1 -- 172.21.15.31:0/3858130594 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff0a002c070 con 0x562d32324800 2024-04-25T01:22:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.919+0000 7ff0af983740 10 client.25675 fetch_fsmap learned FSMap version 28 2024-04-25T01:22:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.919+0000 7ff0af983740 1 -- 172.21.15.31:0/3858130594 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({fsmap=28}) -- 0x562d323442d0 con 0x562d32324800 2024-04-25T01:22:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.921+0000 7ff087400640 1 -- 172.21.15.31:0/3858130594 <== mon.1 v2:172.21.15.169:3300/0 4 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7ff0a001d050 con 0x562d32324800 2024-04-25T01:22:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.921+0000 7ff0af983740 10 client.25675 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T01:22:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.921+0000 7ff0af983740 10 client.25675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:22:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.921+0000 7ff0af983740 20 client.25675 populate_metadata read hostname 'smithi031' 2024-04-25T01:22:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.921+0000 7ff0af983740 1 --2- 172.21.15.31:0/3858130594 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x562d323456f0 0x562d32347b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.921+0000 7ff0af983740 4 client.25675 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:22:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.921+0000 7ff0af983740 1 -- 172.21.15.31:0/3858130594 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d3228eff0 con 0x562d323456f0 2024-04-25T01:22:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.922+0000 7ff0a5e00640 1 --2- 172.21.15.31:0/3858130594 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x562d323456f0 0x562d32347b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.923+0000 7ff0a5e00640 1 --2- 172.21.15.31:0/3858130594 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x562d323456f0 0x562d32347b80 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a53b4f431cdb54 server_cookie=f4040ea31cad708d in_seq=0 out_seq=0 2024-04-25T01:22:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.923+0000 7ff087400640 10 client.25675 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:22:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.925+0000 7ff087400640 1 -- 172.21.15.31:0/3858130594 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562d3228eff0 con 0x562d323456f0 2024-04-25T01:22:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.925+0000 7ff087400640 10 client.25675 handle_command_reply: tid=0 2024-04-25T01:22:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.925+0000 7ff0af983740 1 client.25675 shutdown 2024-04-25T01:22:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.926+0000 7ff0af983740 1 -- 172.21.15.31:0/3858130594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d32324800 msgr2=0x562d32341420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.926+0000 7ff0af983740 1 --2- 172.21.15.31:0/3858130594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d32324800 0x562d32341420 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0004a80 tx=0x7ff0a001fbc0 comp rx=0 tx=0).stop 2024-04-25T01:22:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.927+0000 7ff0af983740 1 -- 172.21.15.31:0/3858130594 shutdown_connections 2024-04-25T01:22:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.927+0000 7ff0af983740 1 --2- 172.21.15.31:0/3858130594 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x562d323456f0 0x562d32347b80 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.927+0000 7ff087400640 0 client.25675 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:22:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.927+0000 7ff0af983740 1 --2- 172.21.15.31:0/3858130594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d32324800 0x562d32341420 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.927+0000 7ff0af983740 1 --2- 172.21.15.31:0/3858130594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d3231f690 0x562d3233a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.927+0000 7ff0af983740 1 --2- 172.21.15.31:0/3858130594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d3231eca0 0x562d32339ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.927+0000 7ff0af983740 1 -- 172.21.15.31:0/3858130594 >> 172.21.15.31:0/3858130594 conn(0x562d3231a310 msgr2=0x562d32319480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:22:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.928+0000 7ff0af983740 1 -- 172.21.15.31:0/3858130594 shutdown_connections 2024-04-25T01:22:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.928+0000 7ff0af983740 1 -- 172.21.15.31:0/3858130594 wait complete. 2024-04-25T01:22:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.928+0000 7ff0af983740 20 client.25675 trim_cache size 0 max 16384 2024-04-25T01:22:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.986+0000 7ff0af983740 1 Processor -- start 2024-04-25T01:22:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.986+0000 7ff0af983740 1 -- start start 2024-04-25T01:22:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.987+0000 7ff0af983740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d3231eca0 0x562d323394c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.987+0000 7ff0af983740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d3231f690 0x562d32339a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.987+0000 7ff0af983740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d32324800 0x562d32339f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.988+0000 7ff0af983740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a81a9400 con 0x562d32324800 2024-04-25T01:22:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.988+0000 7ff0af983740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a81a6c70 con 0x562d3231eca0 2024-04-25T01:22:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.988+0000 7ff0af983740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a8071c90 con 0x562d3231f690 2024-04-25T01:22:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.988+0000 7ff0a5e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d3231eca0 0x562d323394c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.989+0000 7ff0a5e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d3231eca0 0x562d323394c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:33570/0 (socket says 172.21.15.31:33570) 2024-04-25T01:22:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.989+0000 7ff0a5e00640 1 -- 172.21.15.31:0/988271503 learned_addr learned my addr 172.21.15.31:0/988271503 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:22:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.989+0000 7ff0a5400640 1 --2- 172.21.15.31:0/988271503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d3231f690 0x562d32339a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.991+0000 7ff0a5400640 1 -- 172.21.15.31:0/988271503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d3231eca0 msgr2=0x562d323394c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.991+0000 7ff0a5400640 1 --2- 172.21.15.31:0/988271503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d3231eca0 0x562d323394c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.991+0000 7ff0a5400640 1 -- 172.21.15.31:0/988271503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d32324800 msgr2=0x562d32339f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.991+0000 7ff0a5400640 1 --2- 172.21.15.31:0/988271503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d32324800 0x562d32339f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.991+0000 7ff0a5400640 1 -- 172.21.15.31:0/988271503 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d32319c00 con 0x562d3231f690 2024-04-25T01:22:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.991+0000 7ff0a5e00640 1 --2- 172.21.15.31:0/988271503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d3231eca0 0x562d323394c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:22:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.992+0000 7ff0a5400640 1 --2- 172.21.15.31:0/988271503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d3231f690 0x562d32339a00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff094004630 tx=0x7ff094097820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b4dc197ee8b7e11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:22:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.992+0000 7ff092600640 1 -- 172.21.15.31:0/988271503 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0940b8020 con 0x562d3231f690 2024-04-25T01:22:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.992+0000 7ff092600640 1 -- 172.21.15.31:0/988271503 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0940a47f0 con 0x562d3231f690 2024-04-25T01:22:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.993+0000 7ff092600640 1 -- 172.21.15.31:0/988271503 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0940b8020 con 0x562d3231f690 2024-04-25T01:22:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.994+0000 7ff0af983740 1 -- 172.21.15.31:0/988271503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d3231f690 msgr2=0x562d32339a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.994+0000 7ff0af983740 1 --2- 172.21.15.31:0/988271503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d3231f690 0x562d32339a00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff094004630 tx=0x7ff094097820 comp rx=0 tx=0).stop 2024-04-25T01:22:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.995+0000 7ff0af983740 1 -- 172.21.15.31:0/988271503 shutdown_connections 2024-04-25T01:22:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.995+0000 7ff0af983740 1 --2- 172.21.15.31:0/988271503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d32324800 0x562d32339f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.995+0000 7ff0af983740 1 --2- 172.21.15.31:0/988271503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d3231f690 0x562d32339a00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.995+0000 7ff0af983740 1 --2- 172.21.15.31:0/988271503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d3231eca0 0x562d323394c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.996+0000 7ff0af983740 1 -- 172.21.15.31:0/988271503 >> 172.21.15.31:0/988271503 conn(0x562d3231a310 msgr2=0x562d3231d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:22:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.996+0000 7ff0af983740 1 -- 172.21.15.31:0/988271503 shutdown_connections 2024-04-25T01:22:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.996+0000 7ff0af983740 1 -- 172.21.15.31:0/988271503 wait complete. 2024-04-25T01:22:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.997+0000 7ff0af983740 1 -- start start 2024-04-25T01:22:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.998+0000 7ff0af983740 1 Processor -- start 2024-04-25T01:22:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.999+0000 7ff0af983740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d3231eca0 0x562d3233e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:11.999+0000 7ff0af983740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d3231f690 0x562d3233ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.000+0000 7ff0a5e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d3231eca0 0x562d3233e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.000+0000 7ff0af983740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d32324800 0x562d3233fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.000+0000 7ff0a5400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d3231f690 0x562d3233ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.001+0000 7ff0a5400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d3231f690 0x562d3233ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48872/0 (socket says 172.21.15.31:48872) 2024-04-25T01:22:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.001+0000 7ff0a5400640 1 -- 172.21.15.31:0/228502825 learned_addr learned my addr 172.21.15.31:0/228502825 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:22:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.001+0000 7ff0a6800640 1 --2- 172.21.15.31:0/228502825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d32324800 0x562d3233fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.001+0000 7ff0af983740 1 -- 172.21.15.31:0/228502825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a8072300 con 0x562d3231f690 2024-04-25T01:22:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.001+0000 7ff0af983740 1 -- 172.21.15.31:0/228502825 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a8073770 con 0x562d32324800 2024-04-25T01:22:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.001+0000 7ff0af983740 1 -- 172.21.15.31:0/228502825 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a80729d0 con 0x562d3231eca0 2024-04-25T01:22:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.003+0000 7ff0af983740 10 client.0 _ll_register_callbacks cb 0x7ff094097e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:22:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.003+0000 7ff0a5400640 1 -- 172.21.15.31:0/228502825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d3231eca0 msgr2=0x562d3233e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.003+0000 7ff0a5400640 1 --2- 172.21.15.31:0/228502825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d3231eca0 0x562d3233e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.003+0000 7ff0a5400640 1 -- 172.21.15.31:0/228502825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d32324800 msgr2=0x562d3233fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.003+0000 7ff0a5400640 1 --2- 172.21.15.31:0/228502825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d32324800 0x562d3233fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.003+0000 7ff0a5400640 1 -- 172.21.15.31:0/228502825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d32340290 con 0x562d3231f690 2024-04-25T01:22:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.003+0000 7ff0af983740 1 -- 172.21.15.31:0/228502825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0940a47f0 con 0x562d3231f690 2024-04-25T01:22:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.003+0000 7ff0a5e00640 1 --2- 172.21.15.31:0/228502825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d3231eca0 0x562d3233e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:22:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.004+0000 7ff0a5400640 1 --2- 172.21.15.31:0/228502825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d3231f690 0x562d3233ec90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff094097fe0 tx=0x7ff09409a090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2db8e29ed3fb3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:22:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.004+0000 7ff087400640 10 client.15378 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T01:22:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.005+0000 7ff087400640 1 -- 172.21.15.31:0/228502825 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094081070 con 0x562d3231f690 2024-04-25T01:22:11.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.006+0000 7ff087400640 1 -- 172.21.15.31:0/228502825 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7ff0940ba4c0 con 0x562d3231f690 2024-04-25T01:22:11.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.006+0000 7ff087400640 1 -- 172.21.15.31:0/228502825 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff09409afd0 con 0x562d3231f690 2024-04-25T01:22:11.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.007+0000 7ff0af983740 10 client.15378 fetch_fsmap learned FSMap version 28 2024-04-25T01:22:11.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.007+0000 7ff0af983740 1 -- 172.21.15.31:0/228502825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({fsmap=28}) -- 0x562d323ac340 con 0x562d3231f690 2024-04-25T01:22:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.008+0000 7ff087400640 1 -- 172.21.15.31:0/228502825 <== mon.0 v2:172.21.15.31:3300/0 4 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7ff09409b170 con 0x562d3231f690 2024-04-25T01:22:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.008+0000 7ff0af983740 10 client.15378 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T01:22:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.008+0000 7ff0af983740 10 client.15378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:22:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.008+0000 7ff0af983740 20 client.15378 populate_metadata read hostname 'smithi031' 2024-04-25T01:22:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.008+0000 7ff0af983740 1 --2- 172.21.15.31:0/228502825 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x562d323456f0 0x562d323aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.008+0000 7ff0af983740 4 client.15378 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:22:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.008+0000 7ff0af983740 1 -- 172.21.15.31:0/228502825 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562d3231a160 con 0x562d323456f0 2024-04-25T01:22:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.010+0000 7ff0a5e00640 1 --2- 172.21.15.31:0/228502825 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x562d323456f0 0x562d323aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.011+0000 7ff0a5e00640 1 --2- 172.21.15.31:0/228502825 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x562d323456f0 0x562d323aeb70 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad54d66a0440f7a0 server_cookie=341e552316563de0 in_seq=0 out_seq=0 2024-04-25T01:22:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:12.011+0000 7ff087400640 10 client.15378 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:22:12.440 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:12 smithi169 ceph-mon[32759]: pgmap v578: 97 pgs: 97 active+clean; 13 GiB data, 103 GiB used, 970 GiB / 1.0 TiB avail; 40 KiB/s rd, 113 MiB/s wr, 16.38k op/s 2024-04-25T01:22:12.443 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:12 smithi040 ceph-mon[32518]: pgmap v578: 97 pgs: 97 active+clean; 13 GiB data, 103 GiB used, 970 GiB / 1.0 TiB avail; 40 KiB/s rd, 113 MiB/s wr, 16.38k op/s 2024-04-25T01:22:12.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:12 smithi031 ceph-mon[28053]: pgmap v578: 97 pgs: 97 active+clean; 13 GiB data, 103 GiB used, 970 GiB / 1.0 TiB avail; 40 KiB/s rd, 113 MiB/s wr, 16.38k op/s 2024-04-25T01:22:13.502 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:13 smithi031 ceph-mon[28053]: pgmap v579: 97 pgs: 97 active+clean; 13 GiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 40 KiB/s rd, 108 MiB/s wr, 15.97k op/s 2024-04-25T01:22:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.593+0000 7ff087400640 1 -- 172.21.15.31:0/228502825 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562d3231a160 con 0x562d323456f0 2024-04-25T01:22:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.593+0000 7ff087400640 10 client.15378 handle_command_reply: tid=0 2024-04-25T01:22:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.594+0000 7ff0af983740 1 client.15378 shutdown 2024-04-25T01:22:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.595+0000 7ff0af983740 1 -- 172.21.15.31:0/228502825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d3231f690 msgr2=0x562d3233ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.595+0000 7ff0af983740 1 --2- 172.21.15.31:0/228502825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d3231f690 0x562d3233ec90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff094097fe0 tx=0x7ff09409a090 comp rx=0 tx=0).stop 2024-04-25T01:22:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.595+0000 7ff0af983740 1 -- 172.21.15.31:0/228502825 shutdown_connections 2024-04-25T01:22:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.595+0000 7ff0af983740 1 --2- 172.21.15.31:0/228502825 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x562d323456f0 0x562d323aeb70 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.595+0000 7ff0af983740 1 --2- 172.21.15.31:0/228502825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562d32324800 0x562d3233fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.595+0000 7ff0af983740 1 --2- 172.21.15.31:0/228502825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d3231f690 0x562d3233ec90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.595+0000 7ff0af983740 1 --2- 172.21.15.31:0/228502825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d3231eca0 0x562d3233e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.595+0000 7ff0af983740 1 -- 172.21.15.31:0/228502825 >> 172.21.15.31:0/228502825 conn(0x562d3231a310 msgr2=0x562d32318bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:22:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.596+0000 7ff087400640 0 client.15378 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:22:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.596+0000 7ff0af983740 1 -- 172.21.15.31:0/228502825 shutdown_connections 2024-04-25T01:22:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.596+0000 7ff0af983740 1 -- 172.21.15.31:0/228502825 wait complete. 2024-04-25T01:22:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.596+0000 7ff0af983740 20 client.15378 trim_cache size 0 max 16384 2024-04-25T01:22:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:22:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:22:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:22:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:22:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.598+0000 7ff0aca00640 1 -- 172.21.15.31:0/1813200381 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff08c0a2cb0 msgr2=0x7ff08c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.598+0000 7ff0aca00640 1 --2- 172.21.15.31:0/1813200381 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff08c0a2cb0 0x7ff08c0a5170 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000af90 tx=0x7ff0a0019000 comp rx=0 tx=0).stop 2024-04-25T01:22:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.598+0000 7ff0aca00640 1 -- 172.21.15.31:0/1813200381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0a8101100 msgr2=0x7ff0a819a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.598+0000 7ff0aca00640 1 --2- 172.21.15.31:0/1813200381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0a8101100 0x7ff0a819a1c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff09400e9c0 tx=0x7ff09400ee80 comp rx=0 tx=0).stop 2024-04-25T01:22:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.598+0000 7ff0aca00640 1 -- 172.21.15.31:0/1813200381 shutdown_connections 2024-04-25T01:22:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.598+0000 7ff0aca00640 1 --2- 172.21.15.31:0/1813200381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0a8105e00 0x7ff0a81a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.598+0000 7ff0aca00640 1 --2- 172.21.15.31:0/1813200381 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff08c0a2cb0 0x7ff08c0a5170 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.598+0000 7ff0aca00640 1 --2- 172.21.15.31:0/1813200381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0a8101100 0x7ff0a819a1c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.598+0000 7ff0aca00640 1 --2- 172.21.15.31:0/1813200381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0a8068110 0x7ff0a8199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.598+0000 7ff0aca00640 1 -- 172.21.15.31:0/1813200381 >> 172.21.15.31:0/1813200381 conn(0x7ff0a80faf60 msgr2=0x7ff0a80fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:22:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.599+0000 7ff0aca00640 1 -- 172.21.15.31:0/1813200381 shutdown_connections 2024-04-25T01:22:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:13.599+0000 7ff0aca00640 1 -- 172.21.15.31:0/1813200381 wait complete. 2024-04-25T01:22:13.689 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:13 smithi169 ceph-mon[32759]: pgmap v579: 97 pgs: 97 active+clean; 13 GiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 40 KiB/s rd, 108 MiB/s wr, 15.97k op/s 2024-04-25T01:22:13.692 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:13 smithi040 ceph-mon[32518]: pgmap v579: 97 pgs: 97 active+clean; 13 GiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 40 KiB/s rd, 108 MiB/s wr, 15.97k op/s 2024-04-25T01:22:15.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:15 smithi169 ceph-mon[32759]: pgmap v580: 97 pgs: 97 active+clean; 13 GiB data, 103 GiB used, 970 GiB / 1.0 TiB avail; 40 KiB/s rd, 101 MiB/s wr, 15.14k op/s 2024-04-25T01:22:15.693 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:15 smithi040 ceph-mon[32518]: pgmap v580: 97 pgs: 97 active+clean; 13 GiB data, 103 GiB used, 970 GiB / 1.0 TiB avail; 40 KiB/s rd, 101 MiB/s wr, 15.14k op/s 2024-04-25T01:22:15.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:15 smithi031 ceph-mon[28053]: pgmap v580: 97 pgs: 97 active+clean; 13 GiB data, 103 GiB used, 970 GiB / 1.0 TiB avail; 40 KiB/s rd, 101 MiB/s wr, 15.14k op/s 2024-04-25T01:22:17.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:17 smithi169 ceph-mon[32759]: pgmap v581: 97 pgs: 97 active+clean; 13 GiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 62 KiB/s rd, 128 MiB/s wr, 19.08k op/s 2024-04-25T01:22:17.693 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:17 smithi040 ceph-mon[32518]: pgmap v581: 97 pgs: 97 active+clean; 13 GiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 62 KiB/s rd, 128 MiB/s wr, 19.08k op/s 2024-04-25T01:22:17.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:17 smithi031 ceph-mon[28053]: pgmap v581: 97 pgs: 97 active+clean; 13 GiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 62 KiB/s rd, 128 MiB/s wr, 19.08k op/s 2024-04-25T01:22:19.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:19 smithi169 ceph-mon[32759]: pgmap v582: 97 pgs: 97 active+clean; 13 GiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 46 KiB/s rd, 108 MiB/s wr, 15.80k op/s 2024-04-25T01:22:19.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:19 smithi169 ceph-mon[32759]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:22:19.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:19 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:22:19.692 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:19 smithi040 ceph-mon[32518]: pgmap v582: 97 pgs: 97 active+clean; 13 GiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 46 KiB/s rd, 108 MiB/s wr, 15.80k op/s 2024-04-25T01:22:19.693 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:19 smithi040 ceph-mon[32518]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:22:19.693 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:19 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:22:19.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:19 smithi031 ceph-mon[28053]: pgmap v582: 97 pgs: 97 active+clean; 13 GiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 46 KiB/s rd, 108 MiB/s wr, 15.80k op/s 2024-04-25T01:22:19.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:19 smithi031 ceph-mon[28053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:22:19.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:19 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:22:20.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:20 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:22:20.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:20 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:22:20.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:20 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:22:20.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:20 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:22:20.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:20 smithi169 ceph-mon[32759]: Adjusting osd_memory_target on smithi040 to 177.2M 2024-04-25T01:22:20.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:20 smithi169 ceph-mon[32759]: Unable to set osd_memory_target on smithi040 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T01:22:20.693 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:20 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:22:20.693 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:20 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:22:20.693 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:20 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:22:20.693 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:20 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:22:20.693 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:20 smithi040 ceph-mon[32518]: Adjusting osd_memory_target on smithi040 to 177.2M 2024-04-25T01:22:20.693 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:20 smithi040 ceph-mon[32518]: Unable to set osd_memory_target on smithi040 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T01:22:20.782 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:20 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:22:20.782 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:20 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:22:20.783 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:20 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:22:20.783 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:20 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:22:20.783 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:20 smithi031 ceph-mon[28053]: Adjusting osd_memory_target on smithi040 to 177.2M 2024-04-25T01:22:20.783 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:20 smithi031 ceph-mon[28053]: Unable to set osd_memory_target on smithi040 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T01:22:21.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:21 smithi169 ceph-mon[32759]: pgmap v583: 97 pgs: 97 active+clean; 13 GiB data, 105 GiB used, 968 GiB / 1.0 TiB avail; 57 KiB/s rd, 108 MiB/s wr, 15.79k op/s 2024-04-25T01:22:21.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:21 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:22:21.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:21 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:22:21.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:21 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:22:21.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:21 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:22:21.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:21 smithi169 ceph-mon[32759]: Adjusting osd_memory_target on smithi031 to 177.5M 2024-04-25T01:22:21.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:21 smithi169 ceph-mon[32759]: Unable to set osd_memory_target on smithi031 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T01:22:21.690 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:21 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:22:21.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:21 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:22:21.691 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:21 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:22:21.693 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:21 smithi040 ceph-mon[32518]: pgmap v583: 97 pgs: 97 active+clean; 13 GiB data, 105 GiB used, 968 GiB / 1.0 TiB avail; 57 KiB/s rd, 108 MiB/s wr, 15.79k op/s 2024-04-25T01:22:21.693 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:21 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:22:21.693 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:21 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:22:21.693 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:21 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:22:21.693 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:21 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:22:21.693 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:21 smithi040 ceph-mon[32518]: Adjusting osd_memory_target on smithi031 to 177.5M 2024-04-25T01:22:21.693 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:21 smithi040 ceph-mon[32518]: Unable to set osd_memory_target on smithi031 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T01:22:21.693 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:21 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:22:21.693 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:21 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:22:21.693 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:21 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:22:21.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:21 smithi031 ceph-mon[28053]: pgmap v583: 97 pgs: 97 active+clean; 13 GiB data, 105 GiB used, 968 GiB / 1.0 TiB avail; 57 KiB/s rd, 108 MiB/s wr, 15.79k op/s 2024-04-25T01:22:21.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:21 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:22:21.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:21 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:22:21.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:21 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:22:21.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:21 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:22:21.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:21 smithi031 ceph-mon[28053]: Adjusting osd_memory_target on smithi031 to 177.5M 2024-04-25T01:22:21.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:21 smithi031 ceph-mon[28053]: Unable to set osd_memory_target on smithi031 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T01:22:21.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:21 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:22:21.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:21 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:22:21.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:21 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:22:23.693 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:23 smithi040 ceph-mon[32518]: pgmap v584: 97 pgs: 97 active+clean; 13 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 56 KiB/s rd, 118 MiB/s wr, 17.07k op/s 2024-04-25T01:22:23.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:23 smithi031 ceph-mon[28053]: pgmap v584: 97 pgs: 97 active+clean; 13 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 56 KiB/s rd, 118 MiB/s wr, 17.07k op/s 2024-04-25T01:22:23.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:23 smithi169 ceph-mon[32759]: pgmap v584: 97 pgs: 97 active+clean; 13 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 56 KiB/s rd, 118 MiB/s wr, 17.07k op/s 2024-04-25T01:22:25.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:25 smithi031 ceph-mon[28053]: pgmap v585: 97 pgs: 97 active+clean; 13 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 42 KiB/s rd, 105 MiB/s wr, 15.12k op/s 2024-04-25T01:22:25.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:25 smithi169 ceph-mon[32759]: pgmap v585: 97 pgs: 97 active+clean; 13 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 42 KiB/s rd, 105 MiB/s wr, 15.12k op/s 2024-04-25T01:22:25.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:25 smithi040 ceph-mon[32518]: pgmap v585: 97 pgs: 97 active+clean; 13 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 42 KiB/s rd, 105 MiB/s wr, 15.12k op/s 2024-04-25T01:22:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.909+0000 7f5b8b200640 1 -- 172.21.15.31:0/1095787012 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b84108de0 msgr2=0x7f5b84109260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.909+0000 7f5b8b200640 1 --2- 172.21.15.31:0/1095787012 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b84108de0 0x7f5b84109260 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009860 tx=0x7f5b7002a360 comp rx=0 tx=0).stop 2024-04-25T01:22:26.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.911+0000 7f5b8b200640 1 -- 172.21.15.31:0/1095787012 shutdown_connections 2024-04-25T01:22:26.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.911+0000 7f5b8b200640 1 --2- 172.21.15.31:0/1095787012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b841097a0 0x7f5b84110030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:26.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.911+0000 7f5b8b200640 1 --2- 172.21.15.31:0/1095787012 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b84108de0 0x7f5b84109260 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:26.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.911+0000 7f5b8b200640 1 --2- 172.21.15.31:0/1095787012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b84107bf0 0x7f5b84108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:26.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.911+0000 7f5b8b200640 1 -- 172.21.15.31:0/1095787012 >> 172.21.15.31:0/1095787012 conn(0x7f5b84076b90 msgr2=0x7f5b84078fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:22:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.916+0000 7f5b8b200640 1 -- 172.21.15.31:0/1095787012 shutdown_connections 2024-04-25T01:22:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.916+0000 7f5b8b200640 1 -- 172.21.15.31:0/1095787012 wait complete. 2024-04-25T01:22:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.920+0000 7f5b8b200640 1 Processor -- start 2024-04-25T01:22:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.920+0000 7f5b8b200640 1 -- start start 2024-04-25T01:22:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.920+0000 7f5b8b200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b84107bf0 0x7f5b8406b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.920+0000 7f5b8b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b84108de0 0x7f5b8406bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.920+0000 7f5b8b200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b841097a0 0x7f5b84072b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.921+0000 7f5b83400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b84108de0 0x7f5b8406bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.921+0000 7f5b83400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b84108de0 0x7f5b8406bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:56928/0 (socket says 172.21.15.31:56928) 2024-04-25T01:22:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.921+0000 7f5b83400640 1 -- 172.21.15.31:0/4182727353 learned_addr learned my addr 172.21.15.31:0/4182727353 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:22:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.921+0000 7f5b8b200640 1 -- 172.21.15.31:0/4182727353 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841138a0 con 0x7f5b841097a0 2024-04-25T01:22:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.921+0000 7f5b8b200640 1 -- 172.21.15.31:0/4182727353 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5b84113720 con 0x7f5b84107bf0 2024-04-25T01:22:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.921+0000 7f5b8b200640 1 -- 172.21.15.31:0/4182727353 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5b84113a20 con 0x7f5b84108de0 2024-04-25T01:22:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.921+0000 7f5b83e00640 1 --2- 172.21.15.31:0/4182727353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b84107bf0 0x7f5b8406b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.922+0000 7f5b83400640 1 -- 172.21.15.31:0/4182727353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b84107bf0 msgr2=0x7f5b8406b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.922+0000 7f5b83400640 1 --2- 172.21.15.31:0/4182727353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b84107bf0 0x7f5b8406b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.922+0000 7f5b83400640 1 -- 172.21.15.31:0/4182727353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b841097a0 msgr2=0x7f5b84072b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.922+0000 7f5b83400640 1 --2- 172.21.15.31:0/4182727353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b841097a0 0x7f5b84072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.922+0000 7f5b83400640 1 -- 172.21.15.31:0/4182727353 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b840731c0 con 0x7f5b84108de0 2024-04-25T01:22:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.922+0000 7f5b83400640 1 --2- 172.21.15.31:0/4182727353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b84108de0 0x7f5b8406bae0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5b7002a8d0 tx=0x7f5b70033d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:22:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.923+0000 7f5b80c00640 1 -- 172.21.15.31:0/4182727353 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b70037040 con 0x7f5b84108de0 2024-04-25T01:22:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.923+0000 7f5b8b200640 1 -- 172.21.15.31:0/4182727353 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b84073450 con 0x7f5b84108de0 2024-04-25T01:22:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.923+0000 7f5b8b200640 1 -- 172.21.15.31:0/4182727353 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b8419d8d0 con 0x7f5b84108de0 2024-04-25T01:22:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.923+0000 7f5b80c00640 1 -- 172.21.15.31:0/4182727353 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b70040070 con 0x7f5b84108de0 2024-04-25T01:22:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.923+0000 7f5b80c00640 1 -- 172.21.15.31:0/4182727353 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b70005620 con 0x7f5b84108de0 2024-04-25T01:22:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.939+0000 7f5b8dfd3740 1 Processor -- start 2024-04-25T01:22:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.941+0000 7f5b80c00640 1 -- 172.21.15.31:0/4182727353 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5b70030790 con 0x7f5b84108de0 2024-04-25T01:22:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.942+0000 7f5b8dfd3740 1 -- start start 2024-04-25T01:22:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.947+0000 7f5b8dfd3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560adbcdfe50 0x560adbce0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.947+0000 7f5b8dfd3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbce0840 0x560adbce5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.947+0000 7f5b8dfd3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560adbce59b0 0x560adbce7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.947+0000 7f5b8dfd3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560adbc42ab0 con 0x560adbce59b0 2024-04-25T01:22:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.947+0000 7f5b8dfd3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560adbbeb640 con 0x560adbcdfe50 2024-04-25T01:22:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.947+0000 7f5b8dfd3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560adbc3ea70 con 0x560adbce0840 2024-04-25T01:22:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.947+0000 7f5b83e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbce0840 0x560adbce5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.948+0000 7f5b83e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbce0840 0x560adbce5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:56930/0 (socket says 172.21.15.31:56930) 2024-04-25T01:22:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.948+0000 7f5b83e00640 1 -- 172.21.15.31:0/2952608575 learned_addr learned my addr 172.21.15.31:0/2952608575 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:22:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.948+0000 7f5b80c00640 1 --2- 172.21.15.31:0/4182727353 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5b6c0a2c40 0x7f5b6c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.948+0000 7f5b80c00640 1 -- 172.21.15.31:0/4182727353 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f5b700ee9f0 con 0x7f5b84108de0 2024-04-25T01:22:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.947+0000 7f5b89200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560adbcdfe50 0x560adbce0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.949+0000 7f5b83400640 1 --2- 172.21.15.31:0/2952608575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560adbce59b0 0x560adbce7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.950+0000 7f5b83e00640 1 -- 172.21.15.31:0/2952608575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560adbcdfe50 msgr2=0x560adbce0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.950+0000 7f5b83e00640 1 --2- 172.21.15.31:0/2952608575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560adbcdfe50 0x560adbce0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.950+0000 7f5b83e00640 1 -- 172.21.15.31:0/2952608575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560adbce59b0 msgr2=0x560adbce7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.950+0000 7f5b83e00640 1 --2- 172.21.15.31:0/2952608575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560adbce59b0 0x560adbce7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.950+0000 7f5b83e00640 1 -- 172.21.15.31:0/2952608575 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560adb957210 con 0x560adbce0840 2024-04-25T01:22:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.952+0000 7f5b83e00640 1 --2- 172.21.15.31:0/2952608575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbce0840 0x560adbce5340 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800bc90 tx=0x7f5b7800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8bda4c792271691 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:22:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.952+0000 7f5b89200640 1 --2- 172.21.15.31:0/4182727353 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5b6c0a2c40 0x7f5b6c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.953+0000 7f5b75600640 1 -- 172.21.15.31:0/2952608575 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78015070 con 0x560adbce0840 2024-04-25T01:22:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.953+0000 7f5b75600640 1 -- 172.21.15.31:0/2952608575 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b780035f0 con 0x560adbce0840 2024-04-25T01:22:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.953+0000 7f5b75600640 1 -- 172.21.15.31:0/2952608575 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b780116a0 con 0x560adbce0840 2024-04-25T01:22:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.954+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/2952608575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbce0840 msgr2=0x560adbce5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.954+0000 7f5b8dfd3740 1 --2- 172.21.15.31:0/2952608575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbce0840 0x560adbce5340 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800bc90 tx=0x7f5b7800f730 comp rx=0 tx=0).stop 2024-04-25T01:22:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.954+0000 7f5b89200640 1 --2- 172.21.15.31:0/4182727353 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5b6c0a2c40 0x7f5b6c0a5100 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00afd0 tx=0x7f5b7c0012e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:22:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.959+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/2952608575 shutdown_connections 2024-04-25T01:22:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.959+0000 7f5b8dfd3740 1 --2- 172.21.15.31:0/2952608575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560adbce59b0 0x560adbce7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.959+0000 7f5b8dfd3740 1 --2- 172.21.15.31:0/2952608575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbce0840 0x560adbce5340 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.959+0000 7f5b8dfd3740 1 --2- 172.21.15.31:0/2952608575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560adbcdfe50 0x560adbce0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.959+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/2952608575 >> 172.21.15.31:0/2952608575 conn(0x560adbcdb4c0 msgr2=0x560adbcdb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:22:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.961+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/2952608575 shutdown_connections 2024-04-25T01:22:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.961+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/2952608575 wait complete. 2024-04-25T01:22:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.961+0000 7f5b8dfd3740 1 -- start start 2024-04-25T01:22:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.962+0000 7f5b8dfd3740 1 Processor -- start 2024-04-25T01:22:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.962+0000 7f5b8dfd3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560adbcdfe50 0x560adbcea0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.962+0000 7f5b8dfd3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560adbce0840 0x560adbcea5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.962+0000 7f5b8dfd3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbce59b0 0x560adbcf1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.963+0000 7f5b83400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560adbce0840 0x560adbcea5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.963+0000 7f5b83400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560adbce0840 0x560adbcea5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:57136/0 (socket says 172.21.15.31:57136) 2024-04-25T01:22:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.963+0000 7f5b83400640 1 -- 172.21.15.31:0/3449442092 learned_addr learned my addr 172.21.15.31:0/3449442092 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:22:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.963+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/3449442092 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560adbce8960 con 0x560adbcdfe50 2024-04-25T01:22:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.963+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/3449442092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560adbbfe990 con 0x560adbce0840 2024-04-25T01:22:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.963+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/3449442092 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560adbce8ae0 con 0x560adbce59b0 2024-04-25T01:22:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.963+0000 7f5b83e00640 1 --2- 172.21.15.31:0/3449442092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560adbcdfe50 0x560adbcea0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.963+0000 7f5b8dfd3740 10 client.0 _ll_register_callbacks cb 0x560adbc00950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:22:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.963+0000 7f5b89200640 1 --2- 172.21.15.31:0/3449442092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbce59b0 0x560adbcf1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.964+0000 7f5b89200640 1 -- 172.21.15.31:0/3449442092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560adbce0840 msgr2=0x560adbcea5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.964+0000 7f5b89200640 1 --2- 172.21.15.31:0/3449442092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560adbce0840 0x560adbcea5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.964+0000 7f5b89200640 1 -- 172.21.15.31:0/3449442092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560adbcdfe50 msgr2=0x560adbcea0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.964+0000 7f5b89200640 1 --2- 172.21.15.31:0/3449442092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560adbcdfe50 0x560adbcea0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.964+0000 7f5b89200640 1 -- 172.21.15.31:0/3449442092 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560adba23620 con 0x560adbce59b0 2024-04-25T01:22:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.964+0000 7f5b83e00640 1 --2- 172.21.15.31:0/3449442092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560adbcdfe50 0x560adbcea0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:22:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.964+0000 7f5b89200640 1 --2- 172.21.15.31:0/3449442092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbce59b0 0x560adbcf1620 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c001440 tx=0x7f5b7c00a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=981b2f23e0fd73f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:22:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.964+0000 7f5b6a000640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:22:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.965+0000 7f5b6a000640 1 -- 172.21.15.31:0/3449442092 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c019070 con 0x560adbce59b0 2024-04-25T01:22:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.965+0000 7f5b6a000640 1 -- 172.21.15.31:0/3449442092 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f5b7c011dc0 con 0x560adbce59b0 2024-04-25T01:22:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.965+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/3449442092 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b780035f0 con 0x560adbce59b0 2024-04-25T01:22:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.966+0000 7f5b6a000640 1 -- 172.21.15.31:0/3449442092 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5b7c02c450 con 0x560adbce59b0 2024-04-25T01:22:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.966+0000 7f5b8dfd3740 10 client.25693 fetch_fsmap learned FSMap version 29 2024-04-25T01:22:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.966+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/3449442092 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560adbcf4670 con 0x560adbce59b0 2024-04-25T01:22:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.966+0000 7f5b6a000640 1 -- 172.21.15.31:0/3449442092 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b780035f0 con 0x560adbce59b0 2024-04-25T01:22:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.970+0000 7f5b6a000640 1 -- 172.21.15.31:0/3449442092 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 29) ==== 8342+0+0 (secure 0 0 0) 0x7f5b7c022690 con 0x560adbce59b0 2024-04-25T01:22:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.970+0000 7f5b8dfd3740 10 client.25693 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T01:22:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.970+0000 7f5b8dfd3740 10 client.25693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:22:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.970+0000 7f5b8dfd3740 20 client.25693 populate_metadata read hostname 'smithi031' 2024-04-25T01:22:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.970+0000 7f5b8dfd3740 1 --2- 172.21.15.31:0/3449442092 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560adbcf5e00 0x560adbcf8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.970+0000 7f5b8dfd3740 4 client.25693 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:22:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.970+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/3449442092 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560adbc4fff0 con 0x560adbcf5e00 2024-04-25T01:22:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.971+0000 7f5b83e00640 1 --2- 172.21.15.31:0/3449442092 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560adbcf5e00 0x560adbcf8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.975+0000 7f5b6a000640 10 client.25693 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:22:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.975+0000 7f5b83e00640 1 --2- 172.21.15.31:0/3449442092 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560adbcf5e00 0x560adbcf8270 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6957e1246b9e765a server_cookie=9307e5198a86a83b in_seq=0 out_seq=0 2024-04-25T01:22:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.976+0000 7f5b6a000640 1 -- 172.21.15.31:0/3449442092 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560adbc4fff0 con 0x560adbcf5e00 2024-04-25T01:22:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.976+0000 7f5b6a000640 10 client.25693 handle_command_reply: tid=0 2024-04-25T01:22:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.980+0000 7f5b8dfd3740 1 client.25693 shutdown 2024-04-25T01:22:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.981+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/3449442092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbce59b0 msgr2=0x560adbcf1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.982+0000 7f5b8dfd3740 1 --2- 172.21.15.31:0/3449442092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbce59b0 0x560adbcf1620 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c001440 tx=0x7f5b7c00a560 comp rx=0 tx=0).stop 2024-04-25T01:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.982+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/3449442092 shutdown_connections 2024-04-25T01:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.982+0000 7f5b8dfd3740 1 --2- 172.21.15.31:0/3449442092 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560adbcf5e00 0x560adbcf8270 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.982+0000 7f5b8dfd3740 1 --2- 172.21.15.31:0/3449442092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbce59b0 0x560adbcf1620 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.982+0000 7f5b8dfd3740 1 --2- 172.21.15.31:0/3449442092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560adbce0840 0x560adbcea5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.982+0000 7f5b6a000640 0 client.25693 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.982+0000 7f5b8dfd3740 1 --2- 172.21.15.31:0/3449442092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560adbcdfe50 0x560adbcea0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.982+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/3449442092 >> 172.21.15.31:0/3449442092 conn(0x560adbcdb4c0 msgr2=0x560adbcda650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.982+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/3449442092 shutdown_connections 2024-04-25T01:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.982+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/3449442092 wait complete. 2024-04-25T01:22:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:26.982+0000 7f5b8dfd3740 20 client.25693 trim_cache size 0 max 16384 2024-04-25T01:22:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.033+0000 7f5b8dfd3740 1 Processor -- start 2024-04-25T01:22:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.033+0000 7f5b8dfd3740 1 -- start start 2024-04-25T01:22:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.034+0000 7f5b8dfd3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbcdfe50 0x560adbcde3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.034+0000 7f5b8dfd3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560adbce0840 0x560adbcf15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.034+0000 7f5b8dfd3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560adbce59b0 0x560adbcde910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.034+0000 7f5b8dfd3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5b64010c20 con 0x560adbce0840 2024-04-25T01:22:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.034+0000 7f5b8dfd3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6400e4c0 con 0x560adbce59b0 2024-04-25T01:22:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.034+0000 7f5b8dfd3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6400c310 con 0x560adbcdfe50 2024-04-25T01:22:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.034+0000 7f5b83e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbcdfe50 0x560adbcde3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.034+0000 7f5b83e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbcdfe50 0x560adbcde3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:56944/0 (socket says 172.21.15.31:56944) 2024-04-25T01:22:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.034+0000 7f5b83e00640 1 -- 172.21.15.31:0/2687587775 learned_addr learned my addr 172.21.15.31:0/2687587775 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:22:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.035+0000 7f5b89200640 1 --2- 172.21.15.31:0/2687587775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560adbce59b0 0x560adbcde910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.035+0000 7f5b83e00640 1 -- 172.21.15.31:0/2687587775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560adbce59b0 msgr2=0x560adbcde910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.035+0000 7f5b83e00640 1 --2- 172.21.15.31:0/2687587775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560adbce59b0 0x560adbcde910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.035+0000 7f5b83e00640 1 -- 172.21.15.31:0/2687587775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560adbce0840 msgr2=0x560adbcf15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.035+0000 7f5b83e00640 1 --2- 172.21.15.31:0/2687587775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560adbce0840 0x560adbcf15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.035+0000 7f5b83e00640 1 -- 172.21.15.31:0/2687587775 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560adbce1510 con 0x560adbcdfe50 2024-04-25T01:22:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.036+0000 7f5b83e00640 1 --2- 172.21.15.31:0/2687587775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbcdfe50 0x560adbcde3d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800bbb0 tx=0x7f5b7800bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f66042e27a21756 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:22:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.036+0000 7f5b74c00640 1 -- 172.21.15.31:0/2687587775 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78015040 con 0x560adbcdfe50 2024-04-25T01:22:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.037+0000 7f5b74c00640 1 -- 172.21.15.31:0/2687587775 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b78004680 con 0x560adbcdfe50 2024-04-25T01:22:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.037+0000 7f5b74c00640 1 -- 172.21.15.31:0/2687587775 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78015040 con 0x560adbcdfe50 2024-04-25T01:22:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.038+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/2687587775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbcdfe50 msgr2=0x560adbcde3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.038+0000 7f5b8dfd3740 1 --2- 172.21.15.31:0/2687587775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbcdfe50 0x560adbcde3d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800bbb0 tx=0x7f5b7800bda0 comp rx=0 tx=0).stop 2024-04-25T01:22:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.038+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/2687587775 shutdown_connections 2024-04-25T01:22:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.039+0000 7f5b8dfd3740 1 --2- 172.21.15.31:0/2687587775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560adbce59b0 0x560adbcde910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.039+0000 7f5b8dfd3740 1 --2- 172.21.15.31:0/2687587775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560adbce0840 0x560adbcf15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.039+0000 7f5b8dfd3740 1 --2- 172.21.15.31:0/2687587775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbcdfe50 0x560adbcde3d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.039+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/2687587775 >> 172.21.15.31:0/2687587775 conn(0x560adbcdb4c0 msgr2=0x560adbcdb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:22:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.039+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/2687587775 shutdown_connections 2024-04-25T01:22:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.039+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/2687587775 wait complete. 2024-04-25T01:22:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.040+0000 7f5b8dfd3740 1 -- start start 2024-04-25T01:22:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.040+0000 7f5b8dfd3740 1 Processor -- start 2024-04-25T01:22:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.042+0000 7f5b8dfd3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560adbcdfe50 0x560adbcef3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.042+0000 7f5b8dfd3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbce0840 0x560adbcef900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.042+0000 7f5b8dfd3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560adbce59b0 0x560adbcf0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.042+0000 7f5b83400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbce0840 0x560adbcef900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.042+0000 7f5b83400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbce0840 0x560adbcef900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:56950/0 (socket says 172.21.15.31:56950) 2024-04-25T01:22:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.042+0000 7f5b83400640 1 -- 172.21.15.31:0/263443716 learned_addr learned my addr 172.21.15.31:0/263443716 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:22:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.042+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/263443716 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6400c980 con 0x560adbce59b0 2024-04-25T01:22:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.042+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/263443716 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6400ddf0 con 0x560adbcdfe50 2024-04-25T01:22:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.042+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/263443716 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6400d050 con 0x560adbce0840 2024-04-25T01:22:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.042+0000 7f5b8dfd3740 10 client.0 _ll_register_callbacks cb 0x7f5b7c00dae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:22:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.043+0000 7f5b83e00640 1 --2- 172.21.15.31:0/263443716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560adbcdfe50 0x560adbcef3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.043+0000 7f5b89200640 1 --2- 172.21.15.31:0/263443716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560adbce59b0 0x560adbcf0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.043+0000 7f5b83400640 1 -- 172.21.15.31:0/263443716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560adbcdfe50 msgr2=0x560adbcef3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.043+0000 7f5b83400640 1 --2- 172.21.15.31:0/263443716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560adbcdfe50 0x560adbcef3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.043+0000 7f5b83400640 1 -- 172.21.15.31:0/263443716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560adbce59b0 msgr2=0x560adbcf0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.043+0000 7f5b83400640 1 --2- 172.21.15.31:0/263443716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560adbce59b0 0x560adbcf0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.043+0000 7f5b83400640 1 -- 172.21.15.31:0/263443716 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560adbcf0e60 con 0x560adbce0840 2024-04-25T01:22:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.044+0000 7f5b6a000640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:22:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.044+0000 7f5b83400640 1 --2- 172.21.15.31:0/263443716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbce0840 0x560adbcef900 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5b700ec5d0 tx=0x7f5b700bad00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fc8fc68a9432405 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:22:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.044+0000 7f5b6a000640 1 -- 172.21.15.31:0/263443716 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b700c3cc0 con 0x560adbce0840 2024-04-25T01:22:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.044+0000 7f5b6a000640 1 -- 172.21.15.31:0/263443716 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f5b700db0b0 con 0x560adbce0840 2024-04-25T01:22:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.044+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/263443716 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b78004680 con 0x560adbce0840 2024-04-25T01:22:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.044+0000 7f5b6a000640 1 -- 172.21.15.31:0/263443716 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5b700bb9e0 con 0x560adbce0840 2024-04-25T01:22:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.044+0000 7f5b8dfd3740 10 client.25699 fetch_fsmap learned FSMap version 29 2024-04-25T01:22:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.044+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/263443716 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560adbd5caf0 con 0x560adbce0840 2024-04-25T01:22:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.045+0000 7f5b6a000640 1 -- 172.21.15.31:0/263443716 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b78004680 con 0x560adbce0840 2024-04-25T01:22:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.045+0000 7f5b6a000640 1 -- 172.21.15.31:0/263443716 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 29) ==== 8342+0+0 (secure 0 0 0) 0x7f5b7003a070 con 0x560adbce0840 2024-04-25T01:22:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.045+0000 7f5b8dfd3740 10 client.25699 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T01:22:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.045+0000 7f5b8dfd3740 10 client.25699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:22:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.045+0000 7f5b8dfd3740 20 client.25699 populate_metadata read hostname 'smithi031' 2024-04-25T01:22:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.045+0000 7f5b8dfd3740 1 --2- 172.21.15.31:0/263443716 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560adbd5d570 0x560adbd5f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.045+0000 7f5b8dfd3740 4 client.25699 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:22:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.045+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/263443716 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560adbce5720 con 0x560adbd5d570 2024-04-25T01:22:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.045+0000 7f5b83e00640 1 --2- 172.21.15.31:0/263443716 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560adbd5d570 0x560adbd5f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.046+0000 7f5b6a000640 10 client.25699 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:22:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.047+0000 7f5b83e00640 1 --2- 172.21.15.31:0/263443716 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560adbd5d570 0x560adbd5f990 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=723475235b1cc427 server_cookie=8674993c88209673 in_seq=0 out_seq=0 2024-04-25T01:22:27.740 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:27 smithi169 ceph-mon[32759]: pgmap v586: 97 pgs: 97 active+clean; 13 GiB data, 107 GiB used, 966 GiB / 1.0 TiB avail; 53 KiB/s rd, 119 MiB/s wr, 17.07k op/s 2024-04-25T01:22:27.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:27 smithi031 ceph-mon[28053]: pgmap v586: 97 pgs: 97 active+clean; 13 GiB data, 107 GiB used, 966 GiB / 1.0 TiB avail; 53 KiB/s rd, 119 MiB/s wr, 17.07k op/s 2024-04-25T01:22:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.823+0000 7f5b6a000640 1 -- 172.21.15.31:0/263443716 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560adbce5720 con 0x560adbd5d570 2024-04-25T01:22:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.823+0000 7f5b6a000640 10 client.25699 handle_command_reply: tid=0 2024-04-25T01:22:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.823+0000 7f5b8dfd3740 1 client.25699 shutdown 2024-04-25T01:22:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.824+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/263443716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbce0840 msgr2=0x560adbcef900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.824+0000 7f5b8dfd3740 1 --2- 172.21.15.31:0/263443716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbce0840 0x560adbcef900 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5b700ec5d0 tx=0x7f5b700bad00 comp rx=0 tx=0).stop 2024-04-25T01:22:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.824+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/263443716 shutdown_connections 2024-04-25T01:22:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.824+0000 7f5b8dfd3740 1 --2- 172.21.15.31:0/263443716 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560adbd5d570 0x560adbd5f990 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.824+0000 7f5b6a000640 0 client.25699 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:22:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.824+0000 7f5b8dfd3740 1 --2- 172.21.15.31:0/263443716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560adbce59b0 0x560adbcf0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.824+0000 7f5b8dfd3740 1 --2- 172.21.15.31:0/263443716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560adbce0840 0x560adbcef900 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.824+0000 7f5b8dfd3740 1 --2- 172.21.15.31:0/263443716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560adbcdfe50 0x560adbcef3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.824+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/263443716 >> 172.21.15.31:0/263443716 conn(0x560adbcdb4c0 msgr2=0x560adbcde3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:22:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.825+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/263443716 shutdown_connections 2024-04-25T01:22:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.825+0000 7f5b8dfd3740 1 -- 172.21.15.31:0/263443716 wait complete. 2024-04-25T01:22:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.825+0000 7f5b8dfd3740 20 client.25699 trim_cache size 0 max 16384 2024-04-25T01:22:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:22:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:22:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:22:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:22:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.826+0000 7f5b8b200640 1 -- 172.21.15.31:0/4182727353 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5b6c0a2c40 msgr2=0x7f5b6c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.826+0000 7f5b8b200640 1 --2- 172.21.15.31:0/4182727353 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5b6c0a2c40 0x7f5b6c0a5100 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00afd0 tx=0x7f5b7c0012e0 comp rx=0 tx=0).stop 2024-04-25T01:22:27.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.826+0000 7f5b8b200640 1 -- 172.21.15.31:0/4182727353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b84108de0 msgr2=0x7f5b8406bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:27.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.826+0000 7f5b8b200640 1 --2- 172.21.15.31:0/4182727353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b84108de0 0x7f5b8406bae0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5b7002a8d0 tx=0x7f5b70033d10 comp rx=0 tx=0).stop 2024-04-25T01:22:27.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.827+0000 7f5b8b200640 1 -- 172.21.15.31:0/4182727353 shutdown_connections 2024-04-25T01:22:27.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.827+0000 7f5b8b200640 1 --2- 172.21.15.31:0/4182727353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b841097a0 0x7f5b84072b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:27.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.827+0000 7f5b8b200640 1 --2- 172.21.15.31:0/4182727353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b84108de0 0x7f5b8406bae0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:27.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.827+0000 7f5b8b200640 1 --2- 172.21.15.31:0/4182727353 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5b6c0a2c40 0x7f5b6c0a5100 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:27.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.827+0000 7f5b8b200640 1 --2- 172.21.15.31:0/4182727353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b84107bf0 0x7f5b8406b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:27.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.827+0000 7f5b8b200640 1 -- 172.21.15.31:0/4182727353 >> 172.21.15.31:0/4182727353 conn(0x7f5b84076b90 msgr2=0x7f5b8410c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:22:27.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.827+0000 7f5b8b200640 1 -- 172.21.15.31:0/4182727353 shutdown_connections 2024-04-25T01:22:27.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:27.827+0000 7f5b8b200640 1 -- 172.21.15.31:0/4182727353 wait complete. 2024-04-25T01:22:27.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:27 smithi040 ceph-mon[32518]: pgmap v586: 97 pgs: 97 active+clean; 13 GiB data, 107 GiB used, 966 GiB / 1.0 TiB avail; 53 KiB/s rd, 119 MiB/s wr, 17.07k op/s 2024-04-25T01:22:29.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:29 smithi031 ceph-mon[28053]: pgmap v587: 97 pgs: 97 active+clean; 13 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 38 KiB/s rd, 107 MiB/s wr, 15.51k op/s 2024-04-25T01:22:29.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:29 smithi169 ceph-mon[32759]: pgmap v587: 97 pgs: 97 active+clean; 13 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 38 KiB/s rd, 107 MiB/s wr, 15.51k op/s 2024-04-25T01:22:29.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:29 smithi040 ceph-mon[32518]: pgmap v587: 97 pgs: 97 active+clean; 13 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 38 KiB/s rd, 107 MiB/s wr, 15.51k op/s 2024-04-25T01:22:31.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:31 smithi031 ceph-mon[28053]: pgmap v588: 97 pgs: 97 active+clean; 13 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 44 KiB/s rd, 102 MiB/s wr, 14.80k op/s 2024-04-25T01:22:31.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:31 smithi169 ceph-mon[32759]: pgmap v588: 97 pgs: 97 active+clean; 13 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 44 KiB/s rd, 102 MiB/s wr, 14.80k op/s 2024-04-25T01:22:31.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:31 smithi040 ceph-mon[32518]: pgmap v588: 97 pgs: 97 active+clean; 13 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 44 KiB/s rd, 102 MiB/s wr, 14.80k op/s 2024-04-25T01:22:33.751 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:33 smithi031 ceph-mon[28053]: pgmap v589: 97 pgs: 97 active+clean; 13 GiB data, 109 GiB used, 964 GiB / 1.0 TiB avail; 42 KiB/s rd, 112 MiB/s wr, 16.47k op/s 2024-04-25T01:22:33.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:33 smithi169 ceph-mon[32759]: pgmap v589: 97 pgs: 97 active+clean; 13 GiB data, 109 GiB used, 964 GiB / 1.0 TiB avail; 42 KiB/s rd, 112 MiB/s wr, 16.47k op/s 2024-04-25T01:22:33.943 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:33 smithi040 ceph-mon[32518]: pgmap v589: 97 pgs: 97 active+clean; 13 GiB data, 109 GiB used, 964 GiB / 1.0 TiB avail; 42 KiB/s rd, 112 MiB/s wr, 16.47k op/s 2024-04-25T01:22:34.843 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:34 smithi040 ceph-mon[32518]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:22:34.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:34 smithi169 ceph-mon[32759]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:22:35.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:34 smithi031 ceph-mon[28053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:22:35.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:35 smithi169 ceph-mon[32759]: pgmap v590: 97 pgs: 97 active+clean; 13 GiB data, 109 GiB used, 963 GiB / 1.0 TiB avail; 47 KiB/s rd, 102 MiB/s wr, 15.29k op/s 2024-04-25T01:22:35.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:35 smithi040 ceph-mon[32518]: pgmap v590: 97 pgs: 97 active+clean; 13 GiB data, 109 GiB used, 963 GiB / 1.0 TiB avail; 47 KiB/s rd, 102 MiB/s wr, 15.29k op/s 2024-04-25T01:22:36.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:35 smithi031 ceph-mon[28053]: pgmap v590: 97 pgs: 97 active+clean; 13 GiB data, 109 GiB used, 963 GiB / 1.0 TiB avail; 47 KiB/s rd, 102 MiB/s wr, 15.29k op/s 2024-04-25T01:22:37.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:37 smithi169 ceph-mon[32759]: pgmap v591: 97 pgs: 97 active+clean; 13 GiB data, 110 GiB used, 963 GiB / 1.0 TiB avail; 56 KiB/s rd, 112 MiB/s wr, 16.68k op/s 2024-04-25T01:22:37.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:37 smithi040 ceph-mon[32518]: pgmap v591: 97 pgs: 97 active+clean; 13 GiB data, 110 GiB used, 963 GiB / 1.0 TiB avail; 56 KiB/s rd, 112 MiB/s wr, 16.68k op/s 2024-04-25T01:22:38.000 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:37 smithi031 ceph-mon[28053]: pgmap v591: 97 pgs: 97 active+clean; 13 GiB data, 110 GiB used, 963 GiB / 1.0 TiB avail; 56 KiB/s rd, 112 MiB/s wr, 16.68k op/s 2024-04-25T01:22:39.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:39 smithi169 ceph-mon[32759]: pgmap v592: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 50 KiB/s rd, 109 MiB/s wr, 16.28k op/s 2024-04-25T01:22:39.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:39 smithi040 ceph-mon[32518]: pgmap v592: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 50 KiB/s rd, 109 MiB/s wr, 16.28k op/s 2024-04-25T01:22:40.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:39 smithi031 ceph-mon[28053]: pgmap v592: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 50 KiB/s rd, 109 MiB/s wr, 16.28k op/s 2024-04-25T01:22:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.052+0000 7f526f800640 1 -- 172.21.15.31:0/3661771518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52681018c0 msgr2=0x7f5268101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.053+0000 7f526f800640 1 --2- 172.21.15.31:0/3661771518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52681018c0 0x7f5268101d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5250009900 tx=0x7f525002a3f0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.053+0000 7f526f800640 1 -- 172.21.15.31:0/3661771518 shutdown_connections 2024-04-25T01:22:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.053+0000 7f526f800640 1 --2- 172.21.15.31:0/3661771518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5268102280 0x7f5268108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.053+0000 7f526f800640 1 --2- 172.21.15.31:0/3661771518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52681018c0 0x7f5268101d40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.053+0000 7f526f800640 1 --2- 172.21.15.31:0/3661771518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52681006d0 0x7f5268100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.053+0000 7f526f800640 1 -- 172.21.15.31:0/3661771518 >> 172.21.15.31:0/3661771518 conn(0x7f52680fbc40 msgr2=0x7f52680fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:22:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.054+0000 7f526f800640 1 -- 172.21.15.31:0/3661771518 shutdown_connections 2024-04-25T01:22:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.054+0000 7f526f800640 1 -- 172.21.15.31:0/3661771518 wait complete. 2024-04-25T01:22:41.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.055+0000 7f526f800640 1 Processor -- start 2024-04-25T01:22:41.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.056+0000 7f526f800640 1 -- start start 2024-04-25T01:22:41.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.057+0000 7f526f800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52681006d0 0x7f526819e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:41.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.057+0000 7f526f800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52681018c0 0x7f526819e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:41.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.057+0000 7f526f800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5268102280 0x7f52681a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:41.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.057+0000 7f526f800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f526810b2b0 con 0x7f52681018c0 2024-04-25T01:22:41.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.057+0000 7f526f800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f526810b130 con 0x7f52681006d0 2024-04-25T01:22:41.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.057+0000 7f526f800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f526810b430 con 0x7f5268102280 2024-04-25T01:22:41.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.058+0000 7f526d800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5268102280 0x7f52681a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:41.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.058+0000 7f525fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52681018c0 0x7f526819e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:41.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.058+0000 7f526d800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5268102280 0x7f52681a5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:36530/0 (socket says 172.21.15.31:36530) 2024-04-25T01:22:41.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.058+0000 7f526d800640 1 -- 172.21.15.31:0/2040226768 learned_addr learned my addr 172.21.15.31:0/2040226768 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:22:41.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.058+0000 7f526ce00640 1 --2- 172.21.15.31:0/2040226768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52681006d0 0x7f526819e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:41.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.059+0000 7f525fe00640 1 -- 172.21.15.31:0/2040226768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5268102280 msgr2=0x7f52681a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.059+0000 7f525fe00640 1 --2- 172.21.15.31:0/2040226768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5268102280 0x7f52681a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.059+0000 7f525fe00640 1 -- 172.21.15.31:0/2040226768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52681006d0 msgr2=0x7f526819e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.059+0000 7f525fe00640 1 --2- 172.21.15.31:0/2040226768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52681006d0 0x7f526819e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.059+0000 7f525fe00640 1 -- 172.21.15.31:0/2040226768 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52681a5cd0 con 0x7f52681018c0 2024-04-25T01:22:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.059+0000 7f526d800640 1 --2- 172.21.15.31:0/2040226768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5268102280 0x7f52681a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:22:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.060+0000 7f525fe00640 1 --2- 172.21.15.31:0/2040226768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52681018c0 0x7f526819e910 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f525002a910 tx=0x7f5250032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:22:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.060+0000 7f5266000640 1 -- 172.21.15.31:0/2040226768 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525003f030 con 0x7f52681018c0 2024-04-25T01:22:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.061+0000 7f5266000640 1 -- 172.21.15.31:0/2040226768 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5250003ce0 con 0x7f52681018c0 2024-04-25T01:22:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.061+0000 7f526f800640 1 -- 172.21.15.31:0/2040226768 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52681a5f60 con 0x7f52681018c0 2024-04-25T01:22:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.061+0000 7f5266000640 1 -- 172.21.15.31:0/2040226768 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5250035070 con 0x7f52681018c0 2024-04-25T01:22:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.061+0000 7f526f800640 1 -- 172.21.15.31:0/2040226768 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52681a63d0 con 0x7f52681018c0 2024-04-25T01:22:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.066+0000 7f5266000640 1 -- 172.21.15.31:0/2040226768 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5250042030 con 0x7f52681018c0 2024-04-25T01:22:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.066+0000 7f527258f740 1 Processor -- start 2024-04-25T01:22:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.067+0000 7f527258f740 1 -- start start 2024-04-25T01:22:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.067+0000 7f527258f740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559206184ca0 0x5592061850c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.067+0000 7f527258f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559206185690 0x55920618a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.067+0000 7f527258f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55920618a800 0x55920618cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.067+0000 7f527258f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5592060e7ab0 con 0x55920618a800 2024-04-25T01:22:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.067+0000 7f527258f740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x559206090640 con 0x559206184ca0 2024-04-25T01:22:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.067+0000 7f527258f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5592060e3a70 con 0x559206185690 2024-04-25T01:22:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.067+0000 7f526d800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559206184ca0 0x5592061850c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.067+0000 7f525fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55920618a800 0x55920618cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.067+0000 7f526d800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559206184ca0 0x5592061850c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:53586/0 (socket says 172.21.15.31:53586) 2024-04-25T01:22:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.067+0000 7f526d800640 1 -- 172.21.15.31:0/594631931 learned_addr learned my addr 172.21.15.31:0/594631931 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:22:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.067+0000 7f526ce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559206185690 0x55920618a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.068+0000 7f5266000640 1 --2- 172.21.15.31:0/2040226768 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f524c0a2cb0 0x7f524c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.068+0000 7f526ce00640 1 -- 172.21.15.31:0/594631931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559206184ca0 msgr2=0x5592061850c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.068+0000 7f526ce00640 1 --2- 172.21.15.31:0/594631931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559206184ca0 0x5592061850c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.068+0000 7f526ce00640 1 -- 172.21.15.31:0/594631931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55920618a800 msgr2=0x55920618cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.068+0000 7f526ce00640 1 --2- 172.21.15.31:0/594631931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55920618a800 0x55920618cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.068+0000 7f526ce00640 1 -- 172.21.15.31:0/594631931 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559205dfc210 con 0x559206185690 2024-04-25T01:22:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.068+0000 7f526d800640 1 --2- 172.21.15.31:0/594631931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559206184ca0 0x5592061850c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:22:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.068+0000 7f5266000640 1 -- 172.21.15.31:0/2040226768 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f52500ef010 con 0x7f52681018c0 2024-04-25T01:22:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.068+0000 7f526ce00640 1 --2- 172.21.15.31:0/594631931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559206185690 0x55920618a190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f525800bbc0 tx=0x7f52580103d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae9f94d6234d482b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:22:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.069+0000 7f526d800640 1 --2- 172.21.15.31:0/2040226768 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f524c0a2cb0 0x7f524c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.069+0000 7f525e000640 1 -- 172.21.15.31:0/594631931 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258004030 con 0x559206185690 2024-04-25T01:22:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.069+0000 7f525e000640 1 -- 172.21.15.31:0/594631931 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5258005500 con 0x559206185690 2024-04-25T01:22:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.069+0000 7f525e000640 1 -- 172.21.15.31:0/594631931 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258004ca0 con 0x559206185690 2024-04-25T01:22:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.070+0000 7f527258f740 1 -- 172.21.15.31:0/594631931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559206185690 msgr2=0x55920618a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.071+0000 7f527258f740 1 --2- 172.21.15.31:0/594631931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559206185690 0x55920618a190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f525800bbc0 tx=0x7f52580103d0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.073+0000 7f527258f740 1 -- 172.21.15.31:0/594631931 shutdown_connections 2024-04-25T01:22:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.073+0000 7f527258f740 1 --2- 172.21.15.31:0/594631931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55920618a800 0x55920618cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.073+0000 7f527258f740 1 --2- 172.21.15.31:0/594631931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559206185690 0x55920618a190 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.073+0000 7f527258f740 1 --2- 172.21.15.31:0/594631931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559206184ca0 0x5592061850c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.073+0000 7f527258f740 1 -- 172.21.15.31:0/594631931 >> 172.21.15.31:0/594631931 conn(0x559206180310 msgr2=0x559206180710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:22:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.074+0000 7f526d800640 1 --2- 172.21.15.31:0/2040226768 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f524c0a2cb0 0x7f524c0a5170 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x559206185ef0 tx=0x7f5260004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:22:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.075+0000 7f527258f740 1 -- 172.21.15.31:0/594631931 shutdown_connections 2024-04-25T01:22:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.075+0000 7f527258f740 1 -- 172.21.15.31:0/594631931 wait complete. 2024-04-25T01:22:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.076+0000 7f527258f740 1 -- start start 2024-04-25T01:22:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.077+0000 7f527258f740 1 Processor -- start 2024-04-25T01:22:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.080+0000 7f527258f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559206184ca0 0x55920618ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:41.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.081+0000 7f527258f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559206185690 0x55920618f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:41.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.081+0000 7f527258f740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55920618a800 0x559206196490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:41.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.081+0000 7f527258f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55920618d7b0 con 0x559206185690 2024-04-25T01:22:41.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.081+0000 7f527258f740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5592060a3990 con 0x55920618a800 2024-04-25T01:22:41.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.081+0000 7f527258f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55920618d930 con 0x559206184ca0 2024-04-25T01:22:41.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.081+0000 7f527258f740 10 client.0 _ll_register_callbacks cb 0x5592060a5950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:22:41.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.081+0000 7f526d800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55920618a800 0x559206196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:41.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.081+0000 7f526d800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55920618a800 0x559206196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:53588/0 (socket says 172.21.15.31:53588) 2024-04-25T01:22:41.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.081+0000 7f526d800640 1 -- 172.21.15.31:0/70186551 learned_addr learned my addr 172.21.15.31:0/70186551 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:22:41.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.082+0000 7f526ce00640 1 --2- 172.21.15.31:0/70186551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559206184ca0 0x55920618ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:41.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.082+0000 7f525fe00640 1 --2- 172.21.15.31:0/70186551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559206185690 0x55920618f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:41.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.083+0000 7f526d800640 1 -- 172.21.15.31:0/70186551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559206184ca0 msgr2=0x55920618ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:41.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.083+0000 7f526d800640 1 --2- 172.21.15.31:0/70186551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559206184ca0 0x55920618ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.083+0000 7f526d800640 1 -- 172.21.15.31:0/70186551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559206185690 msgr2=0x55920618f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:41.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.083+0000 7f526d800640 1 --2- 172.21.15.31:0/70186551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559206185690 0x55920618f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.083+0000 7f526d800640 1 -- 172.21.15.31:0/70186551 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559205ec8620 con 0x55920618a800 2024-04-25T01:22:41.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.084+0000 7f5256a00640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T01:22:41.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.084+0000 7f526d800640 1 --2- 172.21.15.31:0/70186551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55920618a800 0x559206196490 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5260011550 tx=0x7f5260004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94acaabf554e4e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:22:41.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.085+0000 7f5256a00640 1 -- 172.21.15.31:0/70186551 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526002d060 con 0x55920618a800 2024-04-25T01:22:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.085+0000 7f5256a00640 1 -- 172.21.15.31:0/70186551 <== mon.2 v2:172.21.15.40:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f526000f7d0 con 0x55920618a800 2024-04-25T01:22:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.086+0000 7f527258f740 1 -- 172.21.15.31:0/70186551 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5258005500 con 0x55920618a800 2024-04-25T01:22:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.086+0000 7f5256a00640 1 -- 172.21.15.31:0/70186551 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5260010f00 con 0x55920618a800 2024-04-25T01:22:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.087+0000 7f527258f740 10 client.25676 fetch_fsmap learned FSMap version 30 2024-04-25T01:22:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.087+0000 7f527258f740 1 -- 172.21.15.31:0/70186551 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5592061994e0 con 0x55920618a800 2024-04-25T01:22:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.088+0000 7f5256a00640 1 -- 172.21.15.31:0/70186551 <== mon.2 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5258005500 con 0x55920618a800 2024-04-25T01:22:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.088+0000 7f5256a00640 1 -- 172.21.15.31:0/70186551 <== mon.2 v2:172.21.15.40:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f5260028c00 con 0x55920618a800 2024-04-25T01:22:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.088+0000 7f527258f740 10 client.25676 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T01:22:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.088+0000 7f527258f740 10 client.25676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:22:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.089+0000 7f527258f740 20 client.25676 populate_metadata read hostname 'smithi031' 2024-04-25T01:22:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.089+0000 7f527258f740 1 --2- 172.21.15.31:0/70186551 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55920619abd0 0x55920619d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.089+0000 7f527258f740 4 client.25676 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:22:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.089+0000 7f527258f740 1 -- 172.21.15.31:0/70186551 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592060f4ff0 con 0x55920619abd0 2024-04-25T01:22:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.089+0000 7f526ce00640 1 --2- 172.21.15.31:0/70186551 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55920619abd0 0x55920619d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.090+0000 7f526ce00640 1 --2- 172.21.15.31:0/70186551 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55920619abd0 0x55920619d060 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc630d31c4664bf server_cookie=c2254a6e33e1256e in_seq=0 out_seq=0 2024-04-25T01:22:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.090+0000 7f5256a00640 10 client.25676 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:22:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.091+0000 7f5256a00640 1 -- 172.21.15.31:0/70186551 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5592060f4ff0 con 0x55920619abd0 2024-04-25T01:22:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.091+0000 7f5256a00640 10 client.25676 handle_command_reply: tid=0 2024-04-25T01:22:41.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.091+0000 7f527258f740 1 client.25676 shutdown 2024-04-25T01:22:41.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.092+0000 7f527258f740 1 -- 172.21.15.31:0/70186551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55920618a800 msgr2=0x559206196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:41.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.092+0000 7f527258f740 1 --2- 172.21.15.31:0/70186551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55920618a800 0x559206196490 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5260011550 tx=0x7f5260004ab0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.092+0000 7f527258f740 1 -- 172.21.15.31:0/70186551 shutdown_connections 2024-04-25T01:22:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.092+0000 7f527258f740 1 --2- 172.21.15.31:0/70186551 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55920619abd0 0x55920619d060 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.092+0000 7f527258f740 1 --2- 172.21.15.31:0/70186551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55920618a800 0x559206196490 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.092+0000 7f527258f740 1 --2- 172.21.15.31:0/70186551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559206185690 0x55920618f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.092+0000 7f527258f740 1 --2- 172.21.15.31:0/70186551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559206184ca0 0x55920618ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.092+0000 7f527258f740 1 -- 172.21.15.31:0/70186551 >> 172.21.15.31:0/70186551 conn(0x559206180310 msgr2=0x55920617f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:22:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.093+0000 7f5256a00640 0 client.25676 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:22:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.093+0000 7f527258f740 1 -- 172.21.15.31:0/70186551 shutdown_connections 2024-04-25T01:22:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.093+0000 7f527258f740 1 -- 172.21.15.31:0/70186551 wait complete. 2024-04-25T01:22:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.094+0000 7f527258f740 20 client.25676 trim_cache size 0 max 16384 2024-04-25T01:22:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.130+0000 7f527258f740 1 Processor -- start 2024-04-25T01:22:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.130+0000 7f527258f740 1 -- start start 2024-04-25T01:22:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.131+0000 7f527258f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559206184ca0 0x55920618ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.131+0000 7f527258f740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559206185690 0x55920618f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.131+0000 7f527258f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55920618a800 0x559206196b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.131+0000 7f527258f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f52681b19b0 con 0x55920618a800 2024-04-25T01:22:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.131+0000 7f527258f740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f52681af280 con 0x559206185690 2024-04-25T01:22:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.131+0000 7f527258f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f52681ad0d0 con 0x559206184ca0 2024-04-25T01:22:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.132+0000 7f525fe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559206185690 0x55920618f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.132+0000 7f526ce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559206184ca0 0x55920618ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.132+0000 7f526ce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559206184ca0 0x55920618ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:36542/0 (socket says 172.21.15.31:36542) 2024-04-25T01:22:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.132+0000 7f526ce00640 1 -- 172.21.15.31:0/3611348596 learned_addr learned my addr 172.21.15.31:0/3611348596 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:22:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.133+0000 7f526d800640 1 --2- 172.21.15.31:0/3611348596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55920618a800 0x559206196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.133+0000 7f525fe00640 1 -- 172.21.15.31:0/3611348596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559206184ca0 msgr2=0x55920618ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.133+0000 7f525fe00640 1 --2- 172.21.15.31:0/3611348596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559206184ca0 0x55920618ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.134+0000 7f525fe00640 1 -- 172.21.15.31:0/3611348596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55920618a800 msgr2=0x559206196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.134+0000 7f525fe00640 1 --2- 172.21.15.31:0/3611348596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55920618a800 0x559206196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.134+0000 7f525fe00640 1 -- 172.21.15.31:0/3611348596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55920618a550 con 0x559206185690 2024-04-25T01:22:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.134+0000 7f526ce00640 1 --2- 172.21.15.31:0/3611348596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559206184ca0 0x55920618ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:22:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.135+0000 7f526d800640 1 --2- 172.21.15.31:0/3611348596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55920618a800 0x559206196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:22:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.135+0000 7f525fe00640 1 --2- 172.21.15.31:0/3611348596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559206185690 0x55920618f450 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f52500ecbf0 tx=0x7f525003b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2200d9d7c3b8e4f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:22:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.136+0000 7f525d600640 1 -- 172.21.15.31:0/3611348596 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5250049040 con 0x559206185690 2024-04-25T01:22:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.136+0000 7f525d600640 1 -- 172.21.15.31:0/3611348596 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52500bb9b0 con 0x559206185690 2024-04-25T01:22:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.136+0000 7f525d600640 1 -- 172.21.15.31:0/3611348596 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52500b8020 con 0x559206185690 2024-04-25T01:22:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.137+0000 7f527258f740 1 -- 172.21.15.31:0/3611348596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559206185690 msgr2=0x55920618f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.137+0000 7f527258f740 1 --2- 172.21.15.31:0/3611348596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559206185690 0x55920618f450 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f52500ecbf0 tx=0x7f525003b800 comp rx=0 tx=0).stop 2024-04-25T01:22:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.137+0000 7f527258f740 1 -- 172.21.15.31:0/3611348596 shutdown_connections 2024-04-25T01:22:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.137+0000 7f527258f740 1 --2- 172.21.15.31:0/3611348596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55920618a800 0x559206196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.137+0000 7f527258f740 1 --2- 172.21.15.31:0/3611348596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559206185690 0x55920618f450 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.137+0000 7f527258f740 1 --2- 172.21.15.31:0/3611348596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559206184ca0 0x55920618ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.137+0000 7f527258f740 1 -- 172.21.15.31:0/3611348596 >> 172.21.15.31:0/3611348596 conn(0x559206180310 msgr2=0x55920617afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:22:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.138+0000 7f527258f740 1 -- 172.21.15.31:0/3611348596 shutdown_connections 2024-04-25T01:22:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.138+0000 7f527258f740 1 -- 172.21.15.31:0/3611348596 wait complete. 2024-04-25T01:22:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.139+0000 7f527258f740 1 -- start start 2024-04-25T01:22:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.140+0000 7f527258f740 1 Processor -- start 2024-04-25T01:22:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.141+0000 7f527258f740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559206184ca0 0x5592061fddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.141+0000 7f527258f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559206185690 0x5592061fe310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.142+0000 7f526ce00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559206184ca0 0x5592061fddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.142+0000 7f525fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559206185690 0x5592061fe310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.142+0000 7f525fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559206185690 0x5592061fe310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:36552/0 (socket says 172.21.15.31:36552) 2024-04-25T01:22:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.141+0000 7f527258f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55920618a800 0x559206194480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.142+0000 7f527258f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f52681ad740 con 0x55920618a800 2024-04-25T01:22:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.142+0000 7f527258f740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f52681aebb0 con 0x559206184ca0 2024-04-25T01:22:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.142+0000 7f527258f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f52681ade10 con 0x559206185690 2024-04-25T01:22:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.142+0000 7f527258f740 10 client.0 _ll_register_callbacks cb 0x7f5260011300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:22:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.142+0000 7f525fe00640 1 -- 172.21.15.31:0/273809197 learned_addr learned my addr 172.21.15.31:0/273809197 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:22:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.143+0000 7f525fe00640 1 -- 172.21.15.31:0/273809197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559206184ca0 msgr2=0x5592061fddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.143+0000 7f525fe00640 1 --2- 172.21.15.31:0/273809197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559206184ca0 0x5592061fddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.143+0000 7f525fe00640 1 -- 172.21.15.31:0/273809197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55920618a800 msgr2=0x559206194480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.143+0000 7f525fe00640 1 --2- 172.21.15.31:0/273809197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55920618a800 0x559206194480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.143+0000 7f525fe00640 1 -- 172.21.15.31:0/273809197 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559206194a70 con 0x559206185690 2024-04-25T01:22:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.144+0000 7f525fe00640 1 --2- 172.21.15.31:0/273809197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559206185690 0x5592061fe310 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5250000c00 tx=0x7f52500f2860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dab18fed93fd93f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:22:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.144+0000 7f5256a00640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:22:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.144+0000 7f5256a00640 1 -- 172.21.15.31:0/273809197 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5250049030 con 0x559206185690 2024-04-25T01:22:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.144+0000 7f5256a00640 1 -- 172.21.15.31:0/273809197 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f525003b460 con 0x559206185690 2024-04-25T01:22:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.144+0000 7f527258f740 1 -- 172.21.15.31:0/273809197 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52500bb9b0 con 0x559206185690 2024-04-25T01:22:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.145+0000 7f5256a00640 1 -- 172.21.15.31:0/273809197 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f52500bb4f0 con 0x559206185690 2024-04-25T01:22:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.145+0000 7f527258f740 10 client.25717 fetch_fsmap learned FSMap version 30 2024-04-25T01:22:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.145+0000 7f527258f740 1 -- 172.21.15.31:0/273809197 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559206195d60 con 0x559206185690 2024-04-25T01:22:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.145+0000 7f5256a00640 1 -- 172.21.15.31:0/273809197 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52500bb9b0 con 0x559206185690 2024-04-25T01:22:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.147+0000 7f5256a00640 1 -- 172.21.15.31:0/273809197 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f525004c060 con 0x559206185690 2024-04-25T01:22:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.147+0000 7f527258f740 10 client.25717 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T01:22:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.147+0000 7f527258f740 10 client.25717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:22:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.147+0000 7f527258f740 20 client.25717 populate_metadata read hostname 'smithi031' 2024-04-25T01:22:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.147+0000 7f527258f740 1 --2- 172.21.15.31:0/273809197 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x559206212970 0x559206214d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.147+0000 7f527258f740 4 client.25717 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:22:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.147+0000 7f527258f740 1 -- 172.21.15.31:0/273809197 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5592060ef340 con 0x559206212970 2024-04-25T01:22:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.148+0000 7f526ce00640 1 --2- 172.21.15.31:0/273809197 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x559206212970 0x559206214d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.149+0000 7f526ce00640 1 --2- 172.21.15.31:0/273809197 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x559206212970 0x559206214d90 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=753db18c75b1f89e server_cookie=b29a3b8b8d105d37 in_seq=0 out_seq=0 2024-04-25T01:22:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.149+0000 7f5256a00640 10 client.25717 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:22:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.428+0000 7f5256a00640 1 -- 172.21.15.31:0/273809197 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5592060ef340 con 0x559206212970 2024-04-25T01:22:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.428+0000 7f5256a00640 10 client.25717 handle_command_reply: tid=0 2024-04-25T01:22:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.428+0000 7f527258f740 1 client.25717 shutdown 2024-04-25T01:22:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.430+0000 7f527258f740 1 -- 172.21.15.31:0/273809197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559206185690 msgr2=0x5592061fe310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.430+0000 7f527258f740 1 --2- 172.21.15.31:0/273809197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559206185690 0x5592061fe310 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5250000c00 tx=0x7f52500f2860 comp rx=0 tx=0).stop 2024-04-25T01:22:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.431+0000 7f527258f740 1 -- 172.21.15.31:0/273809197 shutdown_connections 2024-04-25T01:22:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.431+0000 7f527258f740 1 --2- 172.21.15.31:0/273809197 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x559206212970 0x559206214d90 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.431+0000 7f527258f740 1 --2- 172.21.15.31:0/273809197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55920618a800 0x559206194480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.431+0000 7f527258f740 1 --2- 172.21.15.31:0/273809197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559206185690 0x5592061fe310 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.431+0000 7f527258f740 1 --2- 172.21.15.31:0/273809197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559206184ca0 0x5592061fddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.431+0000 7f527258f740 1 -- 172.21.15.31:0/273809197 >> 172.21.15.31:0/273809197 conn(0x559206180310 msgr2=0x5592061868b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:22:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.432+0000 7f5256a00640 0 client.25717 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:22:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.432+0000 7f527258f740 1 -- 172.21.15.31:0/273809197 shutdown_connections 2024-04-25T01:22:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.433+0000 7f527258f740 1 -- 172.21.15.31:0/273809197 wait complete. 2024-04-25T01:22:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.434+0000 7f527258f740 20 client.25717 trim_cache size 0 max 16384 2024-04-25T01:22:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:22:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:22:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:22:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:22:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.441+0000 7f526f800640 1 -- 172.21.15.31:0/2040226768 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f524c0a2cb0 msgr2=0x7f524c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.442+0000 7f526f800640 1 --2- 172.21.15.31:0/2040226768 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f524c0a2cb0 0x7f524c0a5170 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x559206185ef0 tx=0x7f5260004a40 comp rx=0 tx=0).stop 2024-04-25T01:22:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.442+0000 7f526f800640 1 -- 172.21.15.31:0/2040226768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52681018c0 msgr2=0x7f526819e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.442+0000 7f526f800640 1 --2- 172.21.15.31:0/2040226768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52681018c0 0x7f526819e910 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f525002a910 tx=0x7f5250032ce0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.443+0000 7f526f800640 1 -- 172.21.15.31:0/2040226768 shutdown_connections 2024-04-25T01:22:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.443+0000 7f526f800640 1 --2- 172.21.15.31:0/2040226768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5268102280 0x7f52681a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.443+0000 7f526f800640 1 --2- 172.21.15.31:0/2040226768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52681018c0 0x7f526819e910 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.443+0000 7f526f800640 1 --2- 172.21.15.31:0/2040226768 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f524c0a2cb0 0x7f524c0a5170 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.443+0000 7f526f800640 1 --2- 172.21.15.31:0/2040226768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52681006d0 0x7f526819e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.443+0000 7f526f800640 1 -- 172.21.15.31:0/2040226768 >> 172.21.15.31:0/2040226768 conn(0x7f52680fbc40 msgr2=0x7f526810dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:22:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.444+0000 7f526f800640 1 -- 172.21.15.31:0/2040226768 shutdown_connections 2024-04-25T01:22:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:41.444+0000 7f526f800640 1 -- 172.21.15.31:0/2040226768 wait complete. 2024-04-25T01:22:41.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:41 smithi169 ceph-mon[32759]: pgmap v593: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 58 KiB/s rd, 100 MiB/s wr, 14.90k op/s 2024-04-25T01:22:41.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:41 smithi040 ceph-mon[32518]: pgmap v593: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 58 KiB/s rd, 100 MiB/s wr, 14.90k op/s 2024-04-25T01:22:42.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:41 smithi031 ceph-mon[28053]: pgmap v593: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 58 KiB/s rd, 100 MiB/s wr, 14.90k op/s 2024-04-25T01:22:43.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:43 smithi169 ceph-mon[32759]: pgmap v594: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 58 KiB/s rd, 109 MiB/s wr, 15.93k op/s 2024-04-25T01:22:43.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:43 smithi040 ceph-mon[32518]: pgmap v594: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 58 KiB/s rd, 109 MiB/s wr, 15.93k op/s 2024-04-25T01:22:44.003 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:43 smithi031 ceph-mon[28053]: pgmap v594: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 58 KiB/s rd, 109 MiB/s wr, 15.93k op/s 2024-04-25T01:22:45.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:45 smithi169 ceph-mon[32759]: pgmap v595: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 63 KiB/s rd, 110 MiB/s wr, 16.02k op/s 2024-04-25T01:22:45.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:45 smithi040 ceph-mon[32518]: pgmap v595: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 63 KiB/s rd, 110 MiB/s wr, 16.02k op/s 2024-04-25T01:22:46.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:45 smithi031 ceph-mon[28053]: pgmap v595: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 63 KiB/s rd, 110 MiB/s wr, 16.02k op/s 2024-04-25T01:22:48.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:47 smithi031 ceph-mon[28053]: pgmap v596: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 64 KiB/s rd, 112 MiB/s wr, 16.29k op/s 2024-04-25T01:22:48.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:47 smithi169 ceph-mon[32759]: pgmap v596: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 64 KiB/s rd, 112 MiB/s wr, 16.29k op/s 2024-04-25T01:22:48.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:47 smithi040 ceph-mon[32518]: pgmap v596: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 64 KiB/s rd, 112 MiB/s wr, 16.29k op/s 2024-04-25T01:22:50.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:49 smithi169 ceph-mon[32759]: pgmap v597: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 55 KiB/s rd, 108 MiB/s wr, 15.91k op/s 2024-04-25T01:22:50.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:49 smithi040 ceph-mon[32518]: pgmap v597: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 55 KiB/s rd, 108 MiB/s wr, 15.91k op/s 2024-04-25T01:22:50.251 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:49 smithi031 ceph-mon[28053]: pgmap v597: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 55 KiB/s rd, 108 MiB/s wr, 15.91k op/s 2024-04-25T01:22:52.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:51 smithi169 ceph-mon[32759]: pgmap v598: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 57 KiB/s rd, 103 MiB/s wr, 15.20k op/s 2024-04-25T01:22:52.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:51 smithi040 ceph-mon[32518]: pgmap v598: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 57 KiB/s rd, 103 MiB/s wr, 15.20k op/s 2024-04-25T01:22:52.251 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:51 smithi031 ceph-mon[28053]: pgmap v598: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 57 KiB/s rd, 103 MiB/s wr, 15.20k op/s 2024-04-25T01:22:54.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:53 smithi169 ceph-mon[32759]: pgmap v599: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 44 KiB/s rd, 113 MiB/s wr, 16.50k op/s 2024-04-25T01:22:54.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:53 smithi040 ceph-mon[32518]: pgmap v599: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 44 KiB/s rd, 113 MiB/s wr, 16.50k op/s 2024-04-25T01:22:54.250 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:53 smithi031 ceph-mon[28053]: pgmap v599: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 962 GiB / 1.0 TiB avail; 44 KiB/s rd, 113 MiB/s wr, 16.50k op/s 2024-04-25T01:22:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.695+0000 7fbc96a00640 1 -- 172.21.15.31:0/4124558147 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9002f310 con 0x7fbc98107500 2024-04-25T01:22:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.696+0000 7fbc9ec00640 1 -- 172.21.15.31:0/4124558147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc98107500 msgr2=0x7fbc981098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.696+0000 7fbc9ec00640 1 --2- 172.21.15.31:0/4124558147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc98107500 0x7fbc981098f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009ca0 tx=0x7fbc9002a8c0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.696+0000 7fbc9ec00640 1 -- 172.21.15.31:0/4124558147 shutdown_connections 2024-04-25T01:22:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.696+0000 7fbc9ec00640 1 --2- 172.21.15.31:0/4124558147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc98107500 0x7fbc981098f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.696+0000 7fbc9ec00640 1 --2- 172.21.15.31:0/4124558147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc98104bd0 0x7fbc98106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.696+0000 7fbc9ec00640 1 --2- 172.21.15.31:0/4124558147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc981022a0 0x7fbc98104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.696+0000 7fbc9ec00640 1 -- 172.21.15.31:0/4124558147 >> 172.21.15.31:0/4124558147 conn(0x7fbc980fbc10 msgr2=0x7fbc980fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:22:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.696+0000 7fbc9ec00640 1 -- 172.21.15.31:0/4124558147 shutdown_connections 2024-04-25T01:22:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.696+0000 7fbc9ec00640 1 -- 172.21.15.31:0/4124558147 wait complete. 2024-04-25T01:22:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.698+0000 7fbc9ec00640 1 Processor -- start 2024-04-25T01:22:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.698+0000 7fbc9ec00640 1 -- start start 2024-04-25T01:22:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.698+0000 7fbc9ec00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc981022a0 0x7fbc98199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.699+0000 7fbc9ec00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc98104bd0 0x7fbc9819a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.699+0000 7fbc9ec00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc98107500 0x7fbc981a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.699+0000 7fbc97e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc981022a0 0x7fbc98199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.699+0000 7fbc97e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc981022a0 0x7fbc98199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:55400/0 (socket says 172.21.15.31:55400) 2024-04-25T01:22:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.700+0000 7fbc97400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc98104bd0 0x7fbc9819a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.700+0000 7fbc97e00640 1 -- 172.21.15.31:0/1140832743 learned_addr learned my addr 172.21.15.31:0/1140832743 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:22:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.700+0000 7fbc9ec00640 1 -- 172.21.15.31:0/1140832743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9810d560 con 0x7fbc98107500 2024-04-25T01:22:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.700+0000 7fbc9ec00640 1 -- 172.21.15.31:0/1140832743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9810d3e0 con 0x7fbc98104bd0 2024-04-25T01:22:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.700+0000 7fbc9ec00640 1 -- 172.21.15.31:0/1140832743 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9810d6e0 con 0x7fbc981022a0 2024-04-25T01:22:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.700+0000 7fbc97e00640 1 -- 172.21.15.31:0/1140832743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc98104bd0 msgr2=0x7fbc9819a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.700+0000 7fbc97e00640 1 --2- 172.21.15.31:0/1140832743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc98104bd0 0x7fbc9819a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.700+0000 7fbc97e00640 1 -- 172.21.15.31:0/1140832743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc98107500 msgr2=0x7fbc981a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.700+0000 7fbc97e00640 1 --2- 172.21.15.31:0/1140832743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc98107500 0x7fbc981a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.700+0000 7fbc97e00640 1 -- 172.21.15.31:0/1140832743 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc981a17c0 con 0x7fbc981022a0 2024-04-25T01:22:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.701+0000 7fbc97e00640 1 --2- 172.21.15.31:0/1140832743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc981022a0 0x7fbc98199ba0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800b4a0 tx=0x7fbc8800b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:22:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.702+0000 7fbc94c00640 1 -- 172.21.15.31:0/1140832743 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc88015020 con 0x7fbc981022a0 2024-04-25T01:22:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.702+0000 7fbc94c00640 1 -- 172.21.15.31:0/1140832743 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc88011d10 con 0x7fbc981022a0 2024-04-25T01:22:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.702+0000 7fbc94c00640 1 -- 172.21.15.31:0/1140832743 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8800f2e0 con 0x7fbc981022a0 2024-04-25T01:22:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.702+0000 7fbc9ec00640 1 -- 172.21.15.31:0/1140832743 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc981a1ab0 con 0x7fbc981022a0 2024-04-25T01:22:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.702+0000 7fbc9ec00640 1 -- 172.21.15.31:0/1140832743 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc981a2030 con 0x7fbc981022a0 2024-04-25T01:22:54.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.718+0000 7fbca1b6e740 1 Processor -- start 2024-04-25T01:22:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.719+0000 7fbc94c00640 1 -- 172.21.15.31:0/1140832743 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbc88018020 con 0x7fbc981022a0 2024-04-25T01:22:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.725+0000 7fbca1b6e740 1 -- start start 2024-04-25T01:22:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.725+0000 7fbca1b6e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589aedad4b0 0x5589aedad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.725+0000 7fbca1b6e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589aedadea0 0x5589aedb29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.725+0000 7fbca1b6e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5589aedb3010 0x5589aedb54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.725+0000 7fbca1b6e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5589aed07ab0 con 0x5589aedadea0 2024-04-25T01:22:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.726+0000 7fbca1b6e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5589aecb0640 con 0x5589aedb3010 2024-04-25T01:22:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.726+0000 7fbca1b6e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5589aed03a70 con 0x5589aedad4b0 2024-04-25T01:22:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.726+0000 7fbc9cc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589aedad4b0 0x5589aedad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.726+0000 7fbc97400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5589aedb3010 0x5589aedb54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.726+0000 7fbc97400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5589aedb3010 0x5589aedb54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:50070/0 (socket says 172.21.15.31:50070) 2024-04-25T01:22:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.726+0000 7fbc97400640 1 -- 172.21.15.31:0/1353431417 learned_addr learned my addr 172.21.15.31:0/1353431417 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:22:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.727+0000 7fbc97e00640 1 --2- 172.21.15.31:0/1353431417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589aedadea0 0x5589aedb29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.729+0000 7fbc9cc00640 1 -- 172.21.15.31:0/1353431417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5589aedb3010 msgr2=0x5589aedb54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.729+0000 7fbc9cc00640 1 --2- 172.21.15.31:0/1353431417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5589aedb3010 0x5589aedb54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.729+0000 7fbc9cc00640 1 -- 172.21.15.31:0/1353431417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589aedadea0 msgr2=0x5589aedb29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.729+0000 7fbc9cc00640 1 --2- 172.21.15.31:0/1353431417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589aedadea0 0x5589aedb29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.729+0000 7fbc9cc00640 1 -- 172.21.15.31:0/1353431417 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589aea1c210 con 0x5589aedad4b0 2024-04-25T01:22:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.730+0000 7fbc94c00640 1 --2- 172.21.15.31:0/1140832743 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbc7c0a2da0 0x7fbc7c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.730+0000 7fbc94c00640 1 -- 172.21.15.31:0/1140832743 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fbc880ce6d0 con 0x7fbc981022a0 2024-04-25T01:22:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.731+0000 7fbc9cc00640 1 --2- 172.21.15.31:0/1353431417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589aedad4b0 0x5589aedad8d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbc9002aeb0 tx=0x7fbc900098d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49af0c0fdfc6fece server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:22:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.731+0000 7fbc8d200640 1 -- 172.21.15.31:0/1353431417 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc90003d80 con 0x5589aedad4b0 2024-04-25T01:22:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.732+0000 7fbc97e00640 1 --2- 172.21.15.31:0/1140832743 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbc7c0a2da0 0x7fbc7c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.732+0000 7fbc8d200640 1 -- 172.21.15.31:0/1353431417 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc9003f070 con 0x5589aedad4b0 2024-04-25T01:22:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.732+0000 7fbc8d200640 1 -- 172.21.15.31:0/1353431417 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9002f610 con 0x5589aedad4b0 2024-04-25T01:22:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.733+0000 7fbca1b6e740 1 -- 172.21.15.31:0/1353431417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589aedad4b0 msgr2=0x5589aedad8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.733+0000 7fbca1b6e740 1 --2- 172.21.15.31:0/1353431417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589aedad4b0 0x5589aedad8d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbc9002aeb0 tx=0x7fbc900098d0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.733+0000 7fbca1b6e740 1 -- 172.21.15.31:0/1353431417 shutdown_connections 2024-04-25T01:22:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.733+0000 7fbca1b6e740 1 --2- 172.21.15.31:0/1353431417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5589aedb3010 0x5589aedb54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.733+0000 7fbca1b6e740 1 --2- 172.21.15.31:0/1353431417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589aedadea0 0x5589aedb29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.733+0000 7fbca1b6e740 1 --2- 172.21.15.31:0/1353431417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589aedad4b0 0x5589aedad8d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.733+0000 7fbca1b6e740 1 -- 172.21.15.31:0/1353431417 >> 172.21.15.31:0/1353431417 conn(0x5589aeda0310 msgr2=0x5589aeda0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:22:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.734+0000 7fbc97e00640 1 --2- 172.21.15.31:0/1140832743 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbc7c0a2da0 0x7fbc7c0a5260 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbc88097930 tx=0x7fbc88010910 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:22:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.736+0000 7fbca1b6e740 1 -- 172.21.15.31:0/1353431417 shutdown_connections 2024-04-25T01:22:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.736+0000 7fbca1b6e740 1 -- 172.21.15.31:0/1353431417 wait complete. 2024-04-25T01:22:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.737+0000 7fbca1b6e740 1 -- start start 2024-04-25T01:22:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.737+0000 7fbca1b6e740 1 Processor -- start 2024-04-25T01:22:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.740+0000 7fbca1b6e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589aedad4b0 0x5589aedb76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.740+0000 7fbca1b6e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589aedadea0 0x5589aedb7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.740+0000 7fbca1b6e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5589aedb3010 0x5589aedbec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.740+0000 7fbca1b6e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5589aedb5f60 con 0x5589aedadea0 2024-04-25T01:22:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.740+0000 7fbca1b6e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5589aecc3990 con 0x5589aedb3010 2024-04-25T01:22:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.740+0000 7fbca1b6e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5589aedb60e0 con 0x5589aedad4b0 2024-04-25T01:22:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.740+0000 7fbca1b6e740 10 client.0 _ll_register_callbacks cb 0x5589aecc5950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:22:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.741+0000 7fbc97400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589aedadea0 0x5589aedb7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.741+0000 7fbc97400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589aedadea0 0x5589aedb7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35114/0 (socket says 172.21.15.31:35114) 2024-04-25T01:22:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.741+0000 7fbc97400640 1 -- 172.21.15.31:0/2958230541 learned_addr learned my addr 172.21.15.31:0/2958230541 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:22:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.741+0000 7fbc9cc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589aedad4b0 0x5589aedb76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.742+0000 7fbc97400640 1 -- 172.21.15.31:0/2958230541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589aedad4b0 msgr2=0x5589aedb76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:54.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.742+0000 7fbc97400640 1 --2- 172.21.15.31:0/2958230541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589aedad4b0 0x5589aedb76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.742+0000 7fbc97400640 1 -- 172.21.15.31:0/2958230541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5589aedb3010 msgr2=0x5589aedbec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:54.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.744+0000 7fbc9cc00640 1 --2- 172.21.15.31:0/2958230541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5589aedb3010 0x5589aedbec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.744+0000 7fbc97400640 1 --2- 172.21.15.31:0/2958230541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5589aedb3010 0x5589aedbec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.744+0000 7fbc97400640 1 -- 172.21.15.31:0/2958230541 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589aeae8620 con 0x5589aedadea0 2024-04-25T01:22:54.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.744+0000 7fbc9cc00640 1 --2- 172.21.15.31:0/2958230541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5589aedb3010 0x5589aedbec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:22:54.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.746+0000 7fbc97400640 1 --2- 172.21.15.31:0/2958230541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589aedadea0 0x5589aedb7c00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009f60 tx=0x7fbc84009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ebcd3fb831167a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:22:54.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.747+0000 7fbc81600640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T01:22:54.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.748+0000 7fbc81600640 1 -- 172.21.15.31:0/2958230541 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc84019070 con 0x5589aedadea0 2024-04-25T01:22:54.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.748+0000 7fbc81600640 1 -- 172.21.15.31:0/2958230541 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fbc84015870 con 0x5589aedadea0 2024-04-25T01:22:54.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.748+0000 7fbca1b6e740 1 -- 172.21.15.31:0/2958230541 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc9003f070 con 0x5589aedadea0 2024-04-25T01:22:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.752+0000 7fbc81600640 1 -- 172.21.15.31:0/2958230541 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbc84025050 con 0x5589aedadea0 2024-04-25T01:22:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.752+0000 7fbca1b6e740 10 client.15408 fetch_fsmap learned FSMap version 30 2024-04-25T01:22:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.752+0000 7fbca1b6e740 1 -- 172.21.15.31:0/2958230541 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5589aedc1cf0 con 0x5589aedadea0 2024-04-25T01:22:54.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.754+0000 7fbc81600640 1 -- 172.21.15.31:0/2958230541 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc9003f070 con 0x5589aedadea0 2024-04-25T01:22:54.736 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.755+0000 7fbc81600640 1 -- 172.21.15.31:0/2958230541 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fbc84014b00 con 0x5589aedadea0 2024-04-25T01:22:54.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.755+0000 7fbca1b6e740 10 client.15408 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T01:22:54.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.755+0000 7fbca1b6e740 10 client.15408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:22:54.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.755+0000 7fbca1b6e740 20 client.15408 populate_metadata read hostname 'smithi031' 2024-04-25T01:22:54.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.755+0000 7fbca1b6e740 1 --2- 172.21.15.31:0/2958230541 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5589aedc33e0 0x5589aedc57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:54.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.755+0000 7fbca1b6e740 4 client.15408 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:22:54.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.755+0000 7fbca1b6e740 1 -- 172.21.15.31:0/2958230541 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589aed14ff0 con 0x5589aedc33e0 2024-04-25T01:22:54.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.756+0000 7fbc9cc00640 1 --2- 172.21.15.31:0/2958230541 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5589aedc33e0 0x5589aedc57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.761+0000 7fbc81600640 10 client.15408 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:22:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.761+0000 7fbc9cc00640 1 --2- 172.21.15.31:0/2958230541 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5589aedc33e0 0x5589aedc57d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a085811eb7b599e server_cookie=ef495cf835cb8f45 in_seq=0 out_seq=0 2024-04-25T01:22:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.763+0000 7fbc81600640 1 -- 172.21.15.31:0/2958230541 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5589aed14ff0 con 0x5589aedc33e0 2024-04-25T01:22:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.763+0000 7fbc81600640 10 client.15408 handle_command_reply: tid=0 2024-04-25T01:22:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.763+0000 7fbca1b6e740 1 client.15408 shutdown 2024-04-25T01:22:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.763+0000 7fbca1b6e740 1 -- 172.21.15.31:0/2958230541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589aedadea0 msgr2=0x5589aedb7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.763+0000 7fbca1b6e740 1 --2- 172.21.15.31:0/2958230541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589aedadea0 0x5589aedb7c00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009f60 tx=0x7fbc84009f90 comp rx=0 tx=0).stop 2024-04-25T01:22:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.763+0000 7fbca1b6e740 1 -- 172.21.15.31:0/2958230541 shutdown_connections 2024-04-25T01:22:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.763+0000 7fbca1b6e740 1 --2- 172.21.15.31:0/2958230541 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5589aedc33e0 0x5589aedc57d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.763+0000 7fbca1b6e740 1 --2- 172.21.15.31:0/2958230541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5589aedb3010 0x5589aedbec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.763+0000 7fbca1b6e740 1 --2- 172.21.15.31:0/2958230541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589aedadea0 0x5589aedb7c00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.763+0000 7fbc81600640 0 client.15408 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:22:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.763+0000 7fbca1b6e740 1 --2- 172.21.15.31:0/2958230541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589aedad4b0 0x5589aedb76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.763+0000 7fbca1b6e740 1 -- 172.21.15.31:0/2958230541 >> 172.21.15.31:0/2958230541 conn(0x5589aeda0310 msgr2=0x5589aed9f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:22:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.764+0000 7fbca1b6e740 1 -- 172.21.15.31:0/2958230541 shutdown_connections 2024-04-25T01:22:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.765+0000 7fbca1b6e740 1 -- 172.21.15.31:0/2958230541 wait complete. 2024-04-25T01:22:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.765+0000 7fbca1b6e740 20 client.15408 trim_cache size 0 max 16384 2024-04-25T01:22:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.820+0000 7fbca1b6e740 1 Processor -- start 2024-04-25T01:22:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.820+0000 7fbca1b6e740 1 -- start start 2024-04-25T01:22:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.821+0000 7fbca1b6e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589aedad4b0 0x5589aedaf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.821+0000 7fbca1b6e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589aedb3010 0x5589aedaf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.821+0000 7fbca1b6e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5589aedbec00 0x5589aeda3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.821+0000 7fbca1b6e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbc981a9040 con 0x5589aedb3010 2024-04-25T01:22:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.821+0000 7fbca1b6e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbc981a6910 con 0x5589aedbec00 2024-04-25T01:22:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.821+0000 7fbca1b6e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbc981a4810 con 0x5589aedad4b0 2024-04-25T01:22:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.822+0000 7fbc9cc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589aedad4b0 0x5589aedaf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.822+0000 7fbc9cc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589aedad4b0 0x5589aedaf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:55424/0 (socket says 172.21.15.31:55424) 2024-04-25T01:22:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.822+0000 7fbc9cc00640 1 -- 172.21.15.31:0/1081127518 learned_addr learned my addr 172.21.15.31:0/1081127518 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:22:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.822+0000 7fbc97400640 1 --2- 172.21.15.31:0/1081127518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589aedb3010 0x5589aedaf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.823+0000 7fbc9cc00640 1 --2- 172.21.15.31:0/1081127518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5589aedbec00 0x5589aeda3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.823+0000 7fbc97400640 1 -- 172.21.15.31:0/1081127518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589aedad4b0 msgr2=0x5589aedaf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.823+0000 7fbc97400640 1 --2- 172.21.15.31:0/1081127518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589aedad4b0 0x5589aedaf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.823+0000 7fbc97400640 1 -- 172.21.15.31:0/1081127518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5589aedbec00 msgr2=0x5589aeda3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.824+0000 7fbc97400640 1 --2- 172.21.15.31:0/1081127518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5589aedbec00 0x5589aeda3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.824+0000 7fbc97400640 1 -- 172.21.15.31:0/1081127518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589aeda0140 con 0x5589aedb3010 2024-04-25T01:22:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.824+0000 7fbc9cc00640 1 --2- 172.21.15.31:0/1081127518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589aedad4b0 0x5589aedaf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:22:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.825+0000 7fbc97400640 1 --2- 172.21.15.31:0/1081127518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589aedb3010 0x5589aedaf660 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbc8400e000 tx=0x7fbc840046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cec893966997d09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:22:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.825+0000 7fbc83e00640 1 -- 172.21.15.31:0/1081127518 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8402e070 con 0x5589aedb3010 2024-04-25T01:22:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.826+0000 7fbc83e00640 1 -- 172.21.15.31:0/1081127518 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc8401d6f0 con 0x5589aedb3010 2024-04-25T01:22:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.826+0000 7fbc83e00640 1 -- 172.21.15.31:0/1081127518 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc84019040 con 0x5589aedb3010 2024-04-25T01:22:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.828+0000 7fbca1b6e740 1 -- 172.21.15.31:0/1081127518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589aedb3010 msgr2=0x5589aedaf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.828+0000 7fbca1b6e740 1 --2- 172.21.15.31:0/1081127518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589aedb3010 0x5589aedaf660 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbc8400e000 tx=0x7fbc840046b0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.829+0000 7fbca1b6e740 1 -- 172.21.15.31:0/1081127518 shutdown_connections 2024-04-25T01:22:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.830+0000 7fbca1b6e740 1 --2- 172.21.15.31:0/1081127518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5589aedbec00 0x5589aeda3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.830+0000 7fbca1b6e740 1 --2- 172.21.15.31:0/1081127518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589aedb3010 0x5589aedaf660 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.830+0000 7fbca1b6e740 1 --2- 172.21.15.31:0/1081127518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589aedad4b0 0x5589aedaf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.830+0000 7fbca1b6e740 1 -- 172.21.15.31:0/1081127518 >> 172.21.15.31:0/1081127518 conn(0x5589aeda0310 msgr2=0x5589aedb2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:22:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.830+0000 7fbca1b6e740 1 -- 172.21.15.31:0/1081127518 shutdown_connections 2024-04-25T01:22:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.830+0000 7fbca1b6e740 1 -- 172.21.15.31:0/1081127518 wait complete. 2024-04-25T01:22:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.831+0000 7fbca1b6e740 1 -- start start 2024-04-25T01:22:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.831+0000 7fbca1b6e740 1 Processor -- start 2024-04-25T01:22:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.833+0000 7fbca1b6e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589aedad4b0 0x5589aedba7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.834+0000 7fbc9cc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589aedad4b0 0x5589aedba7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.834+0000 7fbc9cc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589aedad4b0 0x5589aedba7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:55436/0 (socket says 172.21.15.31:55436) 2024-04-25T01:22:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.834+0000 7fbc9cc00640 1 -- 172.21.15.31:0/83053195 learned_addr learned my addr 172.21.15.31:0/83053195 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:22:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.834+0000 7fbca1b6e740 1 --2- 172.21.15.31:0/83053195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589aedb3010 0x5589aedbad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.834+0000 7fbca1b6e740 1 --2- 172.21.15.31:0/83053195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5589aedbec00 0x5589aedbdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.834+0000 7fbca1b6e740 1 -- 172.21.15.31:0/83053195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbc981a4dd0 con 0x5589aedb3010 2024-04-25T01:22:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.834+0000 7fbca1b6e740 1 -- 172.21.15.31:0/83053195 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbc981a6240 con 0x5589aedbec00 2024-04-25T01:22:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.834+0000 7fbca1b6e740 1 -- 172.21.15.31:0/83053195 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbc981a54a0 con 0x5589aedad4b0 2024-04-25T01:22:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.835+0000 7fbca1b6e740 10 client.0 _ll_register_callbacks cb 0x7fbc84009500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:22:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.835+0000 7fbc9cc00640 1 --2- 172.21.15.31:0/83053195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5589aedbec00 0x5589aedbdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.836+0000 7fbc9cc00640 1 -- 172.21.15.31:0/83053195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5589aedbec00 msgr2=0x5589aedbdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.836+0000 7fbc9cc00640 1 --2- 172.21.15.31:0/83053195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5589aedbec00 0x5589aedbdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.836+0000 7fbc9cc00640 1 -- 172.21.15.31:0/83053195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589aedb3010 msgr2=0x5589aedbad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:22:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.836+0000 7fbc9cc00640 1 --2- 172.21.15.31:0/83053195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589aedb3010 0x5589aedbad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.836+0000 7fbc9cc00640 1 -- 172.21.15.31:0/83053195 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589aedbe310 con 0x5589aedad4b0 2024-04-25T01:22:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.837+0000 7fbc81600640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:22:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.837+0000 7fbc9cc00640 1 --2- 172.21.15.31:0/83053195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589aedad4b0 0x5589aedba7c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbc9002ade0 tx=0x7fbc9003d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20f1008158eccd60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:22:54.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.839+0000 7fbc81600640 1 -- 172.21.15.31:0/83053195 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9005d060 con 0x5589aedad4b0 2024-04-25T01:22:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.839+0000 7fbc81600640 1 -- 172.21.15.31:0/83053195 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fbc900040c0 con 0x5589aedad4b0 2024-04-25T01:22:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.840+0000 7fbca1b6e740 1 -- 172.21.15.31:0/83053195 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc8401d6f0 con 0x5589aedad4b0 2024-04-25T01:22:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.840+0000 7fbc81600640 1 -- 172.21.15.31:0/83053195 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbc9004e040 con 0x5589aedad4b0 2024-04-25T01:22:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.841+0000 7fbca1b6e740 10 client.25735 fetch_fsmap learned FSMap version 30 2024-04-25T01:22:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.841+0000 7fbca1b6e740 1 -- 172.21.15.31:0/83053195 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5589aee3b3f0 con 0x5589aedad4b0 2024-04-25T01:22:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.842+0000 7fbc81600640 1 -- 172.21.15.31:0/83053195 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc8401d6f0 con 0x5589aedad4b0 2024-04-25T01:22:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.842+0000 7fbc81600640 1 -- 172.21.15.31:0/83053195 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fbc9002f8e0 con 0x5589aedad4b0 2024-04-25T01:22:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.845+0000 7fbca1b6e740 10 client.25735 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T01:22:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.845+0000 7fbca1b6e740 10 client.25735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:22:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.845+0000 7fbca1b6e740 20 client.25735 populate_metadata read hostname 'smithi031' 2024-04-25T01:22:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.845+0000 7fbca1b6e740 1 --2- 172.21.15.31:0/83053195 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5589aee3b9b0 0x5589aee3dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:22:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.845+0000 7fbca1b6e740 4 client.25735 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:22:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.845+0000 7fbca1b6e740 1 -- 172.21.15.31:0/83053195 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5589aed0f340 con 0x5589aee3b9b0 2024-04-25T01:22:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.845+0000 7fbc97400640 1 --2- 172.21.15.31:0/83053195 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5589aee3b9b0 0x5589aee3dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:22:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.846+0000 7fbc97400640 1 --2- 172.21.15.31:0/83053195 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5589aee3b9b0 0x5589aee3dda0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60738c01020ca678 server_cookie=5daf08a1260489e1 in_seq=0 out_seq=0 2024-04-25T01:22:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:54.846+0000 7fbc81600640 10 client.25735 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:22:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.850+0000 7fbc81600640 1 -- 172.21.15.31:0/83053195 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5589aed0f340 con 0x5589aee3b9b0 2024-04-25T01:22:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.850+0000 7fbc81600640 10 client.25735 handle_command_reply: tid=0 2024-04-25T01:22:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.850+0000 7fbca1b6e740 1 client.25735 shutdown 2024-04-25T01:22:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.851+0000 7fbca1b6e740 1 -- 172.21.15.31:0/83053195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589aedad4b0 msgr2=0x5589aedba7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.851+0000 7fbca1b6e740 1 --2- 172.21.15.31:0/83053195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589aedad4b0 0x5589aedba7c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbc9002ade0 tx=0x7fbc9003d040 comp rx=0 tx=0).stop 2024-04-25T01:22:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.851+0000 7fbca1b6e740 1 -- 172.21.15.31:0/83053195 shutdown_connections 2024-04-25T01:22:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.851+0000 7fbca1b6e740 1 --2- 172.21.15.31:0/83053195 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5589aee3b9b0 0x5589aee3dda0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.852+0000 7fbc81600640 0 client.25735 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:22:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.852+0000 7fbca1b6e740 1 --2- 172.21.15.31:0/83053195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5589aedbec00 0x5589aedbdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.852+0000 7fbca1b6e740 1 --2- 172.21.15.31:0/83053195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589aedb3010 0x5589aedbad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.852+0000 7fbca1b6e740 1 --2- 172.21.15.31:0/83053195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589aedad4b0 0x5589aedba7c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.852+0000 7fbca1b6e740 1 -- 172.21.15.31:0/83053195 >> 172.21.15.31:0/83053195 conn(0x5589aeda0310 msgr2=0x5589aedaf120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:22:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.853+0000 7fbca1b6e740 1 -- 172.21.15.31:0/83053195 shutdown_connections 2024-04-25T01:22:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.853+0000 7fbca1b6e740 1 -- 172.21.15.31:0/83053195 wait complete. 2024-04-25T01:22:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.853+0000 7fbca1b6e740 20 client.25735 trim_cache size 0 max 16384 2024-04-25T01:22:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:22:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:22:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:22:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:22:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.854+0000 7fbc9ec00640 1 -- 172.21.15.31:0/1140832743 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbc7c0a2da0 msgr2=0x7fbc7c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.854+0000 7fbc9ec00640 1 --2- 172.21.15.31:0/1140832743 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbc7c0a2da0 0x7fbc7c0a5260 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbc88097930 tx=0x7fbc88010910 comp rx=0 tx=0).stop 2024-04-25T01:22:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.854+0000 7fbc9ec00640 1 -- 172.21.15.31:0/1140832743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc981022a0 msgr2=0x7fbc98199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:22:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.854+0000 7fbc9ec00640 1 --2- 172.21.15.31:0/1140832743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc981022a0 0x7fbc98199ba0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800b4a0 tx=0x7fbc8800b8a0 comp rx=0 tx=0).stop 2024-04-25T01:22:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.855+0000 7fbc9ec00640 1 -- 172.21.15.31:0/1140832743 shutdown_connections 2024-04-25T01:22:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.855+0000 7fbc9ec00640 1 --2- 172.21.15.31:0/1140832743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc98107500 0x7fbc981a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.855+0000 7fbc9ec00640 1 --2- 172.21.15.31:0/1140832743 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbc7c0a2da0 0x7fbc7c0a5260 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.855+0000 7fbc9ec00640 1 --2- 172.21.15.31:0/1140832743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc98104bd0 0x7fbc9819a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.855+0000 7fbc9ec00640 1 --2- 172.21.15.31:0/1140832743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc981022a0 0x7fbc98199ba0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:22:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.855+0000 7fbc9ec00640 1 -- 172.21.15.31:0/1140832743 >> 172.21.15.31:0/1140832743 conn(0x7fbc980fbc10 msgr2=0x7fbc981005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:22:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.855+0000 7fbc9ec00640 1 -- 172.21.15.31:0/1140832743 shutdown_connections 2024-04-25T01:22:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:22:55.855+0000 7fbc9ec00640 1 -- 172.21.15.31:0/1140832743 wait complete. 2024-04-25T01:22:56.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:55 smithi169 ceph-mon[32759]: pgmap v600: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 44 KiB/s rd, 110 MiB/s wr, 16.15k op/s 2024-04-25T01:22:56.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:55 smithi040 ceph-mon[32518]: pgmap v600: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 44 KiB/s rd, 110 MiB/s wr, 16.15k op/s 2024-04-25T01:22:56.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:55 smithi031 ceph-mon[28053]: pgmap v600: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 44 KiB/s rd, 110 MiB/s wr, 16.15k op/s 2024-04-25T01:22:58.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:57 smithi169 ceph-mon[32759]: pgmap v601: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 41 KiB/s rd, 108 MiB/s wr, 15.99k op/s 2024-04-25T01:22:58.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:57 smithi040 ceph-mon[32518]: pgmap v601: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 41 KiB/s rd, 108 MiB/s wr, 15.99k op/s 2024-04-25T01:22:58.250 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:57 smithi031 ceph-mon[28053]: pgmap v601: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 41 KiB/s rd, 108 MiB/s wr, 15.99k op/s 2024-04-25T01:23:00.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:22:59 smithi169 ceph-mon[32759]: pgmap v602: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 27 KiB/s rd, 107 MiB/s wr, 15.62k op/s 2024-04-25T01:23:00.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:22:59 smithi040 ceph-mon[32518]: pgmap v602: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 27 KiB/s rd, 107 MiB/s wr, 15.62k op/s 2024-04-25T01:23:00.251 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:22:59 smithi031 ceph-mon[28053]: pgmap v602: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 27 KiB/s rd, 107 MiB/s wr, 15.62k op/s 2024-04-25T01:23:02.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:01 smithi169 ceph-mon[32759]: pgmap v603: 97 pgs: 97 active+clean; 13 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 30 KiB/s rd, 96 MiB/s wr, 13.97k op/s 2024-04-25T01:23:02.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:01 smithi040 ceph-mon[32518]: pgmap v603: 97 pgs: 97 active+clean; 13 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 30 KiB/s rd, 96 MiB/s wr, 13.97k op/s 2024-04-25T01:23:02.250 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:01 smithi031 ceph-mon[28053]: pgmap v603: 97 pgs: 97 active+clean; 13 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 30 KiB/s rd, 96 MiB/s wr, 13.97k op/s 2024-04-25T01:23:04.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:03 smithi169 ceph-mon[32759]: pgmap v604: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 35 KiB/s rd, 109 MiB/s wr, 16.26k op/s 2024-04-25T01:23:04.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:03 smithi040 ceph-mon[32518]: pgmap v604: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 35 KiB/s rd, 109 MiB/s wr, 16.26k op/s 2024-04-25T01:23:04.250 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:03 smithi031 ceph-mon[28053]: pgmap v604: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 35 KiB/s rd, 109 MiB/s wr, 16.26k op/s 2024-04-25T01:23:06.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:05 smithi169 ceph-mon[32759]: pgmap v605: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 33 KiB/s rd, 99 MiB/s wr, 14.96k op/s 2024-04-25T01:23:06.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:05 smithi040 ceph-mon[32518]: pgmap v605: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 33 KiB/s rd, 99 MiB/s wr, 14.96k op/s 2024-04-25T01:23:06.252 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:05 smithi031 ceph-mon[28053]: pgmap v605: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 33 KiB/s rd, 99 MiB/s wr, 14.96k op/s 2024-04-25T01:23:08.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:07 smithi169 ceph-mon[32759]: pgmap v606: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 46 KiB/s rd, 99 MiB/s wr, 15.18k op/s 2024-04-25T01:23:08.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:07 smithi040 ceph-mon[32518]: pgmap v606: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 46 KiB/s rd, 99 MiB/s wr, 15.18k op/s 2024-04-25T01:23:08.251 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:07 smithi031 ceph-mon[28053]: pgmap v606: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 46 KiB/s rd, 99 MiB/s wr, 15.18k op/s 2024-04-25T01:23:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.071+0000 7ff374a00640 1 -- 172.21.15.31:0/1423937820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff370101890 msgr2=0x7ff370101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.071+0000 7ff374a00640 1 --2- 172.21.15.31:0/1423937820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff370101890 0x7ff370101d10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff35c00b9f0 tx=0x7ff35c02c0a0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.072+0000 7ff374a00640 1 -- 172.21.15.31:0/1423937820 shutdown_connections 2024-04-25T01:23:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.072+0000 7ff374a00640 1 --2- 172.21.15.31:0/1423937820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff370102250 0x7ff370108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.072+0000 7ff374a00640 1 --2- 172.21.15.31:0/1423937820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff370101890 0x7ff370101d10 secure :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff35c00b9f0 tx=0x7ff35c02c0a0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.072+0000 7ff374a00640 1 --2- 172.21.15.31:0/1423937820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3701006a0 0x7ff370100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.072+0000 7ff374a00640 1 -- 172.21.15.31:0/1423937820 >> 172.21.15.31:0/1423937820 conn(0x7ff3700fbc30 msgr2=0x7ff3700fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:23:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.072+0000 7ff374a00640 1 -- 172.21.15.31:0/1423937820 shutdown_connections 2024-04-25T01:23:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.072+0000 7ff374a00640 1 -- 172.21.15.31:0/1423937820 wait complete. 2024-04-25T01:23:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.074+0000 7ff374a00640 1 Processor -- start 2024-04-25T01:23:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.074+0000 7ff374a00640 1 -- start start 2024-04-25T01:23:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.075+0000 7ff374a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3701006a0 0x7ff370070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.075+0000 7ff374a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff370102250 0x7ff370070760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.075+0000 7ff374a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff370192920 0x7ff370194de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.075+0000 7ff374a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff37010b270 con 0x7ff3701006a0 2024-04-25T01:23:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.075+0000 7ff374a00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff37010b0f0 con 0x7ff370102250 2024-04-25T01:23:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.075+0000 7ff36e800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff370192920 0x7ff370194de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.075+0000 7ff36e800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff370192920 0x7ff370194de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:35170/0 (socket says 172.21.15.31:35170) 2024-04-25T01:23:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.076+0000 7ff36e800640 1 -- 172.21.15.31:0/3245251331 learned_addr learned my addr 172.21.15.31:0/3245251331 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:23:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.076+0000 7ff36d400640 1 --2- 172.21.15.31:0/3245251331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff370102250 0x7ff370070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.075+0000 7ff374a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff37010b3f0 con 0x7ff370192920 2024-04-25T01:23:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.076+0000 7ff36de00640 1 --2- 172.21.15.31:0/3245251331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3701006a0 0x7ff370070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.076+0000 7ff36e800640 1 -- 172.21.15.31:0/3245251331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff370102250 msgr2=0x7ff370070760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.076+0000 7ff36e800640 1 --2- 172.21.15.31:0/3245251331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff370102250 0x7ff370070760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.076+0000 7ff36e800640 1 -- 172.21.15.31:0/3245251331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3701006a0 msgr2=0x7ff370070220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.076+0000 7ff36e800640 1 --2- 172.21.15.31:0/3245251331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3701006a0 0x7ff370070220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.076+0000 7ff36e800640 1 -- 172.21.15.31:0/3245251331 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff370199730 con 0x7ff370192920 2024-04-25T01:23:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.077+0000 7ff36de00640 1 --2- 172.21.15.31:0/3245251331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3701006a0 0x7ff370070220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:23:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.077+0000 7ff36e800640 1 --2- 172.21.15.31:0/3245251331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff370192920 0x7ff370194de0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff368007e50 tx=0x7ff368014420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:23:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.078+0000 7ff366a00640 1 -- 172.21.15.31:0/3245251331 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36801f020 con 0x7ff370192920 2024-04-25T01:23:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.079+0000 7ff366a00640 1 -- 172.21.15.31:0/3245251331 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff368014da0 con 0x7ff370192920 2024-04-25T01:23:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.079+0000 7ff374a00640 1 -- 172.21.15.31:0/3245251331 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff370195320 con 0x7ff370192920 2024-04-25T01:23:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.079+0000 7ff366a00640 1 -- 172.21.15.31:0/3245251331 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff368004a90 con 0x7ff370192920 2024-04-25T01:23:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.079+0000 7ff374a00640 1 -- 172.21.15.31:0/3245251331 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff370195840 con 0x7ff370192920 2024-04-25T01:23:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.086+0000 7ff366a00640 1 -- 172.21.15.31:0/3245251331 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff368004c30 con 0x7ff370192920 2024-04-25T01:23:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.089+0000 7ff377824740 1 Processor -- start 2024-04-25T01:23:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.090+0000 7ff377824740 1 -- start start 2024-04-25T01:23:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.090+0000 7ff377824740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1bde0 0x560959e1c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.090+0000 7ff377824740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560959e1cfa0 0x560959e1d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.090+0000 7ff377824740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560959e1d960 0x560959e24070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.090+0000 7ff377824740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560959d6dab0 con 0x560959e1d960 2024-04-25T01:23:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.090+0000 7ff377824740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560959d16640 con 0x560959e1cfa0 2024-04-25T01:23:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.090+0000 7ff377824740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560959d69a70 con 0x560959e1bde0 2024-04-25T01:23:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.091+0000 7ff36d400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1bde0 0x560959e1c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.091+0000 7ff36d400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1bde0 0x560959e1c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:35180/0 (socket says 172.21.15.31:35180) 2024-04-25T01:23:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.091+0000 7ff36d400640 1 -- 172.21.15.31:0/932974342 learned_addr learned my addr 172.21.15.31:0/932974342 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:23:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.091+0000 7ff36d400640 1 --2- 172.21.15.31:0/932974342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560959e1d960 0x560959e24070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.091+0000 7ff36de00640 1 --2- 172.21.15.31:0/932974342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560959e1cfa0 0x560959e1d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.092+0000 7ff366a00640 1 --2- 172.21.15.31:0/3245251331 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff3540a2e70 0x7ff3540a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.092+0000 7ff366a00640 1 -- 172.21.15.31:0/3245251331 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7ff3680d6a00 con 0x7ff370192920 2024-04-25T01:23:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.095+0000 7ff36d400640 1 -- 172.21.15.31:0/932974342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560959e1cfa0 msgr2=0x560959e1d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.095+0000 7ff36d400640 1 --2- 172.21.15.31:0/932974342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560959e1cfa0 0x560959e1d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.095+0000 7ff36e800640 1 --2- 172.21.15.31:0/3245251331 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff3540a2e70 0x7ff3540a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.095+0000 7ff36d400640 1 -- 172.21.15.31:0/932974342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560959e1d960 msgr2=0x560959e24070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.095+0000 7ff36d400640 1 --2- 172.21.15.31:0/932974342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560959e1d960 0x560959e24070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.095+0000 7ff36d400640 1 -- 172.21.15.31:0/932974342 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560959a82210 con 0x560959e1bde0 2024-04-25T01:23:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.096+0000 7ff36d400640 1 --2- 172.21.15.31:0/932974342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1bde0 0x560959e1c200 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff35c02c690 tx=0x7ff35c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bfe0b37972d448e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:23:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.096+0000 7ff35b000640 1 -- 172.21.15.31:0/932974342 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35c0098b0 con 0x560959e1bde0 2024-04-25T01:23:09.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.097+0000 7ff35b000640 1 -- 172.21.15.31:0/932974342 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff35c05b070 con 0x560959e1bde0 2024-04-25T01:23:09.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.098+0000 7ff377824740 1 -- 172.21.15.31:0/932974342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1bde0 msgr2=0x560959e1c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:09.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.098+0000 7ff377824740 1 --2- 172.21.15.31:0/932974342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1bde0 0x560959e1c200 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff35c02c690 tx=0x7ff35c005f90 comp rx=0 tx=0).stop 2024-04-25T01:23:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.099+0000 7ff377824740 1 -- 172.21.15.31:0/932974342 shutdown_connections 2024-04-25T01:23:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.099+0000 7ff377824740 1 --2- 172.21.15.31:0/932974342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560959e1d960 0x560959e24070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.099+0000 7ff377824740 1 --2- 172.21.15.31:0/932974342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560959e1cfa0 0x560959e1d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.099+0000 7ff377824740 1 --2- 172.21.15.31:0/932974342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1bde0 0x560959e1c200 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.099+0000 7ff377824740 1 -- 172.21.15.31:0/932974342 >> 172.21.15.31:0/932974342 conn(0x560959e06310 msgr2=0x560959e06710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:23:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.103+0000 7ff377824740 1 -- 172.21.15.31:0/932974342 shutdown_connections 2024-04-25T01:23:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.103+0000 7ff36e800640 1 --2- 172.21.15.31:0/3245251331 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff3540a2e70 0x7ff3540a5330 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff3680a2fe0 tx=0x7ff368014f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:23:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.103+0000 7ff377824740 1 -- 172.21.15.31:0/932974342 wait complete. 2024-04-25T01:23:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.104+0000 7ff377824740 1 -- start start 2024-04-25T01:23:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.104+0000 7ff377824740 1 Processor -- start 2024-04-25T01:23:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.106+0000 7ff377824740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560959e1bde0 0x560959e127d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.106+0000 7ff377824740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1cfa0 0x560959e12d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.107+0000 7ff377824740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560959e1d960 0x560959e19d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.107+0000 7ff36de00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1cfa0 0x560959e12d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.107+0000 7ff36de00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1cfa0 0x560959e12d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:35184/0 (socket says 172.21.15.31:35184) 2024-04-25T01:23:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.107+0000 7ff36de00640 1 -- 172.21.15.31:0/1499463042 learned_addr learned my addr 172.21.15.31:0/1499463042 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:23:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.107+0000 7ff377824740 1 -- 172.21.15.31:0/1499463042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560959e0aae0 con 0x560959e1bde0 2024-04-25T01:23:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.107+0000 7ff377824740 1 -- 172.21.15.31:0/1499463042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x560959d29990 con 0x560959e1d960 2024-04-25T01:23:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.107+0000 7ff377824740 1 -- 172.21.15.31:0/1499463042 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560959e0ac60 con 0x560959e1cfa0 2024-04-25T01:23:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.107+0000 7ff377824740 10 client.0 _ll_register_callbacks cb 0x560959d2b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:23:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.108+0000 7ff36d400640 1 --2- 172.21.15.31:0/1499463042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560959e1d960 0x560959e19d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.109+0000 7ff36de00640 1 -- 172.21.15.31:0/1499463042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560959e1d960 msgr2=0x560959e19d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.109+0000 7ff36de00640 1 --2- 172.21.15.31:0/1499463042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560959e1d960 0x560959e19d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.109+0000 7ff36de00640 1 -- 172.21.15.31:0/1499463042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560959e1bde0 msgr2=0x560959e127d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.109+0000 7ff36de00640 1 --2- 172.21.15.31:0/1499463042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560959e1bde0 0x560959e127d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.109+0000 7ff36de00640 1 -- 172.21.15.31:0/1499463042 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560959b4e620 con 0x560959e1cfa0 2024-04-25T01:23:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.109+0000 7ff36d400640 1 --2- 172.21.15.31:0/1499463042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560959e1d960 0x560959e19d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:23:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.110+0000 7ff34f400640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:23:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.110+0000 7ff36de00640 1 --2- 172.21.15.31:0/1499463042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1cfa0 0x560959e12d10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff360010b50 tx=0x7ff36000e7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64d837f44b23b837 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:23:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.110+0000 7ff34f400640 1 -- 172.21.15.31:0/1499463042 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36001d020 con 0x560959e1cfa0 2024-04-25T01:23:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.110+0000 7ff34f400640 1 -- 172.21.15.31:0/1499463042 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7ff360016050 con 0x560959e1cfa0 2024-04-25T01:23:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.110+0000 7ff377824740 1 -- 172.21.15.31:0/1499463042 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff35c05b070 con 0x560959e1cfa0 2024-04-25T01:23:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.111+0000 7ff34f400640 1 -- 172.21.15.31:0/1499463042 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff36001d020 con 0x560959e1cfa0 2024-04-25T01:23:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.111+0000 7ff377824740 10 client.25747 fetch_fsmap learned FSMap version 30 2024-04-25T01:23:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.111+0000 7ff377824740 1 -- 172.21.15.31:0/1499463042 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560959e1f420 con 0x560959e1cfa0 2024-04-25T01:23:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.112+0000 7ff34f400640 1 -- 172.21.15.31:0/1499463042 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff36000c580 con 0x560959e1cfa0 2024-04-25T01:23:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.112+0000 7ff34f400640 1 -- 172.21.15.31:0/1499463042 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7ff360018070 con 0x560959e1cfa0 2024-04-25T01:23:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.112+0000 7ff377824740 10 client.25747 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T01:23:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.112+0000 7ff377824740 10 client.25747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:23:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.112+0000 7ff377824740 20 client.25747 populate_metadata read hostname 'smithi031' 2024-04-25T01:23:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.112+0000 7ff377824740 1 --2- 172.21.15.31:0/1499463042 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560959e20b60 0x560959e22f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.112+0000 7ff377824740 4 client.25747 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:23:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.112+0000 7ff377824740 1 -- 172.21.15.31:0/1499463042 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff35c05b070 con 0x560959e20b60 2024-04-25T01:23:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.113+0000 7ff36d400640 1 --2- 172.21.15.31:0/1499463042 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560959e20b60 0x560959e22f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.114+0000 7ff36d400640 1 --2- 172.21.15.31:0/1499463042 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560959e20b60 0x560959e22f90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=746bac4c404cb347 server_cookie=46d661d0ae074707 in_seq=0 out_seq=0 2024-04-25T01:23:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.114+0000 7ff34f400640 10 client.25747 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:23:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.115+0000 7ff34f400640 1 -- 172.21.15.31:0/1499463042 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7ff35c05b070 con 0x560959e20b60 2024-04-25T01:23:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.115+0000 7ff34f400640 10 client.25747 handle_command_reply: tid=0 2024-04-25T01:23:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.115+0000 7ff377824740 1 client.25747 shutdown 2024-04-25T01:23:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.115+0000 7ff377824740 1 -- 172.21.15.31:0/1499463042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1cfa0 msgr2=0x560959e12d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.115+0000 7ff377824740 1 --2- 172.21.15.31:0/1499463042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1cfa0 0x560959e12d10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff360010b50 tx=0x7ff36000e7c0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.116+0000 7ff377824740 1 -- 172.21.15.31:0/1499463042 shutdown_connections 2024-04-25T01:23:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.116+0000 7ff377824740 1 --2- 172.21.15.31:0/1499463042 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560959e20b60 0x560959e22f90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.116+0000 7ff377824740 1 --2- 172.21.15.31:0/1499463042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560959e1d960 0x560959e19d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.116+0000 7ff377824740 1 --2- 172.21.15.31:0/1499463042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1cfa0 0x560959e12d10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.116+0000 7ff377824740 1 --2- 172.21.15.31:0/1499463042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560959e1bde0 0x560959e127d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.116+0000 7ff377824740 1 -- 172.21.15.31:0/1499463042 >> 172.21.15.31:0/1499463042 conn(0x560959e06310 msgr2=0x560959e05480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:23:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.116+0000 7ff34f400640 0 client.25747 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:23:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.116+0000 7ff377824740 1 -- 172.21.15.31:0/1499463042 shutdown_connections 2024-04-25T01:23:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.116+0000 7ff377824740 1 -- 172.21.15.31:0/1499463042 wait complete. 2024-04-25T01:23:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.117+0000 7ff377824740 20 client.25747 trim_cache size 0 max 16384 2024-04-25T01:23:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.151+0000 7ff377824740 1 Processor -- start 2024-04-25T01:23:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.152+0000 7ff377824740 1 -- start start 2024-04-25T01:23:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.153+0000 7ff377824740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1bde0 0x560959e04bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.153+0000 7ff377824740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560959e1cfa0 0x560959e19d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.153+0000 7ff377824740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560959e1d960 0x560959e05130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.153+0000 7ff377824740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff3701a0fa0 con 0x560959e1cfa0 2024-04-25T01:23:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.153+0000 7ff377824740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff37019e870 con 0x560959e1d960 2024-04-25T01:23:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.153+0000 7ff377824740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff37019c6e0 con 0x560959e1bde0 2024-04-25T01:23:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.153+0000 7ff36d400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1bde0 0x560959e04bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.153+0000 7ff36d400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1bde0 0x560959e04bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:35188/0 (socket says 172.21.15.31:35188) 2024-04-25T01:23:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.153+0000 7ff36d400640 1 -- 172.21.15.31:0/3635866633 learned_addr learned my addr 172.21.15.31:0/3635866633 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:23:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.153+0000 7ff36d400640 1 --2- 172.21.15.31:0/3635866633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560959e1d960 0x560959e05130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.155+0000 7ff36d400640 1 -- 172.21.15.31:0/3635866633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560959e1d960 msgr2=0x560959e05130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.155+0000 7ff36de00640 1 --2- 172.21.15.31:0/3635866633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560959e1cfa0 0x560959e19d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.155+0000 7ff36d400640 1 --2- 172.21.15.31:0/3635866633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560959e1d960 0x560959e05130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.155+0000 7ff36d400640 1 -- 172.21.15.31:0/3635866633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560959e1cfa0 msgr2=0x560959e19d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.155+0000 7ff36d400640 1 --2- 172.21.15.31:0/3635866633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560959e1cfa0 0x560959e19d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.156+0000 7ff36d400640 1 -- 172.21.15.31:0/3635866633 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560959e13900 con 0x560959e1bde0 2024-04-25T01:23:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.156+0000 7ff36de00640 1 --2- 172.21.15.31:0/3635866633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560959e1cfa0 0x560959e19d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:23:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.156+0000 7ff36d400640 1 --2- 172.21.15.31:0/3635866633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1bde0 0x560959e04bf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff35c0094e0 tx=0x7ff35c02c0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24e320d763762c48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:23:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.157+0000 7ff35a600640 1 -- 172.21.15.31:0/3635866633 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35c004250 con 0x560959e1bde0 2024-04-25T01:23:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.158+0000 7ff35a600640 1 -- 172.21.15.31:0/3635866633 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff35c05b040 con 0x560959e1bde0 2024-04-25T01:23:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.158+0000 7ff35a600640 1 -- 172.21.15.31:0/3635866633 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35c060a00 con 0x560959e1bde0 2024-04-25T01:23:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.159+0000 7ff377824740 1 -- 172.21.15.31:0/3635866633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1bde0 msgr2=0x560959e04bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.159+0000 7ff377824740 1 --2- 172.21.15.31:0/3635866633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1bde0 0x560959e04bf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff35c0094e0 tx=0x7ff35c02c0a0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.159+0000 7ff377824740 1 -- 172.21.15.31:0/3635866633 shutdown_connections 2024-04-25T01:23:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.159+0000 7ff377824740 1 --2- 172.21.15.31:0/3635866633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560959e1d960 0x560959e05130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.159+0000 7ff377824740 1 --2- 172.21.15.31:0/3635866633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560959e1cfa0 0x560959e19d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.160+0000 7ff377824740 1 --2- 172.21.15.31:0/3635866633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1bde0 0x560959e04bf0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.160+0000 7ff377824740 1 -- 172.21.15.31:0/3635866633 >> 172.21.15.31:0/3635866633 conn(0x560959e06310 msgr2=0x560959e10e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:23:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.160+0000 7ff377824740 1 -- 172.21.15.31:0/3635866633 shutdown_connections 2024-04-25T01:23:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.160+0000 7ff377824740 1 -- 172.21.15.31:0/3635866633 wait complete. 2024-04-25T01:23:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.161+0000 7ff377824740 1 -- start start 2024-04-25T01:23:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.161+0000 7ff377824740 1 Processor -- start 2024-04-25T01:23:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.163+0000 7ff377824740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560959e1bde0 0x560959e17f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.163+0000 7ff377824740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560959e1cfa0 0x560959e18490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.163+0000 7ff377824740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1d960 0x560959e189d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.163+0000 7ff377824740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff37019cd30 con 0x560959e1cfa0 2024-04-25T01:23:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.163+0000 7ff377824740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff37019e1a0 con 0x560959e1bde0 2024-04-25T01:23:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.163+0000 7ff377824740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff37019d400 con 0x560959e1d960 2024-04-25T01:23:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.163+0000 7ff377824740 10 client.0 _ll_register_callbacks cb 0x7ff360009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:23:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.164+0000 7ff36d400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1d960 0x560959e189d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.164+0000 7ff36d400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1d960 0x560959e189d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:35190/0 (socket says 172.21.15.31:35190) 2024-04-25T01:23:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.164+0000 7ff36d400640 1 -- 172.21.15.31:0/756034079 learned_addr learned my addr 172.21.15.31:0/756034079 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:23:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.164+0000 7ff36d400640 1 --2- 172.21.15.31:0/756034079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560959e1bde0 0x560959e17f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.164+0000 7ff36de00640 1 --2- 172.21.15.31:0/756034079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560959e1cfa0 0x560959e18490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.165+0000 7ff36d400640 1 -- 172.21.15.31:0/756034079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560959e1bde0 msgr2=0x560959e17f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.165+0000 7ff36d400640 1 --2- 172.21.15.31:0/756034079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560959e1bde0 0x560959e17f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.165+0000 7ff36d400640 1 -- 172.21.15.31:0/756034079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560959e1cfa0 msgr2=0x560959e18490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.165+0000 7ff36d400640 1 --2- 172.21.15.31:0/756034079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560959e1cfa0 0x560959e18490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.166+0000 7ff36d400640 1 -- 172.21.15.31:0/756034079 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560959e19140 con 0x560959e1d960 2024-04-25T01:23:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.166+0000 7ff36de00640 1 --2- 172.21.15.31:0/756034079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560959e1cfa0 0x560959e18490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:23:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.166+0000 7ff36d400640 1 --2- 172.21.15.31:0/756034079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1d960 0x560959e189d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff35c00b9f0 tx=0x7ff35c00b780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b08fef63311d9dea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:23:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.166+0000 7ff34f400640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:23:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.167+0000 7ff34f400640 1 -- 172.21.15.31:0/756034079 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35c0607c0 con 0x560959e1d960 2024-04-25T01:23:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.167+0000 7ff377824740 1 -- 172.21.15.31:0/756034079 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff35c05b040 con 0x560959e1d960 2024-04-25T01:23:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.167+0000 7ff34f400640 1 -- 172.21.15.31:0/756034079 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7ff35c060960 con 0x560959e1d960 2024-04-25T01:23:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.168+0000 7ff34f400640 1 -- 172.21.15.31:0/756034079 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff35c06e2a0 con 0x560959e1d960 2024-04-25T01:23:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.168+0000 7ff377824740 10 client.25753 fetch_fsmap learned FSMap version 30 2024-04-25T01:23:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.168+0000 7ff377824740 1 -- 172.21.15.31:0/756034079 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560959e87170 con 0x560959e1d960 2024-04-25T01:23:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.168+0000 7ff34f400640 1 -- 172.21.15.31:0/756034079 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff35c05b040 con 0x560959e1d960 2024-04-25T01:23:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.169+0000 7ff34f400640 1 -- 172.21.15.31:0/756034079 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7ff35c07c430 con 0x560959e1d960 2024-04-25T01:23:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.170+0000 7ff377824740 10 client.25753 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T01:23:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.170+0000 7ff377824740 10 client.25753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:23:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.170+0000 7ff377824740 20 client.25753 populate_metadata read hostname 'smithi031' 2024-04-25T01:23:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.170+0000 7ff377824740 1 --2- 172.21.15.31:0/756034079 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560959e87560 0x560959e89980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.170+0000 7ff377824740 4 client.25753 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:23:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.170+0000 7ff377824740 1 -- 172.21.15.31:0/756034079 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560959d7aff0 con 0x560959e87560 2024-04-25T01:23:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.171+0000 7ff36de00640 1 --2- 172.21.15.31:0/756034079 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560959e87560 0x560959e89980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.171+0000 7ff36de00640 1 --2- 172.21.15.31:0/756034079 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560959e87560 0x560959e89980 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe496d31941dfd6d server_cookie=c1d30df6872a840f in_seq=0 out_seq=0 2024-04-25T01:23:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.171+0000 7ff34f400640 10 client.25753 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:23:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.924+0000 7ff34f400640 1 -- 172.21.15.31:0/756034079 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560959d7aff0 con 0x560959e87560 2024-04-25T01:23:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.924+0000 7ff34f400640 10 client.25753 handle_command_reply: tid=0 2024-04-25T01:23:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.924+0000 7ff377824740 1 client.25753 shutdown 2024-04-25T01:23:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.925+0000 7ff377824740 1 -- 172.21.15.31:0/756034079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1d960 msgr2=0x560959e189d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.925+0000 7ff377824740 1 --2- 172.21.15.31:0/756034079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1d960 0x560959e189d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff35c00b9f0 tx=0x7ff35c00b780 comp rx=0 tx=0).stop 2024-04-25T01:23:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.925+0000 7ff377824740 1 -- 172.21.15.31:0/756034079 shutdown_connections 2024-04-25T01:23:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.925+0000 7ff377824740 1 --2- 172.21.15.31:0/756034079 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x560959e87560 0x560959e89980 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.925+0000 7ff377824740 1 --2- 172.21.15.31:0/756034079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560959e1d960 0x560959e189d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.925+0000 7ff377824740 1 --2- 172.21.15.31:0/756034079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560959e1cfa0 0x560959e18490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.925+0000 7ff377824740 1 --2- 172.21.15.31:0/756034079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560959e1bde0 0x560959e17f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.925+0000 7ff377824740 1 -- 172.21.15.31:0/756034079 >> 172.21.15.31:0/756034079 conn(0x560959e06310 msgr2=0x560959e09300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:23:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.925+0000 7ff34f400640 0 client.25753 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:23:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.926+0000 7ff377824740 1 -- 172.21.15.31:0/756034079 shutdown_connections 2024-04-25T01:23:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.928+0000 7ff377824740 1 -- 172.21.15.31:0/756034079 wait complete. 2024-04-25T01:23:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.928+0000 7ff377824740 20 client.25753 trim_cache size 0 max 16384 2024-04-25T01:23:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:23:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:23:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:23:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:23:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.931+0000 7ff374a00640 1 -- 172.21.15.31:0/3245251331 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff3540a2e70 msgr2=0x7ff3540a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.931+0000 7ff374a00640 1 --2- 172.21.15.31:0/3245251331 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff3540a2e70 0x7ff3540a5330 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff3680a2fe0 tx=0x7ff368014f50 comp rx=0 tx=0).stop 2024-04-25T01:23:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.931+0000 7ff374a00640 1 -- 172.21.15.31:0/3245251331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff370192920 msgr2=0x7ff370194de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.931+0000 7ff374a00640 1 --2- 172.21.15.31:0/3245251331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff370192920 0x7ff370194de0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff368007e50 tx=0x7ff368014420 comp rx=0 tx=0).stop 2024-04-25T01:23:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.931+0000 7ff374a00640 1 -- 172.21.15.31:0/3245251331 shutdown_connections 2024-04-25T01:23:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.931+0000 7ff374a00640 1 --2- 172.21.15.31:0/3245251331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff370192920 0x7ff370194de0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.931+0000 7ff374a00640 1 --2- 172.21.15.31:0/3245251331 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff3540a2e70 0x7ff3540a5330 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.931+0000 7ff374a00640 1 --2- 172.21.15.31:0/3245251331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff370102250 0x7ff370070760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.931+0000 7ff374a00640 1 --2- 172.21.15.31:0/3245251331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3701006a0 0x7ff370070220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.931+0000 7ff374a00640 1 -- 172.21.15.31:0/3245251331 >> 172.21.15.31:0/3245251331 conn(0x7ff3700fbc30 msgr2=0x7ff370104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:23:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.932+0000 7ff374a00640 1 -- 172.21.15.31:0/3245251331 shutdown_connections 2024-04-25T01:23:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:09.932+0000 7ff374a00640 1 -- 172.21.15.31:0/3245251331 wait complete. 2024-04-25T01:23:10.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:09 smithi169 ceph-mon[32759]: pgmap v607: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 34 KiB/s rd, 97 MiB/s wr, 14.80k op/s 2024-04-25T01:23:10.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:09 smithi040 ceph-mon[32518]: pgmap v607: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 34 KiB/s rd, 97 MiB/s wr, 14.80k op/s 2024-04-25T01:23:10.250 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:09 smithi031 ceph-mon[28053]: pgmap v607: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 34 KiB/s rd, 97 MiB/s wr, 14.80k op/s 2024-04-25T01:23:12.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:11 smithi169 ceph-mon[32759]: pgmap v608: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 40 KiB/s rd, 90 MiB/s wr, 14.07k op/s 2024-04-25T01:23:12.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:11 smithi040 ceph-mon[32518]: pgmap v608: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 40 KiB/s rd, 90 MiB/s wr, 14.07k op/s 2024-04-25T01:23:12.250 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:11 smithi031 ceph-mon[28053]: pgmap v608: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 40 KiB/s rd, 90 MiB/s wr, 14.07k op/s 2024-04-25T01:23:14.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:13 smithi169 ceph-mon[32759]: pgmap v609: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 52 KiB/s rd, 106 MiB/s wr, 16.38k op/s 2024-04-25T01:23:14.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:13 smithi040 ceph-mon[32518]: pgmap v609: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 52 KiB/s rd, 106 MiB/s wr, 16.38k op/s 2024-04-25T01:23:14.250 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:13 smithi031 ceph-mon[28053]: pgmap v609: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 52 KiB/s rd, 106 MiB/s wr, 16.38k op/s 2024-04-25T01:23:16.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:15 smithi169 ceph-mon[32759]: pgmap v610: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 41 KiB/s rd, 91 MiB/s wr, 13.97k op/s 2024-04-25T01:23:16.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:15 smithi040 ceph-mon[32518]: pgmap v610: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 41 KiB/s rd, 91 MiB/s wr, 13.97k op/s 2024-04-25T01:23:16.251 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:15 smithi031 ceph-mon[28053]: pgmap v610: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 41 KiB/s rd, 91 MiB/s wr, 13.97k op/s 2024-04-25T01:23:18.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:17 smithi169 ceph-mon[32759]: pgmap v611: 97 pgs: 97 active+clean; 13 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 62 KiB/s rd, 101 MiB/s wr, 15.44k op/s 2024-04-25T01:23:18.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:17 smithi040 ceph-mon[32518]: pgmap v611: 97 pgs: 97 active+clean; 13 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 62 KiB/s rd, 101 MiB/s wr, 15.44k op/s 2024-04-25T01:23:18.252 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:17 smithi031 ceph-mon[28053]: pgmap v611: 97 pgs: 97 active+clean; 13 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 62 KiB/s rd, 101 MiB/s wr, 15.44k op/s 2024-04-25T01:23:20.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:19 smithi169 ceph-mon[32759]: pgmap v612: 97 pgs: 97 active+clean; 13 GiB data, 115 GiB used, 958 GiB / 1.0 TiB avail; 43 KiB/s rd, 104 MiB/s wr, 15.89k op/s 2024-04-25T01:23:20.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:19 smithi040 ceph-mon[32518]: pgmap v612: 97 pgs: 97 active+clean; 13 GiB data, 115 GiB used, 958 GiB / 1.0 TiB avail; 43 KiB/s rd, 104 MiB/s wr, 15.89k op/s 2024-04-25T01:23:20.250 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:19 smithi031 ceph-mon[28053]: pgmap v612: 97 pgs: 97 active+clean; 13 GiB data, 115 GiB used, 958 GiB / 1.0 TiB avail; 43 KiB/s rd, 104 MiB/s wr, 15.89k op/s 2024-04-25T01:23:21.250 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:20 smithi031 ceph-mon[28053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:23:21.439 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:20 smithi169 ceph-mon[32759]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:23:21.442 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:20 smithi040 ceph-mon[32518]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:23:22.251 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:22 smithi031 ceph-mon[28053]: pgmap v613: 97 pgs: 97 active+clean; 13 GiB data, 115 GiB used, 958 GiB / 1.0 TiB avail; 44 KiB/s rd, 93 MiB/s wr, 14.12k op/s 2024-04-25T01:23:22.251 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:22 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:23:22.439 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:22 smithi169 ceph-mon[32759]: pgmap v613: 97 pgs: 97 active+clean; 13 GiB data, 115 GiB used, 958 GiB / 1.0 TiB avail; 44 KiB/s rd, 93 MiB/s wr, 14.12k op/s 2024-04-25T01:23:22.439 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:22 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:23:22.442 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:22 smithi040 ceph-mon[32518]: pgmap v613: 97 pgs: 97 active+clean; 13 GiB data, 115 GiB used, 958 GiB / 1.0 TiB avail; 44 KiB/s rd, 93 MiB/s wr, 14.12k op/s 2024-04-25T01:23:22.442 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:22 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:23:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.174+0000 7f93c6a00640 1 -- 172.21.15.31:0/3491697414 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c0031940 con 0x7f93c8103210 2024-04-25T01:23:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.175+0000 7f93cd600640 1 -- 172.21.15.31:0/3491697414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93c8103210 msgr2=0x7f93c8109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.175+0000 7f93cd600640 1 --2- 172.21.15.31:0/3491697414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93c8103210 0x7f93c8109aa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f93c0009900 tx=0x7f93c002a410 comp rx=0 tx=0).stop 2024-04-25T01:23:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.176+0000 7f93cd600640 1 -- 172.21.15.31:0/3491697414 shutdown_connections 2024-04-25T01:23:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.176+0000 7f93cd600640 1 --2- 172.21.15.31:0/3491697414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93c8103210 0x7f93c8109aa0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.176+0000 7f93cd600640 1 --2- 172.21.15.31:0/3491697414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93c8102850 0x7f93c8102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.176+0000 7f93cd600640 1 --2- 172.21.15.31:0/3491697414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93c8101660 0x7f93c8101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.176+0000 7f93cd600640 1 -- 172.21.15.31:0/3491697414 >> 172.21.15.31:0/3491697414 conn(0x7f93c80fcbd0 msgr2=0x7f93c80ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:23:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.178+0000 7f93cd600640 1 -- 172.21.15.31:0/3491697414 shutdown_connections 2024-04-25T01:23:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.178+0000 7f93cd600640 1 -- 172.21.15.31:0/3491697414 wait complete. 2024-04-25T01:23:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.181+0000 7f93cd600640 1 Processor -- start 2024-04-25T01:23:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.184+0000 7f93cd600640 1 -- start start 2024-04-25T01:23:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.184+0000 7f93cd600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93c8101660 0x7f93c8195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.184+0000 7f93cd600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93c8102850 0x7f93c819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.184+0000 7f93cd600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93c8103210 0x7f93c81959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.184+0000 7f93cd600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f93c810d2f0 con 0x7f93c8103210 2024-04-25T01:23:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.184+0000 7f93cd600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f93c810d170 con 0x7f93c8101660 2024-04-25T01:23:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.184+0000 7f93cd600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f93c810d470 con 0x7f93c8102850 2024-04-25T01:23:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.186+0000 7f93c7e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93c8101660 0x7f93c8195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.186+0000 7f93c7e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93c8101660 0x7f93c8195470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:39024/0 (socket says 172.21.15.31:39024) 2024-04-25T01:23:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.186+0000 7f93c7e00640 1 -- 172.21.15.31:0/4154435564 learned_addr learned my addr 172.21.15.31:0/4154435564 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:23:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.188+0000 7f93c7400640 1 --2- 172.21.15.31:0/4154435564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93c8102850 0x7f93c819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.188+0000 7f93c7400640 1 -- 172.21.15.31:0/4154435564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93c8101660 msgr2=0x7f93c8195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.188+0000 7f93c7400640 1 --2- 172.21.15.31:0/4154435564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93c8101660 0x7f93c8195470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.188+0000 7f93c7400640 1 -- 172.21.15.31:0/4154435564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93c8103210 msgr2=0x7f93c81959b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.188+0000 7f93c7400640 1 --2- 172.21.15.31:0/4154435564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93c8103210 0x7f93c81959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.188+0000 7f93c7400640 1 -- 172.21.15.31:0/4154435564 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c806b160 con 0x7f93c8102850 2024-04-25T01:23:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.189+0000 7f93c7400640 1 --2- 172.21.15.31:0/4154435564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93c8102850 0x7f93c819aa10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f93b800cce0 tx=0x7f93b800a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:23:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.189+0000 7f93c4c00640 1 -- 172.21.15.31:0/4154435564 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b800f450 con 0x7f93c8102850 2024-04-25T01:23:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.190+0000 7f93c4c00640 1 -- 172.21.15.31:0/4154435564 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93b800fad0 con 0x7f93c8102850 2024-04-25T01:23:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.190+0000 7f93c4c00640 1 -- 172.21.15.31:0/4154435564 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b800ec30 con 0x7f93c8102850 2024-04-25T01:23:23.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.190+0000 7f93cd600640 1 -- 172.21.15.31:0/4154435564 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c806b450 con 0x7f93c8102850 2024-04-25T01:23:23.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.191+0000 7f93cd600640 1 -- 172.21.15.31:0/4154435564 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c806b920 con 0x7f93c8102850 2024-04-25T01:23:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.200+0000 7f93c4c00640 1 -- 172.21.15.31:0/4154435564 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f93b800fc80 con 0x7f93c8102850 2024-04-25T01:23:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.209+0000 7f93d1b63740 1 Processor -- start 2024-04-25T01:23:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.209+0000 7f93d1b63740 1 -- start start 2024-04-25T01:23:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.209+0000 7f93c4c00640 1 --2- 172.21.15.31:0/4154435564 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f93ac0a2c80 0x7f93ac0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.210+0000 7f93c4c00640 1 -- 172.21.15.31:0/4154435564 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f93b80cf510 con 0x7f93c8102850 2024-04-25T01:23:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.212+0000 7f93d1b63740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4e9fe50 0x5585d4ea0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.212+0000 7f93d1b63740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585d4ea0840 0x5585d4ea5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.212+0000 7f93d1b63740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5585d4ea59b0 0x5585d4ea7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.213+0000 7f93c7e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4e9fe50 0x5585d4ea0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.213+0000 7f93d1b63740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5585d4e02ab0 con 0x5585d4ea59b0 2024-04-25T01:23:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.213+0000 7f93d1b63740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5585d4dab640 con 0x5585d4e9fe50 2024-04-25T01:23:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.213+0000 7f93d1b63740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5585d4dfea70 con 0x5585d4ea0840 2024-04-25T01:23:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.214+0000 7f93c7e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4e9fe50 0x5585d4ea0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:39036/0 (socket says 172.21.15.31:39036) 2024-04-25T01:23:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.214+0000 7f93c7e00640 1 -- 172.21.15.31:0/3354331543 learned_addr learned my addr 172.21.15.31:0/3354331543 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:23:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.215+0000 7f93c7400640 1 --2- 172.21.15.31:0/3354331543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585d4ea0840 0x5585d4ea5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.216+0000 7f93c7e00640 1 -- 172.21.15.31:0/3354331543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585d4ea0840 msgr2=0x5585d4ea5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.216+0000 7f93c7e00640 1 --2- 172.21.15.31:0/3354331543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585d4ea0840 0x5585d4ea5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.216+0000 7f93c7e00640 1 -- 172.21.15.31:0/3354331543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5585d4ea59b0 msgr2=0x5585d4ea7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.216+0000 7f93c7e00640 1 --2- 172.21.15.31:0/3354331543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5585d4ea59b0 0x5585d4ea7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.216+0000 7f93c7e00640 1 -- 172.21.15.31:0/3354331543 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585d4b17210 con 0x5585d4e9fe50 2024-04-25T01:23:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.217+0000 7f93c7e00640 1 --2- 172.21.15.31:0/3354331543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4e9fe50 0x5585d4ea0270 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f93b0007810 tx=0x7f93b0007cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47e71a5f0b4d8cd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:23:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.217+0000 7f93bd200640 1 -- 172.21.15.31:0/3354331543 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b000d950 con 0x5585d4e9fe50 2024-04-25T01:23:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.218+0000 7f93ccc00640 1 --2- 172.21.15.31:0/4154435564 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f93ac0a2c80 0x7f93ac0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.218+0000 7f93bd200640 1 -- 172.21.15.31:0/3354331543 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93b0018070 con 0x5585d4e9fe50 2024-04-25T01:23:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.218+0000 7f93bd200640 1 -- 172.21.15.31:0/3354331543 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b000d950 con 0x5585d4e9fe50 2024-04-25T01:23:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.219+0000 7f93d1b63740 1 -- 172.21.15.31:0/3354331543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4e9fe50 msgr2=0x5585d4ea0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.219+0000 7f93d1b63740 1 --2- 172.21.15.31:0/3354331543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4e9fe50 0x5585d4ea0270 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f93b0007810 tx=0x7f93b0007cd0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.220+0000 7f93d1b63740 1 -- 172.21.15.31:0/3354331543 shutdown_connections 2024-04-25T01:23:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.220+0000 7f93d1b63740 1 --2- 172.21.15.31:0/3354331543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5585d4ea59b0 0x5585d4ea7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.221+0000 7f93d1b63740 1 --2- 172.21.15.31:0/3354331543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585d4ea0840 0x5585d4ea5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.221+0000 7f93d1b63740 1 --2- 172.21.15.31:0/3354331543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4e9fe50 0x5585d4ea0270 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.221+0000 7f93d1b63740 1 -- 172.21.15.31:0/3354331543 >> 172.21.15.31:0/3354331543 conn(0x5585d4e9b4c0 msgr2=0x5585d4e9b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:23:23.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.222+0000 7f93d1b63740 1 -- 172.21.15.31:0/3354331543 shutdown_connections 2024-04-25T01:23:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.223+0000 7f93d1b63740 1 -- 172.21.15.31:0/3354331543 wait complete. 2024-04-25T01:23:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.224+0000 7f93d1b63740 1 -- start start 2024-04-25T01:23:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.224+0000 7f93d1b63740 1 Processor -- start 2024-04-25T01:23:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.226+0000 7f93ccc00640 1 --2- 172.21.15.31:0/4154435564 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f93ac0a2c80 0x7f93ac0a5140 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f93c0005940 tx=0x7f93c00058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:23:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.228+0000 7f93d1b63740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5585d4e9fe50 0x5585d4eaa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.230+0000 7f93d1b63740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4ea0840 0x5585d4eaa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.230+0000 7f93c7400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4ea0840 0x5585d4eaa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.230+0000 7f93c7400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4ea0840 0x5585d4eaa5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:39040/0 (socket says 172.21.15.31:39040) 2024-04-25T01:23:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.231+0000 7f93d1b63740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585d4ea59b0 0x5585d4eb1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.232+0000 7f93c7400640 1 -- 172.21.15.31:0/4216365365 learned_addr learned my addr 172.21.15.31:0/4216365365 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:23:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.232+0000 7f93d1b63740 1 -- 172.21.15.31:0/4216365365 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5585d4ea8960 con 0x5585d4e9fe50 2024-04-25T01:23:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.232+0000 7f93d1b63740 1 -- 172.21.15.31:0/4216365365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x5585d4dbe990 con 0x5585d4ea0840 2024-04-25T01:23:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.232+0000 7f93d1b63740 1 -- 172.21.15.31:0/4216365365 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5585d4ea8ae0 con 0x5585d4ea59b0 2024-04-25T01:23:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.232+0000 7f93ccc00640 1 --2- 172.21.15.31:0/4216365365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585d4ea59b0 0x5585d4eb1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.233+0000 7f93ccc00640 1 -- 172.21.15.31:0/4216365365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4ea0840 msgr2=0x5585d4eaa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.233+0000 7f93ccc00640 1 --2- 172.21.15.31:0/4216365365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4ea0840 0x5585d4eaa5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.233+0000 7f93ccc00640 1 -- 172.21.15.31:0/4216365365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5585d4e9fe50 msgr2=0x5585d4eaa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.233+0000 7f93ccc00640 1 --2- 172.21.15.31:0/4216365365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5585d4e9fe50 0x5585d4eaa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.233+0000 7f93ccc00640 1 -- 172.21.15.31:0/4216365365 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585d4be3620 con 0x5585d4ea59b0 2024-04-25T01:23:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.235+0000 7f93c7400640 1 --2- 172.21.15.31:0/4216365365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4ea0840 0x5585d4eaa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:23:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.235+0000 7f93d1b63740 10 client.0 _ll_register_callbacks cb 0x5585d4dc0950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:23:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.235+0000 7f93d1b63740 1 -- 172.21.15.31:0/4216365365 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93b0018070 con 0x5585d4ea59b0 2024-04-25T01:23:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.235+0000 7f93ccc00640 1 --2- 172.21.15.31:0/4216365365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585d4ea59b0 0x5585d4eb1620 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f93c0009850 tx=0x7f93c0031ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6c8270f21daaf9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:23:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.235+0000 7f93b5600640 10 client.25762 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:23:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.236+0000 7f93b5600640 1 -- 172.21.15.31:0/4216365365 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c0054050 con 0x5585d4ea59b0 2024-04-25T01:23:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.237+0000 7f93b5600640 1 -- 172.21.15.31:0/4216365365 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f93c00046c0 con 0x5585d4ea59b0 2024-04-25T01:23:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.237+0000 7f93b5600640 1 -- 172.21.15.31:0/4216365365 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f93c004a070 con 0x5585d4ea59b0 2024-04-25T01:23:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.237+0000 7f93d1b63740 10 client.25762 fetch_fsmap learned FSMap version 31 2024-04-25T01:23:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.238+0000 7f93d1b63740 1 -- 172.21.15.31:0/4216365365 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({fsmap=31}) -- 0x5585d4eb4530 con 0x5585d4ea59b0 2024-04-25T01:23:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.240+0000 7f93b5600640 1 -- 172.21.15.31:0/4216365365 <== mon.1 v2:172.21.15.169:3300/0 4 ==== fsmap(e 31) ==== 8342+0+0 (secure 0 0 0) 0x7f93c0045040 con 0x5585d4ea59b0 2024-04-25T01:23:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.240+0000 7f93d1b63740 10 client.25762 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T01:23:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.240+0000 7f93d1b63740 10 client.25762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:23:23.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.240+0000 7f93d1b63740 20 client.25762 populate_metadata read hostname 'smithi031' 2024-04-25T01:23:23.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.241+0000 7f93d1b63740 1 --2- 172.21.15.31:0/4216365365 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5585d4eb5bd0 0x5585d4eb8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:23.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.241+0000 7f93d1b63740 4 client.25762 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:23:23.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.241+0000 7f93d1b63740 1 -- 172.21.15.31:0/4216365365 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585d4e0fff0 con 0x5585d4eb5bd0 2024-04-25T01:23:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.241+0000 7f93c7e00640 1 --2- 172.21.15.31:0/4216365365 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5585d4eb5bd0 0x5585d4eb8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.242+0000 7f93c7e00640 1 --2- 172.21.15.31:0/4216365365 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5585d4eb5bd0 0x5585d4eb8090 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea4a097b1c447103 server_cookie=3a88c025e5dbbc53 in_seq=0 out_seq=0 2024-04-25T01:23:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.242+0000 7f93b5600640 10 client.25762 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:23:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.243+0000 7f93b5600640 1 -- 172.21.15.31:0/4216365365 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5585d4e0fff0 con 0x5585d4eb5bd0 2024-04-25T01:23:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.244+0000 7f93b5600640 10 client.25762 handle_command_reply: tid=0 2024-04-25T01:23:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.244+0000 7f93d1b63740 1 client.25762 shutdown 2024-04-25T01:23:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.244+0000 7f93d1b63740 1 -- 172.21.15.31:0/4216365365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585d4ea59b0 msgr2=0x5585d4eb1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.244+0000 7f93d1b63740 1 --2- 172.21.15.31:0/4216365365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585d4ea59b0 0x5585d4eb1620 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f93c0009850 tx=0x7f93c0031ef0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.245+0000 7f93d1b63740 1 -- 172.21.15.31:0/4216365365 shutdown_connections 2024-04-25T01:23:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.245+0000 7f93d1b63740 1 --2- 172.21.15.31:0/4216365365 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5585d4eb5bd0 0x5585d4eb8090 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.245+0000 7f93d1b63740 1 --2- 172.21.15.31:0/4216365365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585d4ea59b0 0x5585d4eb1620 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.245+0000 7f93d1b63740 1 --2- 172.21.15.31:0/4216365365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4ea0840 0x5585d4eaa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.245+0000 7f93d1b63740 1 --2- 172.21.15.31:0/4216365365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5585d4e9fe50 0x5585d4eaa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.245+0000 7f93d1b63740 1 -- 172.21.15.31:0/4216365365 >> 172.21.15.31:0/4216365365 conn(0x5585d4e9b4c0 msgr2=0x5585d4e9a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:23:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.245+0000 7f93b5600640 0 client.25762 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:23:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.246+0000 7f93d1b63740 1 -- 172.21.15.31:0/4216365365 shutdown_connections 2024-04-25T01:23:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.248+0000 7f93d1b63740 1 -- 172.21.15.31:0/4216365365 wait complete. 2024-04-25T01:23:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.248+0000 7f93d1b63740 20 client.25762 trim_cache size 0 max 16384 2024-04-25T01:23:23.250 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:23 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:23:23.250 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:23 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:23:23.250 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:23 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:23:23.250 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:23 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:23:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.283+0000 7f93d1b63740 1 Processor -- start 2024-04-25T01:23:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.283+0000 7f93d1b63740 1 -- start start 2024-04-25T01:23:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.283+0000 7f93d1b63740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585d4e9fe50 0x5585d4ea1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.283+0000 7f93d1b63740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4ea0840 0x5585d4ea1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.283+0000 7f93d1b63740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5585d4ea59b0 0x5585d4eb15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.283+0000 7f93d1b63740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f93c81c2610 con 0x5585d4ea59b0 2024-04-25T01:23:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.283+0000 7f93d1b63740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f93c81bfee0 con 0x5585d4ea0840 2024-04-25T01:23:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.283+0000 7f93d1b63740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f93c8072480 con 0x5585d4e9fe50 2024-04-25T01:23:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.284+0000 7f93c7400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4ea0840 0x5585d4ea1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.284+0000 7f93c7400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4ea0840 0x5585d4ea1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:39054/0 (socket says 172.21.15.31:39054) 2024-04-25T01:23:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.284+0000 7f93c7400640 1 -- 172.21.15.31:0/2245280076 learned_addr learned my addr 172.21.15.31:0/2245280076 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:23:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.284+0000 7f93c7e00640 1 --2- 172.21.15.31:0/2245280076 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585d4e9fe50 0x5585d4ea1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.284+0000 7f93ccc00640 1 --2- 172.21.15.31:0/2245280076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5585d4ea59b0 0x5585d4eb15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.285+0000 7f93c7400640 1 -- 172.21.15.31:0/2245280076 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585d4e9fe50 msgr2=0x5585d4ea1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.285+0000 7f93c7400640 1 --2- 172.21.15.31:0/2245280076 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585d4e9fe50 0x5585d4ea1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.285+0000 7f93c7400640 1 -- 172.21.15.31:0/2245280076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5585d4ea59b0 msgr2=0x5585d4eb15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.285+0000 7f93c7400640 1 --2- 172.21.15.31:0/2245280076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5585d4ea59b0 0x5585d4eb15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.285+0000 7f93c7400640 1 -- 172.21.15.31:0/2245280076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585d4eb8710 con 0x5585d4ea0840 2024-04-25T01:23:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.287+0000 7f93c7400640 1 --2- 172.21.15.31:0/2245280076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4ea0840 0x5585d4ea1fd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f93b80a44a0 tx=0x7f93b800e6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7da6c1ce27627a53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:23:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.288+0000 7f93b7e00640 1 -- 172.21.15.31:0/2245280076 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b80b8020 con 0x5585d4ea0840 2024-04-25T01:23:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.288+0000 7f93b7e00640 1 -- 172.21.15.31:0/2245280076 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93b80a0110 con 0x5585d4ea0840 2024-04-25T01:23:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.288+0000 7f93b7e00640 1 -- 172.21.15.31:0/2245280076 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b80cf070 con 0x5585d4ea0840 2024-04-25T01:23:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.289+0000 7f93d1b63740 1 -- 172.21.15.31:0/2245280076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4ea0840 msgr2=0x5585d4ea1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.289+0000 7f93d1b63740 1 --2- 172.21.15.31:0/2245280076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4ea0840 0x5585d4ea1fd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f93b80a44a0 tx=0x7f93b800e6d0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.292+0000 7f93d1b63740 1 -- 172.21.15.31:0/2245280076 shutdown_connections 2024-04-25T01:23:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.292+0000 7f93d1b63740 1 --2- 172.21.15.31:0/2245280076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5585d4ea59b0 0x5585d4eb15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.292+0000 7f93d1b63740 1 --2- 172.21.15.31:0/2245280076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4ea0840 0x5585d4ea1fd0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.292+0000 7f93d1b63740 1 --2- 172.21.15.31:0/2245280076 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585d4e9fe50 0x5585d4ea1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.292+0000 7f93d1b63740 1 -- 172.21.15.31:0/2245280076 >> 172.21.15.31:0/2245280076 conn(0x5585d4e9b4c0 msgr2=0x5585d4ea8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:23:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.292+0000 7f93d1b63740 1 -- 172.21.15.31:0/2245280076 shutdown_connections 2024-04-25T01:23:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.292+0000 7f93d1b63740 1 -- 172.21.15.31:0/2245280076 wait complete. 2024-04-25T01:23:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.293+0000 7f93d1b63740 1 -- start start 2024-04-25T01:23:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.295+0000 7f93d1b63740 1 Processor -- start 2024-04-25T01:23:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.298+0000 7f93d1b63740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5585d4e9fe50 0x5585d4eaec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.299+0000 7f93d1b63740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4ea0840 0x5585d4eaf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.299+0000 7f93d1b63740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585d4ea59b0 0x5585d4eb0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.299+0000 7f93d1b63740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f93c8072af0 con 0x5585d4e9fe50 2024-04-25T01:23:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.299+0000 7f93d1b63740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f93c81bf810 con 0x5585d4ea0840 2024-04-25T01:23:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.299+0000 7f93d1b63740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f93c80731c0 con 0x5585d4ea59b0 2024-04-25T01:23:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.299+0000 7f93d1b63740 10 client.0 _ll_register_callbacks cb 0x7f93c0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:23:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.299+0000 7f93c7400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4ea0840 0x5585d4eaf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.300+0000 7f93c7400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4ea0840 0x5585d4eaf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:39070/0 (socket says 172.21.15.31:39070) 2024-04-25T01:23:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.300+0000 7f93c7400640 1 -- 172.21.15.31:0/2050484901 learned_addr learned my addr 172.21.15.31:0/2050484901 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:23:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.300+0000 7f93ccc00640 1 --2- 172.21.15.31:0/2050484901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585d4ea59b0 0x5585d4eb0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.301+0000 7f93c7e00640 1 --2- 172.21.15.31:0/2050484901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5585d4e9fe50 0x5585d4eaec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.302+0000 7f93c7400640 1 -- 172.21.15.31:0/2050484901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585d4ea59b0 msgr2=0x5585d4eb0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.302+0000 7f93c7400640 1 --2- 172.21.15.31:0/2050484901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585d4ea59b0 0x5585d4eb0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.302+0000 7f93c7400640 1 -- 172.21.15.31:0/2050484901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5585d4e9fe50 msgr2=0x5585d4eaec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.302+0000 7f93c7400640 1 --2- 172.21.15.31:0/2050484901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5585d4e9fe50 0x5585d4eaec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.302+0000 7f93c7400640 1 -- 172.21.15.31:0/2050484901 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585d4eb0730 con 0x5585d4ea0840 2024-04-25T01:23:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.302+0000 7f93ccc00640 1 --2- 172.21.15.31:0/2050484901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585d4ea59b0 0x5585d4eb0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:23:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.305+0000 7f93c7400640 1 --2- 172.21.15.31:0/2050484901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4ea0840 0x5585d4eaf1a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f93b800ab20 tx=0x7f93b8098090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c8599ee3c63f9c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:23:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.305+0000 7f93b5600640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T01:23:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.305+0000 7f93b5600640 1 -- 172.21.15.31:0/2050484901 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b80b8050 con 0x5585d4ea0840 2024-04-25T01:23:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.306+0000 7f93b5600640 1 -- 172.21.15.31:0/2050484901 <== mon.2 v2:172.21.15.40:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f93b8004b40 con 0x5585d4ea0840 2024-04-25T01:23:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.306+0000 7f93d1b63740 1 -- 172.21.15.31:0/2050484901 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93b80a0110 con 0x5585d4ea0840 2024-04-25T01:23:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.306+0000 7f93b5600640 1 -- 172.21.15.31:0/2050484901 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f93b80cf310 con 0x5585d4ea0840 2024-04-25T01:23:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.306+0000 7f93d1b63740 10 client.25721 fetch_fsmap learned FSMap version 31 2024-04-25T01:23:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.307+0000 7f93d1b63740 1 -- 172.21.15.31:0/2050484901 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5585d4f1c210 con 0x5585d4ea0840 2024-04-25T01:23:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.307+0000 7f93b5600640 1 -- 172.21.15.31:0/2050484901 <== mon.2 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93b80a0110 con 0x5585d4ea0840 2024-04-25T01:23:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.308+0000 7f93b5600640 1 -- 172.21.15.31:0/2050484901 <== mon.2 v2:172.21.15.40:3300/0 5 ==== fsmap(e 31) ==== 8342+0+0 (secure 0 0 0) 0x7f93b80b2040 con 0x5585d4ea0840 2024-04-25T01:23:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.311+0000 7f93d1b63740 10 client.25721 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T01:23:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.311+0000 7f93d1b63740 10 client.25721 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:23:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.311+0000 7f93d1b63740 20 client.25721 populate_metadata read hostname 'smithi031' 2024-04-25T01:23:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.311+0000 7f93d1b63740 1 --2- 172.21.15.31:0/2050484901 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5585d4f1ca30 0x5585d4f1ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.311+0000 7f93d1b63740 4 client.25721 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:23:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.311+0000 7f93d1b63740 1 -- 172.21.15.31:0/2050484901 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5585d4e0a340 con 0x5585d4f1ca30 2024-04-25T01:23:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.311+0000 7f93c7e00640 1 --2- 172.21.15.31:0/2050484901 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5585d4f1ca30 0x5585d4f1ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.312+0000 7f93c7e00640 1 --2- 172.21.15.31:0/2050484901 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5585d4f1ca30 0x5585d4f1ee50 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=503e9762b99082e1 server_cookie=e720a3b30337a9f5 in_seq=0 out_seq=0 2024-04-25T01:23:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.312+0000 7f93b5600640 10 client.25721 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:23:23.439 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:23 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:23:23.439 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:23 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:23:23.439 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:23 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:23:23.439 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:23 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:23:23.442 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:23 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:23:23.442 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:23 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:23:23.442 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:23 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:23:23.442 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:23 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:23:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.699+0000 7f93b5600640 1 -- 172.21.15.31:0/2050484901 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5585d4e0a340 con 0x5585d4f1ca30 2024-04-25T01:23:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.699+0000 7f93b5600640 10 client.25721 handle_command_reply: tid=0 2024-04-25T01:23:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.699+0000 7f93d1b63740 1 client.25721 shutdown 2024-04-25T01:23:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.700+0000 7f93d1b63740 1 -- 172.21.15.31:0/2050484901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4ea0840 msgr2=0x5585d4eaf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.700+0000 7f93d1b63740 1 --2- 172.21.15.31:0/2050484901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4ea0840 0x5585d4eaf1a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f93b800ab20 tx=0x7f93b8098090 comp rx=0 tx=0).stop 2024-04-25T01:23:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.700+0000 7f93d1b63740 1 -- 172.21.15.31:0/2050484901 shutdown_connections 2024-04-25T01:23:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.700+0000 7f93d1b63740 1 --2- 172.21.15.31:0/2050484901 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5585d4f1ca30 0x5585d4f1ee50 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.700+0000 7f93d1b63740 1 --2- 172.21.15.31:0/2050484901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5585d4ea59b0 0x5585d4eb0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.700+0000 7f93d1b63740 1 --2- 172.21.15.31:0/2050484901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5585d4ea0840 0x5585d4eaf1a0 secure :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f93b800ab20 tx=0x7f93b8098090 comp rx=0 tx=0).stop 2024-04-25T01:23:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.700+0000 7f93d1b63740 1 --2- 172.21.15.31:0/2050484901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5585d4e9fe50 0x5585d4eaec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.700+0000 7f93d1b63740 1 -- 172.21.15.31:0/2050484901 >> 172.21.15.31:0/2050484901 conn(0x5585d4e9b4c0 msgr2=0x5585d4eaa070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:23:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.701+0000 7f93b5600640 0 client.25721 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:23:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.701+0000 7f93d1b63740 1 -- 172.21.15.31:0/2050484901 shutdown_connections 2024-04-25T01:23:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.702+0000 7f93d1b63740 1 -- 172.21.15.31:0/2050484901 wait complete. 2024-04-25T01:23:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.702+0000 7f93d1b63740 20 client.25721 trim_cache size 0 max 16384 2024-04-25T01:23:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:23:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:23:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:23:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:23:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.703+0000 7f93cd600640 1 -- 172.21.15.31:0/4154435564 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f93ac0a2c80 msgr2=0x7f93ac0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.703+0000 7f93cd600640 1 --2- 172.21.15.31:0/4154435564 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f93ac0a2c80 0x7f93ac0a5140 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f93c0005940 tx=0x7f93c00058d0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.703+0000 7f93cd600640 1 -- 172.21.15.31:0/4154435564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93c8102850 msgr2=0x7f93c819aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.703+0000 7f93cd600640 1 --2- 172.21.15.31:0/4154435564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93c8102850 0x7f93c819aa10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f93b800cce0 tx=0x7f93b800a620 comp rx=0 tx=0).stop 2024-04-25T01:23:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.704+0000 7f93cd600640 1 -- 172.21.15.31:0/4154435564 shutdown_connections 2024-04-25T01:23:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.704+0000 7f93cd600640 1 --2- 172.21.15.31:0/4154435564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93c8103210 0x7f93c81959b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.704+0000 7f93cd600640 1 --2- 172.21.15.31:0/4154435564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93c8102850 0x7f93c819aa10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.704+0000 7f93cd600640 1 --2- 172.21.15.31:0/4154435564 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f93ac0a2c80 0x7f93ac0a5140 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.704+0000 7f93cd600640 1 --2- 172.21.15.31:0/4154435564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93c8101660 0x7f93c8195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.704+0000 7f93cd600640 1 -- 172.21.15.31:0/4154435564 >> 172.21.15.31:0/4154435564 conn(0x7f93c80fcbd0 msgr2=0x7f93c8103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:23:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.704+0000 7f93cd600640 1 -- 172.21.15.31:0/4154435564 shutdown_connections 2024-04-25T01:23:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:23.705+0000 7f93cd600640 1 -- 172.21.15.31:0/4154435564 wait complete. 2024-04-25T01:23:24.439 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:24 smithi169 ceph-mon[32759]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-04-25T01:23:24.439 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:24 smithi169 ceph-mon[32759]: Unable to set osd_memory_target on smithi169 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T01:23:24.439 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:24 smithi169 ceph-mon[32759]: pgmap v614: 97 pgs: 97 active+clean; 13 GiB data, 116 GiB used, 957 GiB / 1.0 TiB avail; 58 KiB/s rd, 109 MiB/s wr, 16.47k op/s 2024-04-25T01:23:24.439 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:24 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:23:24.440 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:24 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:23:24.440 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:24 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:23:24.442 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:24 smithi040 ceph-mon[32518]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-04-25T01:23:24.442 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:24 smithi040 ceph-mon[32518]: Unable to set osd_memory_target on smithi169 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T01:23:24.442 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:24 smithi040 ceph-mon[32518]: pgmap v614: 97 pgs: 97 active+clean; 13 GiB data, 116 GiB used, 957 GiB / 1.0 TiB avail; 58 KiB/s rd, 109 MiB/s wr, 16.47k op/s 2024-04-25T01:23:24.442 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:24 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:23:24.442 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:24 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:23:24.443 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:24 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:23:24.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:24 smithi031 ceph-mon[28053]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-04-25T01:23:24.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:24 smithi031 ceph-mon[28053]: Unable to set osd_memory_target on smithi169 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T01:23:24.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:24 smithi031 ceph-mon[28053]: pgmap v614: 97 pgs: 97 active+clean; 13 GiB data, 116 GiB used, 957 GiB / 1.0 TiB avail; 58 KiB/s rd, 109 MiB/s wr, 16.47k op/s 2024-04-25T01:23:24.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:24 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:23:24.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:24 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:23:24.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:24 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:23:26.439 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:26 smithi169 ceph-mon[32759]: pgmap v615: 97 pgs: 97 active+clean; 13 GiB data, 116 GiB used, 957 GiB / 1.0 TiB avail; 43 KiB/s rd, 100 MiB/s wr, 14.99k op/s 2024-04-25T01:23:26.440 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:26 smithi169 ceph-mon[32759]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:23:26.442 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:26 smithi040 ceph-mon[32518]: pgmap v615: 97 pgs: 97 active+clean; 13 GiB data, 116 GiB used, 957 GiB / 1.0 TiB avail; 43 KiB/s rd, 100 MiB/s wr, 14.99k op/s 2024-04-25T01:23:26.442 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:26 smithi040 ceph-mon[32518]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:23:26.503 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:26 smithi031 ceph-mon[28053]: pgmap v615: 97 pgs: 97 active+clean; 13 GiB data, 116 GiB used, 957 GiB / 1.0 TiB avail; 43 KiB/s rd, 100 MiB/s wr, 14.99k op/s 2024-04-25T01:23:26.503 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:26 smithi031 ceph-mon[28053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:23:28.439 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:28 smithi169 ceph-mon[32759]: pgmap v616: 97 pgs: 97 active+clean; 13 GiB data, 117 GiB used, 956 GiB / 1.0 TiB avail; 66 KiB/s rd, 101 MiB/s wr, 14.99k op/s 2024-04-25T01:23:28.442 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:28 smithi040 ceph-mon[32518]: pgmap v616: 97 pgs: 97 active+clean; 13 GiB data, 117 GiB used, 956 GiB / 1.0 TiB avail; 66 KiB/s rd, 101 MiB/s wr, 14.99k op/s 2024-04-25T01:23:28.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:28 smithi031 ceph-mon[28053]: pgmap v616: 97 pgs: 97 active+clean; 13 GiB data, 117 GiB used, 956 GiB / 1.0 TiB avail; 66 KiB/s rd, 101 MiB/s wr, 14.99k op/s 2024-04-25T01:23:30.439 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:30 smithi169 ceph-mon[32759]: pgmap v617: 97 pgs: 97 active+clean; 13 GiB data, 115 GiB used, 958 GiB / 1.0 TiB avail; 47 KiB/s rd, 100 MiB/s wr, 14.59k op/s 2024-04-25T01:23:30.442 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:30 smithi040 ceph-mon[32518]: pgmap v617: 97 pgs: 97 active+clean; 13 GiB data, 115 GiB used, 958 GiB / 1.0 TiB avail; 47 KiB/s rd, 100 MiB/s wr, 14.59k op/s 2024-04-25T01:23:30.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:30 smithi031 ceph-mon[28053]: pgmap v617: 97 pgs: 97 active+clean; 13 GiB data, 115 GiB used, 958 GiB / 1.0 TiB avail; 47 KiB/s rd, 100 MiB/s wr, 14.59k op/s 2024-04-25T01:23:32.439 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:32 smithi169 ceph-mon[32759]: pgmap v618: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 961 GiB / 1.0 TiB avail; 47 KiB/s rd, 95 MiB/s wr, 13.71k op/s 2024-04-25T01:23:32.442 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:32 smithi040 ceph-mon[32518]: pgmap v618: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 961 GiB / 1.0 TiB avail; 47 KiB/s rd, 95 MiB/s wr, 13.71k op/s 2024-04-25T01:23:32.500 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:32 smithi031 ceph-mon[28053]: pgmap v618: 97 pgs: 97 active+clean; 13 GiB data, 111 GiB used, 961 GiB / 1.0 TiB avail; 47 KiB/s rd, 95 MiB/s wr, 13.71k op/s 2024-04-25T01:23:34.439 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:34 smithi169 ceph-mon[32759]: pgmap v619: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 65 KiB/s rd, 111 MiB/s wr, 16.14k op/s 2024-04-25T01:23:34.442 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:34 smithi040 ceph-mon[32518]: pgmap v619: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 65 KiB/s rd, 111 MiB/s wr, 16.14k op/s 2024-04-25T01:23:34.502 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:34 smithi031 ceph-mon[28053]: pgmap v619: 97 pgs: 97 active+clean; 13 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 65 KiB/s rd, 111 MiB/s wr, 16.14k op/s 2024-04-25T01:23:36.439 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:36 smithi169 ceph-mon[32759]: pgmap v620: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 44 KiB/s rd, 92 MiB/s wr, 13.35k op/s 2024-04-25T01:23:36.442 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:36 smithi040 ceph-mon[32518]: pgmap v620: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 44 KiB/s rd, 92 MiB/s wr, 13.35k op/s 2024-04-25T01:23:36.500 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:36 smithi031 ceph-mon[28053]: pgmap v620: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 44 KiB/s rd, 92 MiB/s wr, 13.35k op/s 2024-04-25T01:23:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.969+0000 7f9fa3a00640 1 -- 172.21.15.31:0/1158399828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f9c102280 msgr2=0x7f9f9c104670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.969+0000 7f9fa3a00640 1 --2- 172.21.15.31:0/1158399828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f9c102280 0x7f9f9c104670 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009ca0 tx=0x7f9f9402a8e0 comp rx=0 tx=0).stop 2024-04-25T01:23:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.971+0000 7f9fa3a00640 1 -- 172.21.15.31:0/1158399828 shutdown_connections 2024-04-25T01:23:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.971+0000 7f9fa3a00640 1 --2- 172.21.15.31:0/1158399828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f9c1074e0 0x7f9f9c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.971+0000 7f9fa3a00640 1 --2- 172.21.15.31:0/1158399828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9c104bb0 0x7f9f9c106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.971+0000 7f9fa3a00640 1 --2- 172.21.15.31:0/1158399828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f9c102280 0x7f9f9c104670 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.971+0000 7f9fa3a00640 1 -- 172.21.15.31:0/1158399828 >> 172.21.15.31:0/1158399828 conn(0x7f9f9c0fbc10 msgr2=0x7f9f9c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:23:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.971+0000 7f9fa3a00640 1 -- 172.21.15.31:0/1158399828 shutdown_connections 2024-04-25T01:23:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.971+0000 7f9fa3a00640 1 -- 172.21.15.31:0/1158399828 wait complete. 2024-04-25T01:23:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.972+0000 7f9fa3a00640 1 Processor -- start 2024-04-25T01:23:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.974+0000 7f9fa3a00640 1 -- start start 2024-04-25T01:23:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.974+0000 7f9fa3a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9c102280 0x7f9f9c199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.974+0000 7f9fa3a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f9c104bb0 0x7f9f9c19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.974+0000 7f9fa3a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f9c1074e0 0x7f9f9c1a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.974+0000 7f9fa3a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9c10d4e0 con 0x7f9f9c102280 2024-04-25T01:23:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.974+0000 7f9fa3a00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9c10d360 con 0x7f9f9c104bb0 2024-04-25T01:23:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.975+0000 7f9fa3a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9c10d660 con 0x7f9f9c1074e0 2024-04-25T01:23:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.975+0000 7f9fa1a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f9c1074e0 0x7f9f9c1a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.975+0000 7f9f9be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f9c104bb0 0x7f9f9c19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.975+0000 7f9f9be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f9c104bb0 0x7f9f9c19a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:33748/0 (socket says 172.21.15.31:33748) 2024-04-25T01:23:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.975+0000 7f9fa1a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f9c1074e0 0x7f9f9c1a1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:58060/0 (socket says 172.21.15.31:58060) 2024-04-25T01:23:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.975+0000 7f9f9be00640 1 -- 172.21.15.31:0/3769693724 learned_addr learned my addr 172.21.15.31:0/3769693724 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:23:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.975+0000 7f9f9be00640 1 -- 172.21.15.31:0/3769693724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f9c1074e0 msgr2=0x7f9f9c1a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.975+0000 7f9f9be00640 1 --2- 172.21.15.31:0/3769693724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f9c1074e0 0x7f9f9c1a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.975+0000 7f9f9be00640 1 -- 172.21.15.31:0/3769693724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9c102280 msgr2=0x7f9f9c199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.975+0000 7f9f9be00640 1 --2- 172.21.15.31:0/3769693724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9c102280 0x7f9f9c199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.975+0000 7f9f9be00640 1 -- 172.21.15.31:0/3769693724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f9c06b2c0 con 0x7f9f9c104bb0 2024-04-25T01:23:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.976+0000 7f9fa1a00640 1 --2- 172.21.15.31:0/3769693724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f9c1074e0 0x7f9f9c1a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:23:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.976+0000 7f9f9be00640 1 --2- 172.21.15.31:0/3769693724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f9c104bb0 0x7f9f9c19a200 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800b450 tx=0x7f9f8800b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:23:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.977+0000 7f9f99600640 1 -- 172.21.15.31:0/3769693724 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f88015020 con 0x7f9f9c104bb0 2024-04-25T01:23:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.977+0000 7f9fa3a00640 1 -- 172.21.15.31:0/3769693724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f9c06b5b0 con 0x7f9f9c104bb0 2024-04-25T01:23:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.977+0000 7f9fa3a00640 1 -- 172.21.15.31:0/3769693724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f9c06bbd0 con 0x7f9f9c104bb0 2024-04-25T01:23:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.978+0000 7f9f99600640 1 -- 172.21.15.31:0/3769693724 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f88011d70 con 0x7f9f9c104bb0 2024-04-25T01:23:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.978+0000 7f9f99600640 1 -- 172.21.15.31:0/3769693724 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8800f340 con 0x7f9f9c104bb0 2024-04-25T01:23:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.993+0000 7f9f99600640 1 -- 172.21.15.31:0/3769693724 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9f88018020 con 0x7f9f9c104bb0 2024-04-25T01:23:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.993+0000 7f9fa67ef740 1 Processor -- start 2024-04-25T01:23:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.994+0000 7f9fa67ef740 1 -- start start 2024-04-25T01:23:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.994+0000 7f9fa67ef740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55841f32cca0 0x55841f32d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.995+0000 7f9fa67ef740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55841f32d690 0x55841f332190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.995+0000 7f9fa67ef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55841f332800 0x55841f334c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.995+0000 7f9fa67ef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55841f28fab0 con 0x55841f332800 2024-04-25T01:23:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.995+0000 7f9fa67ef740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55841f238640 con 0x55841f32cca0 2024-04-25T01:23:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.995+0000 7f9fa67ef740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55841f28ba70 con 0x55841f32d690 2024-04-25T01:23:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.996+0000 7f9f99600640 1 --2- 172.21.15.31:0/3769693724 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f9f800a2d30 0x7f9f800a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.996+0000 7f9fa1a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55841f32cca0 0x55841f32d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.996+0000 7f9fa1a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55841f32cca0 0x55841f32d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:33756/0 (socket says 172.21.15.31:33756) 2024-04-25T01:23:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.996+0000 7f9fa1a00640 1 -- 172.21.15.31:0/1940817358 learned_addr learned my addr 172.21.15.31:0/1940817358 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:23:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.996+0000 7f9f99600640 1 -- 172.21.15.31:0/3769693724 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f9f880ce990 con 0x7f9f9c104bb0 2024-04-25T01:23:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.996+0000 7f9fa1000640 1 --2- 172.21.15.31:0/1940817358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55841f32d690 0x55841f332190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.996+0000 7f9fa1a00640 1 --2- 172.21.15.31:0/3769693724 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f9f800a2d30 0x7f9f800a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.996+0000 7f9f9be00640 1 --2- 172.21.15.31:0/1940817358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55841f332800 0x55841f334c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.997+0000 7f9fa1000640 1 -- 172.21.15.31:0/1940817358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55841f32cca0 msgr2=0x55841f32d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.997+0000 7f9fa1000640 1 --2- 172.21.15.31:0/1940817358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55841f32cca0 0x55841f32d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.997+0000 7f9fa1000640 1 -- 172.21.15.31:0/1940817358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55841f332800 msgr2=0x55841f334c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.998+0000 7f9fa1000640 1 --2- 172.21.15.31:0/1940817358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55841f332800 0x55841f334c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.998+0000 7f9fa1000640 1 -- 172.21.15.31:0/1940817358 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55841efa4210 con 0x55841f32d690 2024-04-25T01:23:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.998+0000 7f9f9be00640 1 --2- 172.21.15.31:0/1940817358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55841f332800 0x55841f334c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:23:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:36.998+0000 7f9fa1a00640 1 --2- 172.21.15.31:0/3769693724 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f9f800a2d30 0x7f9f800a51f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55841f32def0 tx=0x7f9f8c004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:23:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.001+0000 7f9fa1000640 1 --2- 172.21.15.31:0/1940817358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55841f32d690 0x55841f332190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9f9402ae00 tx=0x7f9f94009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be05dee56ec4aef2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:23:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.001+0000 7f9f92000640 1 -- 172.21.15.31:0/1940817358 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f94031cc0 con 0x55841f32d690 2024-04-25T01:23:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.002+0000 7f9f92000640 1 -- 172.21.15.31:0/1940817358 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f94036050 con 0x55841f32d690 2024-04-25T01:23:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.003+0000 7f9fa67ef740 1 -- 172.21.15.31:0/1940817358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55841f32d690 msgr2=0x55841f332190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.003+0000 7f9fa67ef740 1 --2- 172.21.15.31:0/1940817358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55841f32d690 0x55841f332190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9f9402ae00 tx=0x7f9f94009a20 comp rx=0 tx=0).stop 2024-04-25T01:23:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.004+0000 7f9fa67ef740 1 -- 172.21.15.31:0/1940817358 shutdown_connections 2024-04-25T01:23:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.004+0000 7f9fa67ef740 1 --2- 172.21.15.31:0/1940817358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55841f332800 0x55841f334c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.004+0000 7f9fa67ef740 1 --2- 172.21.15.31:0/1940817358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55841f32d690 0x55841f332190 secure :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9f9402ae00 tx=0x7f9f94009a20 comp rx=0 tx=0).stop 2024-04-25T01:23:36.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.004+0000 7f9fa67ef740 1 --2- 172.21.15.31:0/1940817358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55841f32cca0 0x55841f32d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:36.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.004+0000 7f9fa67ef740 1 -- 172.21.15.31:0/1940817358 >> 172.21.15.31:0/1940817358 conn(0x55841f328310 msgr2=0x55841f328710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:23:36.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.006+0000 7f9fa67ef740 1 -- 172.21.15.31:0/1940817358 shutdown_connections 2024-04-25T01:23:36.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.006+0000 7f9fa67ef740 1 -- 172.21.15.31:0/1940817358 wait complete. 2024-04-25T01:23:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.007+0000 7f9fa67ef740 1 -- start start 2024-04-25T01:23:36.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.009+0000 7f9fa67ef740 1 Processor -- start 2024-04-25T01:23:36.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.009+0000 7f9fa67ef740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55841f32cca0 0x55841f337300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:36.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.009+0000 7f9fa67ef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55841f332800 0x55841f337840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:36.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.010+0000 7f9fa67ef740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55841f33c870 0x55841f33ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:36.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.010+0000 7f9fa67ef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55841f3357f0 con 0x55841f332800 2024-04-25T01:23:36.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.010+0000 7f9fa67ef740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55841f24b990 con 0x55841f33c870 2024-04-25T01:23:36.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.010+0000 7f9fa67ef740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55841f335970 con 0x55841f32cca0 2024-04-25T01:23:36.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.010+0000 7f9fa1000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55841f32cca0 0x55841f337300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:36.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.010+0000 7f9fa1000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55841f32cca0 0x55841f337300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:58088/0 (socket says 172.21.15.31:58088) 2024-04-25T01:23:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.010+0000 7f9fa1000640 1 -- 172.21.15.31:0/1673369331 learned_addr learned my addr 172.21.15.31:0/1673369331 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:23:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.010+0000 7f9fa1a00640 1 --2- 172.21.15.31:0/1673369331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55841f33c870 0x55841f33ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.010+0000 7f9fa67ef740 10 client.0 _ll_register_callbacks cb 0x55841f24d950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:23:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.011+0000 7f9fa1a00640 1 -- 172.21.15.31:0/1673369331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55841f32cca0 msgr2=0x55841f337300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.011+0000 7f9fa1a00640 1 --2- 172.21.15.31:0/1673369331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55841f32cca0 0x55841f337300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.011+0000 7f9fa1a00640 1 -- 172.21.15.31:0/1673369331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55841f332800 msgr2=0x55841f337840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.011+0000 7f9fa1a00640 1 --2- 172.21.15.31:0/1673369331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55841f332800 0x55841f337840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.011+0000 7f9fa1a00640 1 -- 172.21.15.31:0/1673369331 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55841f070620 con 0x55841f33c870 2024-04-25T01:23:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.014+0000 7f9fa1a00640 1 --2- 172.21.15.31:0/1673369331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55841f33c870 0x55841f33ec60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c00f690 tx=0x7f9f8c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a01f8acc6d42e67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:23:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.015+0000 7f9f86a00640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T01:23:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.015+0000 7f9f86a00640 1 -- 172.21.15.31:0/1673369331 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c019070 con 0x55841f33c870 2024-04-25T01:23:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.015+0000 7f9f86a00640 1 -- 172.21.15.31:0/1673369331 <== mon.2 v2:172.21.15.40:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f9f8c01f840 con 0x55841f33c870 2024-04-25T01:23:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.015+0000 7f9fa67ef740 1 -- 172.21.15.31:0/1673369331 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55841f29cff0 con 0x55841f33c870 2024-04-25T01:23:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.017+0000 7f9f86a00640 1 -- 172.21.15.31:0/1673369331 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9f8c01fa40 con 0x55841f33c870 2024-04-25T01:23:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.017+0000 7f9fa67ef740 10 client.25733 fetch_fsmap learned FSMap version 32 2024-04-25T01:23:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.017+0000 7f9fa67ef740 1 -- 172.21.15.31:0/1673369331 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55841f341cc0 con 0x55841f33c870 2024-04-25T01:23:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.018+0000 7f9f86a00640 1 -- 172.21.15.31:0/1673369331 <== mon.2 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55841f29cff0 con 0x55841f33c870 2024-04-25T01:23:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.018+0000 7f9f86a00640 1 -- 172.21.15.31:0/1673369331 <== mon.2 v2:172.21.15.40:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f9f8c020430 con 0x55841f33c870 2024-04-25T01:23:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.019+0000 7f9fa67ef740 10 client.25733 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T01:23:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.019+0000 7f9fa67ef740 10 client.25733 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:23:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.019+0000 7f9fa67ef740 20 client.25733 populate_metadata read hostname 'smithi031' 2024-04-25T01:23:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.020+0000 7f9fa67ef740 1 --2- 172.21.15.31:0/1673369331 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55841f343470 0x55841f345910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.020+0000 7f9fa67ef740 4 client.25733 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:23:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.020+0000 7f9fa67ef740 1 -- 172.21.15.31:0/1673369331 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55841f297340 con 0x55841f343470 2024-04-25T01:23:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.020+0000 7f9fa1000640 1 --2- 172.21.15.31:0/1673369331 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55841f343470 0x55841f345910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.022+0000 7f9fa1000640 1 --2- 172.21.15.31:0/1673369331 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55841f343470 0x55841f345910 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=217d733fe03d7481 server_cookie=c70fa3ff8b748308 in_seq=0 out_seq=0 2024-04-25T01:23:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.022+0000 7f9f86a00640 10 client.25733 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:23:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.023+0000 7f9f86a00640 1 -- 172.21.15.31:0/1673369331 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55841f297340 con 0x55841f343470 2024-04-25T01:23:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.023+0000 7f9f86a00640 10 client.25733 handle_command_reply: tid=0 2024-04-25T01:23:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.025+0000 7f9fa67ef740 1 client.25733 shutdown 2024-04-25T01:23:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.025+0000 7f9fa67ef740 1 -- 172.21.15.31:0/1673369331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55841f33c870 msgr2=0x55841f33ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.025+0000 7f9fa67ef740 1 --2- 172.21.15.31:0/1673369331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55841f33c870 0x55841f33ec60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c00f690 tx=0x7f9f8c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:23:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.025+0000 7f9fa67ef740 1 -- 172.21.15.31:0/1673369331 shutdown_connections 2024-04-25T01:23:37.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.025+0000 7f9fa67ef740 1 --2- 172.21.15.31:0/1673369331 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55841f343470 0x55841f345910 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:37.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.025+0000 7f9fa67ef740 1 --2- 172.21.15.31:0/1673369331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55841f33c870 0x55841f33ec60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:37.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.025+0000 7f9fa67ef740 1 --2- 172.21.15.31:0/1673369331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55841f332800 0x55841f337840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:37.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.025+0000 7f9fa67ef740 1 --2- 172.21.15.31:0/1673369331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55841f32cca0 0x55841f337300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:37.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.025+0000 7f9fa67ef740 1 -- 172.21.15.31:0/1673369331 >> 172.21.15.31:0/1673369331 conn(0x55841f328310 msgr2=0x55841f327480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:23:37.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.025+0000 7f9f86a00640 0 client.25733 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:23:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.026+0000 7f9fa67ef740 1 -- 172.21.15.31:0/1673369331 shutdown_connections 2024-04-25T01:23:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.026+0000 7f9fa67ef740 1 -- 172.21.15.31:0/1673369331 wait complete. 2024-04-25T01:23:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.026+0000 7f9fa67ef740 20 client.25733 trim_cache size 0 max 16384 2024-04-25T01:23:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.073+0000 7f9fa67ef740 1 Processor -- start 2024-04-25T01:23:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.073+0000 7f9fa67ef740 1 -- start start 2024-04-25T01:23:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.073+0000 7f9fa67ef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55841f32cca0 0x55841f32aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.073+0000 7f9fa67ef740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55841f332800 0x55841f32b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.073+0000 7f9fa67ef740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55841f33c870 0x55841f32ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.073+0000 7f9fa67ef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c010c30 con 0x55841f32cca0 2024-04-25T01:23:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.073+0000 7f9fa67ef740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c00e500 con 0x55841f33c870 2024-04-25T01:23:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.073+0000 7f9fa67ef740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c00c350 con 0x55841f332800 2024-04-25T01:23:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.074+0000 7f9fa1a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55841f33c870 0x55841f32ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.074+0000 7f9fa1a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55841f33c870 0x55841f32ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:33776/0 (socket says 172.21.15.31:33776) 2024-04-25T01:23:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.074+0000 7f9fa1a00640 1 -- 172.21.15.31:0/2464510204 learned_addr learned my addr 172.21.15.31:0/2464510204 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:23:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.074+0000 7f9f9be00640 1 --2- 172.21.15.31:0/2464510204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55841f332800 0x55841f32b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.075+0000 7f9fa1000640 1 --2- 172.21.15.31:0/2464510204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55841f32cca0 0x55841f32aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.076+0000 7f9f9be00640 1 -- 172.21.15.31:0/2464510204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55841f33c870 msgr2=0x55841f32ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.076+0000 7f9f9be00640 1 --2- 172.21.15.31:0/2464510204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55841f33c870 0x55841f32ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.076+0000 7f9f9be00640 1 -- 172.21.15.31:0/2464510204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55841f32cca0 msgr2=0x55841f32aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.077+0000 7f9f9be00640 1 --2- 172.21.15.31:0/2464510204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55841f32cca0 0x55841f32aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.078+0000 7f9f9be00640 1 -- 172.21.15.31:0/2464510204 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55841f327e10 con 0x55841f332800 2024-04-25T01:23:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.078+0000 7f9fa1000640 1 --2- 172.21.15.31:0/2464510204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55841f32cca0 0x55841f32aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:23:37.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.079+0000 7f9f9be00640 1 --2- 172.21.15.31:0/2464510204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55841f332800 0x55841f32b530 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x55841f3387e0 tx=0x7f9f880b44a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db9b69c4020f5068 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:23:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.080+0000 7f9f91600640 1 -- 172.21.15.31:0/2464510204 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8809f590 con 0x55841f332800 2024-04-25T01:23:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.082+0000 7f9f91600640 1 -- 172.21.15.31:0/2464510204 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f880a3e50 con 0x55841f332800 2024-04-25T01:23:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.083+0000 7f9fa67ef740 1 -- 172.21.15.31:0/2464510204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55841f332800 msgr2=0x55841f32b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.083+0000 7f9fa67ef740 1 --2- 172.21.15.31:0/2464510204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55841f332800 0x55841f32b530 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x55841f3387e0 tx=0x7f9f880b44a0 comp rx=0 tx=0).stop 2024-04-25T01:23:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.084+0000 7f9fa67ef740 1 -- 172.21.15.31:0/2464510204 shutdown_connections 2024-04-25T01:23:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.084+0000 7f9fa67ef740 1 --2- 172.21.15.31:0/2464510204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55841f33c870 0x55841f32ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.084+0000 7f9fa67ef740 1 --2- 172.21.15.31:0/2464510204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55841f332800 0x55841f32b530 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.084+0000 7f9fa67ef740 1 --2- 172.21.15.31:0/2464510204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55841f32cca0 0x55841f32aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.084+0000 7f9fa67ef740 1 -- 172.21.15.31:0/2464510204 >> 172.21.15.31:0/2464510204 conn(0x55841f328310 msgr2=0x55841f336680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:23:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.085+0000 7f9fa67ef740 1 -- 172.21.15.31:0/2464510204 shutdown_connections 2024-04-25T01:23:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.085+0000 7f9fa67ef740 1 -- 172.21.15.31:0/2464510204 wait complete. 2024-04-25T01:23:37.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.085+0000 7f9fa67ef740 1 -- start start 2024-04-25T01:23:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.086+0000 7f9fa67ef740 1 Processor -- start 2024-04-25T01:23:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.087+0000 7f9fa67ef740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55841f32cca0 0x55841f354d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.087+0000 7f9fa67ef740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55841f332800 0x55841f3552b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.087+0000 7f9fa67ef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55841f33c870 0x55841f3562c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.087+0000 7f9fa1000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55841f32cca0 0x55841f354d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.087+0000 7f9f9be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55841f332800 0x55841f3552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.087+0000 7f9fa1000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55841f32cca0 0x55841f354d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:58108/0 (socket says 172.21.15.31:58108) 2024-04-25T01:23:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.087+0000 7f9f9be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55841f332800 0x55841f3552b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:33778/0 (socket says 172.21.15.31:33778) 2024-04-25T01:23:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.089+0000 7f9fa1000640 1 -- 172.21.15.31:0/1029828151 learned_addr learned my addr 172.21.15.31:0/1029828151 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:23:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.089+0000 7f9fa67ef740 1 -- 172.21.15.31:0/1029828151 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c00c9c0 con 0x55841f33c870 2024-04-25T01:23:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.089+0000 7f9fa1a00640 1 --2- 172.21.15.31:0/1029828151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55841f33c870 0x55841f3562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.089+0000 7f9fa67ef740 1 -- 172.21.15.31:0/1029828151 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c00de30 con 0x55841f332800 2024-04-25T01:23:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.089+0000 7f9fa67ef740 1 -- 172.21.15.31:0/1029828151 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c00d090 con 0x55841f32cca0 2024-04-25T01:23:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.090+0000 7f9fa67ef740 10 client.0 _ll_register_callbacks cb 0x7f9f8c013f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:23:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.091+0000 7f9f9be00640 1 -- 172.21.15.31:0/1029828151 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55841f32cca0 msgr2=0x55841f354d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.091+0000 7f9f9be00640 1 --2- 172.21.15.31:0/1029828151 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55841f32cca0 0x55841f354d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.091+0000 7f9f9be00640 1 -- 172.21.15.31:0/1029828151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55841f33c870 msgr2=0x55841f3562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.091+0000 7f9f9be00640 1 --2- 172.21.15.31:0/1029828151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55841f33c870 0x55841f3562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:37.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.091+0000 7f9f9be00640 1 -- 172.21.15.31:0/1029828151 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55841f3568b0 con 0x55841f332800 2024-04-25T01:23:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.091+0000 7f9f9be00640 1 --2- 172.21.15.31:0/1029828151 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55841f332800 0x55841f3552b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55841f3387e0 tx=0x7f9f8809b0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56cd66af32ecdcde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:23:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.091+0000 7f9f86a00640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T01:23:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.091+0000 7f9f86a00640 1 -- 172.21.15.31:0/1029828151 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8809f590 con 0x55841f332800 2024-04-25T01:23:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.092+0000 7f9fa67ef740 1 -- 172.21.15.31:0/1029828151 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f880a3e50 con 0x55841f332800 2024-04-25T01:23:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.092+0000 7f9f86a00640 1 -- 172.21.15.31:0/1029828151 <== mon.2 v2:172.21.15.40:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f9f8809f120 con 0x55841f332800 2024-04-25T01:23:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.092+0000 7f9f86a00640 1 -- 172.21.15.31:0/1029828151 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9f8809f320 con 0x55841f332800 2024-04-25T01:23:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.094+0000 7f9fa67ef740 10 client.25739 fetch_fsmap learned FSMap version 32 2024-04-25T01:23:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.094+0000 7f9fa67ef740 1 -- 172.21.15.31:0/1029828151 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55841f3a9e50 con 0x55841f332800 2024-04-25T01:23:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.094+0000 7f9f86a00640 1 -- 172.21.15.31:0/1029828151 <== mon.2 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f880a3e50 con 0x55841f332800 2024-04-25T01:23:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.096+0000 7f9f86a00640 1 -- 172.21.15.31:0/1029828151 <== mon.2 v2:172.21.15.40:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f9f88011890 con 0x55841f332800 2024-04-25T01:23:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.096+0000 7f9fa67ef740 10 client.25739 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T01:23:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.096+0000 7f9fa67ef740 10 client.25739 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:23:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.096+0000 7f9fa67ef740 20 client.25739 populate_metadata read hostname 'smithi031' 2024-04-25T01:23:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.096+0000 7f9fa67ef740 1 --2- 172.21.15.31:0/1029828151 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55841f3aa770 0x55841f3acb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.096+0000 7f9fa67ef740 4 client.25739 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:23:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.096+0000 7f9fa67ef740 1 -- 172.21.15.31:0/1029828151 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55841f2157a0 con 0x55841f3aa770 2024-04-25T01:23:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.097+0000 7f9fa1000640 1 --2- 172.21.15.31:0/1029828151 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55841f3aa770 0x55841f3acb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.099+0000 7f9fa1000640 1 --2- 172.21.15.31:0/1029828151 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55841f3aa770 0x55841f3acb90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6ff33b31d3ce75a server_cookie=c415edf9e720d03e in_seq=0 out_seq=0 2024-04-25T01:23:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.099+0000 7f9f86a00640 10 client.25739 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:23:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.891+0000 7f9f86a00640 1 -- 172.21.15.31:0/1029828151 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55841f2157a0 con 0x55841f3aa770 2024-04-25T01:23:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.891+0000 7f9f86a00640 10 client.25739 handle_command_reply: tid=0 2024-04-25T01:23:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.891+0000 7f9fa67ef740 1 client.25739 shutdown 2024-04-25T01:23:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.891+0000 7f9fa67ef740 1 -- 172.21.15.31:0/1029828151 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55841f332800 msgr2=0x55841f3552b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.891+0000 7f9fa67ef740 1 --2- 172.21.15.31:0/1029828151 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55841f332800 0x55841f3552b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55841f3387e0 tx=0x7f9f8809b0e0 comp rx=0 tx=0).stop 2024-04-25T01:23:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.891+0000 7f9fa67ef740 1 -- 172.21.15.31:0/1029828151 shutdown_connections 2024-04-25T01:23:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.891+0000 7f9fa67ef740 1 --2- 172.21.15.31:0/1029828151 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55841f3aa770 0x55841f3acb90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.891+0000 7f9fa67ef740 1 --2- 172.21.15.31:0/1029828151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55841f33c870 0x55841f3562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.891+0000 7f9fa67ef740 1 --2- 172.21.15.31:0/1029828151 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55841f332800 0x55841f3552b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.891+0000 7f9fa67ef740 1 --2- 172.21.15.31:0/1029828151 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55841f32cca0 0x55841f354d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.891+0000 7f9fa67ef740 1 -- 172.21.15.31:0/1029828151 >> 172.21.15.31:0/1029828151 conn(0x55841f328310 msgr2=0x55841f326bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:23:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.892+0000 7f9f86a00640 0 client.25739 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:23:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.892+0000 7f9fa67ef740 1 -- 172.21.15.31:0/1029828151 shutdown_connections 2024-04-25T01:23:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.892+0000 7f9fa67ef740 1 -- 172.21.15.31:0/1029828151 wait complete. 2024-04-25T01:23:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.893+0000 7f9fa67ef740 20 client.25739 trim_cache size 0 max 16384 2024-04-25T01:23:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:23:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:23:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:23:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:23:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.894+0000 7f9fa3a00640 1 -- 172.21.15.31:0/3769693724 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f9f800a2d30 msgr2=0x7f9f800a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.894+0000 7f9fa3a00640 1 --2- 172.21.15.31:0/3769693724 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f9f800a2d30 0x7f9f800a51f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55841f32def0 tx=0x7f9f8c004730 comp rx=0 tx=0).stop 2024-04-25T01:23:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.894+0000 7f9fa3a00640 1 -- 172.21.15.31:0/3769693724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f9c104bb0 msgr2=0x7f9f9c19a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.894+0000 7f9fa3a00640 1 --2- 172.21.15.31:0/3769693724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f9c104bb0 0x7f9f9c19a200 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800b450 tx=0x7f9f8800b910 comp rx=0 tx=0).stop 2024-04-25T01:23:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.894+0000 7f9fa3a00640 1 -- 172.21.15.31:0/3769693724 shutdown_connections 2024-04-25T01:23:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.894+0000 7f9fa3a00640 1 --2- 172.21.15.31:0/3769693724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f9c1074e0 0x7f9f9c1a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.894+0000 7f9fa3a00640 1 --2- 172.21.15.31:0/3769693724 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f9f800a2d30 0x7f9f800a51f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.894+0000 7f9fa3a00640 1 --2- 172.21.15.31:0/3769693724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f9c104bb0 0x7f9f9c19a200 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.894+0000 7f9fa3a00640 1 --2- 172.21.15.31:0/3769693724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9c102280 0x7f9f9c199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.895+0000 7f9fa3a00640 1 -- 172.21.15.31:0/3769693724 >> 172.21.15.31:0/3769693724 conn(0x7f9f9c0fbc10 msgr2=0x7f9f9c101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:23:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.895+0000 7f9fa3a00640 1 -- 172.21.15.31:0/3769693724 shutdown_connections 2024-04-25T01:23:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:37.895+0000 7f9fa3a00640 1 -- 172.21.15.31:0/3769693724 wait complete. 2024-04-25T01:23:38.439 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:38 smithi169 ceph-mon[32759]: pgmap v621: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 82 KiB/s rd, 106 MiB/s wr, 15.12k op/s 2024-04-25T01:23:38.442 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:38 smithi040 ceph-mon[32518]: pgmap v621: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 82 KiB/s rd, 106 MiB/s wr, 15.12k op/s 2024-04-25T01:23:38.500 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:38 smithi031 ceph-mon[28053]: pgmap v621: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 959 GiB / 1.0 TiB avail; 82 KiB/s rd, 106 MiB/s wr, 15.12k op/s 2024-04-25T01:23:40.439 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:40 smithi169 ceph-mon[32759]: pgmap v622: 97 pgs: 97 active+clean; 13 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 75 KiB/s rd, 110 MiB/s wr, 15.69k op/s 2024-04-25T01:23:40.442 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:40 smithi040 ceph-mon[32518]: pgmap v622: 97 pgs: 97 active+clean; 13 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 75 KiB/s rd, 110 MiB/s wr, 15.69k op/s 2024-04-25T01:23:40.500 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:40 smithi031 ceph-mon[28053]: pgmap v622: 97 pgs: 97 active+clean; 13 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 75 KiB/s rd, 110 MiB/s wr, 15.69k op/s 2024-04-25T01:23:42.500 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:42 smithi031 ceph-mon[28053]: pgmap v623: 97 pgs: 97 active+clean; 13 GiB data, 114 GiB used, 958 GiB / 1.0 TiB avail; 73 KiB/s rd, 99 MiB/s wr, 14.09k op/s 2024-04-25T01:23:42.689 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:42 smithi169 ceph-mon[32759]: pgmap v623: 97 pgs: 97 active+clean; 13 GiB data, 114 GiB used, 958 GiB / 1.0 TiB avail; 73 KiB/s rd, 99 MiB/s wr, 14.09k op/s 2024-04-25T01:23:42.692 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:42 smithi040 ceph-mon[32518]: pgmap v623: 97 pgs: 97 active+clean; 13 GiB data, 114 GiB used, 958 GiB / 1.0 TiB avail; 73 KiB/s rd, 99 MiB/s wr, 14.09k op/s 2024-04-25T01:23:44.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:44 smithi031 ceph-mon[28053]: pgmap v624: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 85 KiB/s rd, 107 MiB/s wr, 15.49k op/s 2024-04-25T01:23:44.689 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:44 smithi169 ceph-mon[32759]: pgmap v624: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 85 KiB/s rd, 107 MiB/s wr, 15.49k op/s 2024-04-25T01:23:44.692 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:44 smithi040 ceph-mon[32518]: pgmap v624: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 85 KiB/s rd, 107 MiB/s wr, 15.49k op/s 2024-04-25T01:23:45.500 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:45 smithi031 ceph-mon[28053]: pgmap v625: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 68 KiB/s rd, 99 MiB/s wr, 14.38k op/s 2024-04-25T01:23:45.689 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:45 smithi169 ceph-mon[32759]: pgmap v625: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 68 KiB/s rd, 99 MiB/s wr, 14.38k op/s 2024-04-25T01:23:45.692 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:45 smithi040 ceph-mon[32518]: pgmap v625: 97 pgs: 97 active+clean; 13 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 68 KiB/s rd, 99 MiB/s wr, 14.38k op/s 2024-04-25T01:23:47.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:47 smithi031 ceph-mon[28053]: pgmap v626: 97 pgs: 97 active+clean; 13 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 80 KiB/s rd, 99 MiB/s wr, 14.48k op/s 2024-04-25T01:23:47.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:47 smithi169 ceph-mon[32759]: pgmap v626: 97 pgs: 97 active+clean; 13 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 80 KiB/s rd, 99 MiB/s wr, 14.48k op/s 2024-04-25T01:23:47.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:47 smithi040 ceph-mon[32518]: pgmap v626: 97 pgs: 97 active+clean; 13 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 80 KiB/s rd, 99 MiB/s wr, 14.48k op/s 2024-04-25T01:23:49.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:49 smithi031 ceph-mon[28053]: pgmap v627: 97 pgs: 97 active+clean; 13 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 46 KiB/s rd, 104 MiB/s wr, 15.66k op/s 2024-04-25T01:23:49.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:49 smithi169 ceph-mon[32759]: pgmap v627: 97 pgs: 97 active+clean; 13 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 46 KiB/s rd, 104 MiB/s wr, 15.66k op/s 2024-04-25T01:23:49.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:49 smithi040 ceph-mon[32518]: pgmap v627: 97 pgs: 97 active+clean; 13 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 46 KiB/s rd, 104 MiB/s wr, 15.66k op/s 2024-04-25T01:23:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.126+0000 7fb1df200640 1 -- 172.21.15.31:0/4201303404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1e01031b0 msgr2=0x7fb1e0109a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.126+0000 7fb1df200640 1 --2- 172.21.15.31:0/4201303404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1e01031b0 0x7fb1e0109a40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8009ca0 tx=0x7fb1d802a8e0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.127+0000 7fb1df200640 1 -- 172.21.15.31:0/4201303404 shutdown_connections 2024-04-25T01:23:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.127+0000 7fb1df200640 1 --2- 172.21.15.31:0/4201303404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1e01031b0 0x7fb1e0109a40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.127+0000 7fb1df200640 1 --2- 172.21.15.31:0/4201303404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1e01027f0 0x7fb1e0102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.127+0000 7fb1df200640 1 --2- 172.21.15.31:0/4201303404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1e0101600 0x7fb1e0101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.127+0000 7fb1df200640 1 -- 172.21.15.31:0/4201303404 >> 172.21.15.31:0/4201303404 conn(0x7fb1e00fcb70 msgr2=0x7fb1e00fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:23:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.127+0000 7fb1df200640 1 -- 172.21.15.31:0/4201303404 shutdown_connections 2024-04-25T01:23:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.127+0000 7fb1df200640 1 -- 172.21.15.31:0/4201303404 wait complete. 2024-04-25T01:23:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.129+0000 7fb1df200640 1 Processor -- start 2024-04-25T01:23:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.130+0000 7fb1df200640 1 -- start start 2024-04-25T01:23:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.130+0000 7fb1df200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1e0101600 0x7fb1e006bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.130+0000 7fb1df200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1e01027f0 0x7fb1e006c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.130+0000 7fb1df200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1e01031b0 0x7fb1e00730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.130+0000 7fb1df200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e010d280 con 0x7fb1e01027f0 2024-04-25T01:23:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.130+0000 7fb1df200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e010d100 con 0x7fb1e01031b0 2024-04-25T01:23:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.130+0000 7fb1df200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e010d400 con 0x7fb1e0101600 2024-04-25T01:23:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.131+0000 7fb1dde00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1e0101600 0x7fb1e006bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.131+0000 7fb1dde00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1e0101600 0x7fb1e006bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:53250/0 (socket says 172.21.15.31:53250) 2024-04-25T01:23:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.131+0000 7fb1dde00640 1 -- 172.21.15.31:0/1473637324 learned_addr learned my addr 172.21.15.31:0/1473637324 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:23:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.131+0000 7fb1dd400640 1 --2- 172.21.15.31:0/1473637324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1e01027f0 0x7fb1e006c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.131+0000 7fb1de800640 1 --2- 172.21.15.31:0/1473637324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1e01031b0 0x7fb1e00730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.132+0000 7fb1dde00640 1 -- 172.21.15.31:0/1473637324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1e01031b0 msgr2=0x7fb1e00730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.132+0000 7fb1dde00640 1 --2- 172.21.15.31:0/1473637324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1e01031b0 0x7fb1e00730c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.132+0000 7fb1dde00640 1 -- 172.21.15.31:0/1473637324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1e01027f0 msgr2=0x7fb1e006c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.132+0000 7fb1dde00640 1 --2- 172.21.15.31:0/1473637324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1e01027f0 0x7fb1e006c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.132+0000 7fb1dde00640 1 -- 172.21.15.31:0/1473637324 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e0073730 con 0x7fb1e0101600 2024-04-25T01:23:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.132+0000 7fb1dde00640 1 --2- 172.21.15.31:0/1473637324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1e0101600 0x7fb1e006bb10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000d950 tx=0x7fb1d000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:23:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.133+0000 7fb1d6a00640 1 -- 172.21.15.31:0/1473637324 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0017070 con 0x7fb1e0101600 2024-04-25T01:23:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.133+0000 7fb1d6a00640 1 -- 172.21.15.31:0/1473637324 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1d0009c30 con 0x7fb1e0101600 2024-04-25T01:23:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.133+0000 7fb1df200640 1 -- 172.21.15.31:0/1473637324 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e01a7fc0 con 0x7fb1e0101600 2024-04-25T01:23:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.133+0000 7fb1d6a00640 1 -- 172.21.15.31:0/1473637324 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d000f620 con 0x7fb1e0101600 2024-04-25T01:23:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.134+0000 7fb1df200640 1 -- 172.21.15.31:0/1473637324 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e01a83c0 con 0x7fb1e0101600 2024-04-25T01:23:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.138+0000 7fb1e8169740 1 Processor -- start 2024-04-25T01:23:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.139+0000 7fb1e8169740 1 -- start start 2024-04-25T01:23:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.139+0000 7fb1e8169740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173f7e50 0x5556173f8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.139+0000 7fb1e8169740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556173f8840 0x5556173fd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.140+0000 7fb1e8169740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556173fd9b0 0x5556173ffe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.140+0000 7fb1e8169740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55561735aab0 con 0x5556173fd9b0 2024-04-25T01:23:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.140+0000 7fb1e8169740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x555617303640 con 0x5556173f8840 2024-04-25T01:23:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.140+0000 7fb1e8169740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555617356a70 con 0x5556173f7e50 2024-04-25T01:23:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.140+0000 7fb1dd400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556173fd9b0 0x5556173ffe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.142+0000 7fb1de800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173f7e50 0x5556173f8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.143+0000 7fb1de800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173f7e50 0x5556173f8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:53252/0 (socket says 172.21.15.31:53252) 2024-04-25T01:23:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.143+0000 7fb1de800640 1 -- 172.21.15.31:0/4096961663 learned_addr learned my addr 172.21.15.31:0/4096961663 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:23:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.143+0000 7fb1dd400640 1 --2- 172.21.15.31:0/4096961663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556173fd9b0 0x5556173ffe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33512/0 (socket says 172.21.15.31:33512) 2024-04-25T01:23:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.143+0000 7fb1d6a00640 1 -- 172.21.15.31:0/1473637324 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb1d001a070 con 0x7fb1e0101600 2024-04-25T01:23:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.143+0000 7fb1dde00640 1 --2- 172.21.15.31:0/4096961663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556173f8840 0x5556173fd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.143+0000 7fb1de800640 1 -- 172.21.15.31:0/4096961663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556173f8840 msgr2=0x5556173fd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.143+0000 7fb1de800640 1 --2- 172.21.15.31:0/4096961663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556173f8840 0x5556173fd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.143+0000 7fb1de800640 1 -- 172.21.15.31:0/4096961663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556173fd9b0 msgr2=0x5556173ffe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.143+0000 7fb1de800640 1 --2- 172.21.15.31:0/4096961663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556173fd9b0 0x5556173ffe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.143+0000 7fb1de800640 1 -- 172.21.15.31:0/4096961663 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55561706f210 con 0x5556173f7e50 2024-04-25T01:23:51.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.144+0000 7fb1de800640 1 --2- 172.21.15.31:0/4096961663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173f7e50 0x5556173f8270 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8035000 tx=0x7fb1d8005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76212dc0a0027c8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:23:51.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.144+0000 7fb1cb400640 1 -- 172.21.15.31:0/4096961663 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8031670 con 0x5556173f7e50 2024-04-25T01:23:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.145+0000 7fb1cb400640 1 -- 172.21.15.31:0/4096961663 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1d8036040 con 0x5556173f7e50 2024-04-25T01:23:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.145+0000 7fb1cb400640 1 -- 172.21.15.31:0/4096961663 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8004530 con 0x5556173f7e50 2024-04-25T01:23:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.145+0000 7fb1dd400640 1 --2- 172.21.15.31:0/4096961663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556173fd9b0 0x5556173ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:23:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.146+0000 7fb1e8169740 1 -- 172.21.15.31:0/4096961663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173f7e50 msgr2=0x5556173f8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.146+0000 7fb1e8169740 1 --2- 172.21.15.31:0/4096961663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173f7e50 0x5556173f8270 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8035000 tx=0x7fb1d8005ef0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.146+0000 7fb1e8169740 1 -- 172.21.15.31:0/4096961663 shutdown_connections 2024-04-25T01:23:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.146+0000 7fb1e8169740 1 --2- 172.21.15.31:0/4096961663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556173fd9b0 0x5556173ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.147+0000 7fb1e8169740 1 --2- 172.21.15.31:0/4096961663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556173f8840 0x5556173fd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.147+0000 7fb1e8169740 1 --2- 172.21.15.31:0/4096961663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173f7e50 0x5556173f8270 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.147+0000 7fb1e8169740 1 -- 172.21.15.31:0/4096961663 >> 172.21.15.31:0/4096961663 conn(0x5556173f34c0 msgr2=0x5556173f38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:23:51.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.147+0000 7fb1e8169740 1 -- 172.21.15.31:0/4096961663 shutdown_connections 2024-04-25T01:23:51.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.148+0000 7fb1d6a00640 1 --2- 172.21.15.31:0/1473637324 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb1c40a2b50 0x7fb1c40a5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.148+0000 7fb1e8169740 1 -- 172.21.15.31:0/4096961663 wait complete. 2024-04-25T01:23:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.148+0000 7fb1dd400640 1 --2- 172.21.15.31:0/1473637324 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb1c40a2b50 0x7fb1c40a5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.149+0000 7fb1e8169740 1 -- start start 2024-04-25T01:23:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.149+0000 7fb1e8169740 1 Processor -- start 2024-04-25T01:23:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.150+0000 7fb1e8169740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173f7e50 0x5556173fae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.150+0000 7fb1d6a00640 1 -- 172.21.15.31:0/1473637324 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fb1d00ce910 con 0x7fb1e0101600 2024-04-25T01:23:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.150+0000 7fb1e8169740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556173f8840 0x5556173fb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.150+0000 7fb1e8169740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556173fd9b0 0x5556173fc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.150+0000 7fb1e8169740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5556174006e0 con 0x5556173fd9b0 2024-04-25T01:23:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.150+0000 7fb1e8169740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x555617316990 con 0x5556173f8840 2024-04-25T01:23:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.150+0000 7fb1e8169740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555617400860 con 0x5556173f7e50 2024-04-25T01:23:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.151+0000 7fb1e8169740 10 client.0 _ll_register_callbacks cb 0x555617318950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:23:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.151+0000 7fb1dd400640 1 --2- 172.21.15.31:0/1473637324 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb1c40a2b50 0x7fb1c40a5010 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x5556173f4390 tx=0x7fb1cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:23:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.151+0000 7fb1de800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556173fd9b0 0x5556173fc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.151+0000 7fb1dd400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556173f8840 0x5556173fb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.150+0000 7fb1dde00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173f7e50 0x5556173fae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.151+0000 7fb1dde00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173f7e50 0x5556173fae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:53258/0 (socket says 172.21.15.31:53258) 2024-04-25T01:23:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.151+0000 7fb1dd400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556173f8840 0x5556173fb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:55928/0 (socket says 172.21.15.31:55928) 2024-04-25T01:23:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.151+0000 7fb1dde00640 1 -- 172.21.15.31:0/2319616486 learned_addr learned my addr 172.21.15.31:0/2319616486 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:23:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.151+0000 7fb1dd400640 1 -- 172.21.15.31:0/2319616486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173f7e50 msgr2=0x5556173fae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.151+0000 7fb1dd400640 1 --2- 172.21.15.31:0/2319616486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173f7e50 0x5556173fae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.151+0000 7fb1dd400640 1 -- 172.21.15.31:0/2319616486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556173fd9b0 msgr2=0x5556173fc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.151+0000 7fb1dd400640 1 --2- 172.21.15.31:0/2319616486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556173fd9b0 0x5556173fc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.152+0000 7fb1dd400640 1 -- 172.21.15.31:0/2319616486 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55561713b620 con 0x5556173f8840 2024-04-25T01:23:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.152+0000 7fb1dd400640 1 --2- 172.21.15.31:0/2319616486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556173f8840 0x5556173fb370 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc003820 tx=0x7fb1cc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3d381d116942ea6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:23:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.153+0000 7fb1dde00640 1 --2- 172.21.15.31:0/2319616486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173f7e50 0x5556173fae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:23:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.153+0000 7fb1bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T01:23:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.153+0000 7fb1bfe00640 1 -- 172.21.15.31:0/2319616486 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc011af0 con 0x5556173f8840 2024-04-25T01:23:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.153+0000 7fb1e8169740 1 -- 172.21.15.31:0/2319616486 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1d8036040 con 0x5556173f8840 2024-04-25T01:23:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.153+0000 7fb1bfe00640 1 -- 172.21.15.31:0/2319616486 <== mon.2 v2:172.21.15.40:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fb1cc011c90 con 0x5556173f8840 2024-04-25T01:23:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.154+0000 7fb1bfe00640 1 -- 172.21.15.31:0/2319616486 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb1cc011e90 con 0x5556173f8840 2024-04-25T01:23:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.154+0000 7fb1e8169740 10 client.25751 fetch_fsmap learned FSMap version 32 2024-04-25T01:23:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.154+0000 7fb1e8169740 1 -- 172.21.15.31:0/2319616486 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555617414e10 con 0x5556173f8840 2024-04-25T01:23:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.155+0000 7fb1bfe00640 1 -- 172.21.15.31:0/2319616486 <== mon.2 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1d8036040 con 0x5556173f8840 2024-04-25T01:23:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.156+0000 7fb1bfe00640 1 -- 172.21.15.31:0/2319616486 <== mon.2 v2:172.21.15.40:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fb1cc028420 con 0x5556173f8840 2024-04-25T01:23:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.156+0000 7fb1e8169740 10 client.25751 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T01:23:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.156+0000 7fb1e8169740 10 client.25751 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:23:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.156+0000 7fb1e8169740 20 client.25751 populate_metadata read hostname 'smithi031' 2024-04-25T01:23:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.156+0000 7fb1e8169740 1 --2- 172.21.15.31:0/2319616486 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5556174164b0 0x555617418950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.156+0000 7fb1e8169740 4 client.25751 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:23:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.156+0000 7fb1e8169740 1 -- 172.21.15.31:0/2319616486 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555617367ff0 con 0x5556174164b0 2024-04-25T01:23:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.157+0000 7fb1dde00640 1 --2- 172.21.15.31:0/2319616486 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5556174164b0 0x555617418950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.158+0000 7fb1bfe00640 10 client.25751 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:23:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.158+0000 7fb1dde00640 1 --2- 172.21.15.31:0/2319616486 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5556174164b0 0x555617418950 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7375c93da60862f4 server_cookie=17950d5e16ed5103 in_seq=0 out_seq=0 2024-04-25T01:23:51.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.160+0000 7fb1bfe00640 1 -- 172.21.15.31:0/2319616486 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555617367ff0 con 0x5556174164b0 2024-04-25T01:23:51.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.160+0000 7fb1bfe00640 10 client.25751 handle_command_reply: tid=0 2024-04-25T01:23:51.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.160+0000 7fb1e8169740 1 client.25751 shutdown 2024-04-25T01:23:51.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.161+0000 7fb1e8169740 1 -- 172.21.15.31:0/2319616486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556173f8840 msgr2=0x5556173fb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:51.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.162+0000 7fb1e8169740 1 --2- 172.21.15.31:0/2319616486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556173f8840 0x5556173fb370 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc003820 tx=0x7fb1cc0046c0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.163+0000 7fb1e8169740 1 -- 172.21.15.31:0/2319616486 shutdown_connections 2024-04-25T01:23:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.163+0000 7fb1e8169740 1 --2- 172.21.15.31:0/2319616486 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x5556174164b0 0x555617418950 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.163+0000 7fb1e8169740 1 --2- 172.21.15.31:0/2319616486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556173fd9b0 0x5556173fc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.163+0000 7fb1e8169740 1 --2- 172.21.15.31:0/2319616486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556173f8840 0x5556173fb370 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.163+0000 7fb1bfe00640 0 client.25751 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:23:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.163+0000 7fb1e8169740 1 --2- 172.21.15.31:0/2319616486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173f7e50 0x5556173fae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.163+0000 7fb1e8169740 1 -- 172.21.15.31:0/2319616486 >> 172.21.15.31:0/2319616486 conn(0x5556173f34c0 msgr2=0x5556173f2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:23:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.163+0000 7fb1e8169740 1 -- 172.21.15.31:0/2319616486 shutdown_connections 2024-04-25T01:23:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.163+0000 7fb1e8169740 1 -- 172.21.15.31:0/2319616486 wait complete. 2024-04-25T01:23:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.164+0000 7fb1e8169740 20 client.25751 trim_cache size 0 max 16384 2024-04-25T01:23:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.213+0000 7fb1e8169740 1 Processor -- start 2024-04-25T01:23:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.216+0000 7fb1e8169740 1 -- start start 2024-04-25T01:23:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.216+0000 7fb1e8169740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556173f7e50 0x5556174188a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:51.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.216+0000 7fb1e8169740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556173f8840 0x5556173f6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.216+0000 7fb1e8169740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173fd9b0 0x5556173f6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.216+0000 7fb1e8169740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e01b1850 con 0x5556173f7e50 2024-04-25T01:23:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.216+0000 7fb1e8169740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e01af120 con 0x5556173f8840 2024-04-25T01:23:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.216+0000 7fb1e8169740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e01acf70 con 0x5556173fd9b0 2024-04-25T01:23:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.217+0000 7fb1de800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173fd9b0 0x5556173f6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.217+0000 7fb1dde00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556173f7e50 0x5556174188a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.217+0000 7fb1de800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173fd9b0 0x5556173f6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:53274/0 (socket says 172.21.15.31:53274) 2024-04-25T01:23:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.217+0000 7fb1de800640 1 -- 172.21.15.31:0/1587653675 learned_addr learned my addr 172.21.15.31:0/1587653675 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:23:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.217+0000 7fb1dd400640 1 --2- 172.21.15.31:0/1587653675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556173f8840 0x5556173f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.218+0000 7fb1dd400640 1 -- 172.21.15.31:0/1587653675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173fd9b0 msgr2=0x5556173f6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.218+0000 7fb1dd400640 1 --2- 172.21.15.31:0/1587653675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173fd9b0 0x5556173f6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.218+0000 7fb1dd400640 1 -- 172.21.15.31:0/1587653675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556173f7e50 msgr2=0x5556174188a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.218+0000 7fb1dd400640 1 --2- 172.21.15.31:0/1587653675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556173f7e50 0x5556174188a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.218+0000 7fb1dd400640 1 -- 172.21.15.31:0/1587653675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556173f32f0 con 0x5556173f8840 2024-04-25T01:23:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.219+0000 7fb1dd400640 1 --2- 172.21.15.31:0/1587653675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556173f8840 0x5556173f6410 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc000c00 tx=0x7fb1cc0118f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4a9dab6640806e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:23:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.219+0000 7fb1caa00640 1 -- 172.21.15.31:0/1587653675 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc011c90 con 0x5556173f8840 2024-04-25T01:23:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.220+0000 7fb1caa00640 1 -- 172.21.15.31:0/1587653675 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1cc023030 con 0x5556173f8840 2024-04-25T01:23:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.220+0000 7fb1caa00640 1 -- 172.21.15.31:0/1587653675 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc011c90 con 0x5556173f8840 2024-04-25T01:23:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.221+0000 7fb1e8169740 1 -- 172.21.15.31:0/1587653675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556173f8840 msgr2=0x5556173f6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.221+0000 7fb1e8169740 1 --2- 172.21.15.31:0/1587653675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556173f8840 0x5556173f6410 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc000c00 tx=0x7fb1cc0118f0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.221+0000 7fb1e8169740 1 -- 172.21.15.31:0/1587653675 shutdown_connections 2024-04-25T01:23:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.221+0000 7fb1e8169740 1 --2- 172.21.15.31:0/1587653675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173fd9b0 0x5556173f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.221+0000 7fb1e8169740 1 --2- 172.21.15.31:0/1587653675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556173f8840 0x5556173f6410 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.221+0000 7fb1e8169740 1 --2- 172.21.15.31:0/1587653675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556173f7e50 0x5556174188a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.221+0000 7fb1e8169740 1 -- 172.21.15.31:0/1587653675 >> 172.21.15.31:0/1587653675 conn(0x5556173f34c0 msgr2=0x5556173f38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:23:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.221+0000 7fb1e8169740 1 -- 172.21.15.31:0/1587653675 shutdown_connections 2024-04-25T01:23:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.222+0000 7fb1e8169740 1 -- 172.21.15.31:0/1587653675 wait complete. 2024-04-25T01:23:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.222+0000 7fb1e8169740 1 -- start start 2024-04-25T01:23:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.222+0000 7fb1e8169740 1 Processor -- start 2024-04-25T01:23:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.223+0000 7fb1e8169740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556173f7e50 0x555617410320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.223+0000 7fb1e8169740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173f8840 0x555617410860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.223+0000 7fb1e8169740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556173fd9b0 0x555617411820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.223+0000 7fb1e8169740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e01ad5e0 con 0x5556173f7e50 2024-04-25T01:23:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.223+0000 7fb1e8169740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e01aea50 con 0x5556173fd9b0 2024-04-25T01:23:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.223+0000 7fb1e8169740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e01adcb0 con 0x5556173f8840 2024-04-25T01:23:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.223+0000 7fb1e8169740 10 client.0 _ll_register_callbacks cb 0x7fb1cc0099d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:23:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.224+0000 7fb1de800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556173fd9b0 0x555617411820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.224+0000 7fb1de800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556173fd9b0 0x555617411820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:55954/0 (socket says 172.21.15.31:55954) 2024-04-25T01:23:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.224+0000 7fb1de800640 1 -- 172.21.15.31:0/116026534 learned_addr learned my addr 172.21.15.31:0/116026534 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:23:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.224+0000 7fb1dd400640 1 --2- 172.21.15.31:0/116026534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173f8840 0x555617410860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.224+0000 7fb1dde00640 1 --2- 172.21.15.31:0/116026534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556173f7e50 0x555617410320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.225+0000 7fb1dd400640 1 -- 172.21.15.31:0/116026534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556173fd9b0 msgr2=0x555617411820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.225+0000 7fb1dd400640 1 --2- 172.21.15.31:0/116026534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556173fd9b0 0x555617411820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.225+0000 7fb1dd400640 1 -- 172.21.15.31:0/116026534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556173f7e50 msgr2=0x555617410320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:23:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.225+0000 7fb1dd400640 1 --2- 172.21.15.31:0/116026534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556173f7e50 0x555617410320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.225+0000 7fb1dd400640 1 -- 172.21.15.31:0/116026534 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555617411e10 con 0x5556173f8840 2024-04-25T01:23:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.226+0000 7fb1dde00640 1 --2- 172.21.15.31:0/116026534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556173f7e50 0x555617410320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:23:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.226+0000 7fb1dd400640 1 --2- 172.21.15.31:0/116026534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173f8840 0x555617410860 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc028890 tx=0x7fb1cc0114f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19ec18c84eee1fb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:23:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.227+0000 7fb1bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:23:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.227+0000 7fb1bfe00640 1 -- 172.21.15.31:0/116026534 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc028620 con 0x5556173f8840 2024-04-25T01:23:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.228+0000 7fb1bfe00640 1 -- 172.21.15.31:0/116026534 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fb1cc03a450 con 0x5556173f8840 2024-04-25T01:23:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.228+0000 7fb1e8169740 1 -- 172.21.15.31:0/116026534 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1cc023030 con 0x5556173f8840 2024-04-25T01:23:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.229+0000 7fb1bfe00640 1 -- 172.21.15.31:0/116026534 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb1cc049050 con 0x5556173f8840 2024-04-25T01:23:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.229+0000 7fb1e8169740 10 client.25801 fetch_fsmap learned FSMap version 32 2024-04-25T01:23:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.229+0000 7fb1e8169740 1 -- 172.21.15.31:0/116026534 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555617413120 con 0x5556173f8840 2024-04-25T01:23:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.230+0000 7fb1bfe00640 1 -- 172.21.15.31:0/116026534 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1cc023030 con 0x5556173f8840 2024-04-25T01:23:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.231+0000 7fb1bfe00640 1 -- 172.21.15.31:0/116026534 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fb1cc01f910 con 0x5556173f8840 2024-04-25T01:23:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.233+0000 7fb1e8169740 10 client.25801 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T01:23:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.233+0000 7fb1e8169740 10 client.25801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:23:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.233+0000 7fb1e8169740 20 client.25801 populate_metadata read hostname 'smithi031' 2024-04-25T01:23:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.233+0000 7fb1e8169740 1 --2- 172.21.15.31:0/116026534 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55561747d090 0x55561747f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:23:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.233+0000 7fb1e8169740 4 client.25801 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:23:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.233+0000 7fb1e8169740 1 -- 172.21.15.31:0/116026534 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555617362340 con 0x55561747d090 2024-04-25T01:23:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.234+0000 7fb1dde00640 1 --2- 172.21.15.31:0/116026534 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55561747d090 0x55561747f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:23:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.235+0000 7fb1dde00640 1 --2- 172.21.15.31:0/116026534 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55561747d090 0x55561747f4b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=522dace220acfa38 server_cookie=72b2f40f3aa038e1 in_seq=0 out_seq=0 2024-04-25T01:23:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:51.235+0000 7fb1bfe00640 10 client.25801 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:23:51.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:51 smithi031 ceph-mon[28053]: pgmap v628: 97 pgs: 97 active+clean; 13 GiB data, 115 GiB used, 958 GiB / 1.0 TiB avail; 32 KiB/s rd, 96 MiB/s wr, 14.88k op/s 2024-04-25T01:23:51.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:51 smithi169 ceph-mon[32759]: pgmap v628: 97 pgs: 97 active+clean; 13 GiB data, 115 GiB used, 958 GiB / 1.0 TiB avail; 32 KiB/s rd, 96 MiB/s wr, 14.88k op/s 2024-04-25T01:23:51.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:51 smithi040 ceph-mon[32518]: pgmap v628: 97 pgs: 97 active+clean; 13 GiB data, 115 GiB used, 958 GiB / 1.0 TiB avail; 32 KiB/s rd, 96 MiB/s wr, 14.88k op/s 2024-04-25T01:23:53.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:53 smithi031 ceph-mon[28053]: pgmap v629: 97 pgs: 97 active+clean; 13 GiB data, 116 GiB used, 957 GiB / 1.0 TiB avail; 33 KiB/s rd, 111 MiB/s wr, 17.33k op/s 2024-04-25T01:23:53.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:53 smithi169 ceph-mon[32759]: pgmap v629: 97 pgs: 97 active+clean; 13 GiB data, 116 GiB used, 957 GiB / 1.0 TiB avail; 33 KiB/s rd, 111 MiB/s wr, 17.33k op/s 2024-04-25T01:23:53.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:53 smithi040 ceph-mon[32518]: pgmap v629: 97 pgs: 97 active+clean; 13 GiB data, 116 GiB used, 957 GiB / 1.0 TiB avail; 33 KiB/s rd, 111 MiB/s wr, 17.33k op/s 2024-04-25T01:23:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.916+0000 7fb1bfe00640 1 -- 172.21.15.31:0/116026534 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555617362340 con 0x55561747d090 2024-04-25T01:23:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.916+0000 7fb1bfe00640 10 client.25801 handle_command_reply: tid=0 2024-04-25T01:23:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.917+0000 7fb1e8169740 1 client.25801 shutdown 2024-04-25T01:23:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.917+0000 7fb1e8169740 1 -- 172.21.15.31:0/116026534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173f8840 msgr2=0x555617410860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.917+0000 7fb1e8169740 1 --2- 172.21.15.31:0/116026534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173f8840 0x555617410860 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc028890 tx=0x7fb1cc0114f0 comp rx=0 tx=0).stop 2024-04-25T01:23:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.918+0000 7fb1e8169740 1 -- 172.21.15.31:0/116026534 shutdown_connections 2024-04-25T01:23:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.918+0000 7fb1e8169740 1 --2- 172.21.15.31:0/116026534 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55561747d090 0x55561747f4b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.918+0000 7fb1e8169740 1 --2- 172.21.15.31:0/116026534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5556173fd9b0 0x555617411820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.918+0000 7fb1e8169740 1 --2- 172.21.15.31:0/116026534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556173f8840 0x555617410860 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.918+0000 7fb1e8169740 1 --2- 172.21.15.31:0/116026534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556173f7e50 0x555617410320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.918+0000 7fb1e8169740 1 -- 172.21.15.31:0/116026534 >> 172.21.15.31:0/116026534 conn(0x5556173f34c0 msgr2=0x5556173f6410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:23:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.918+0000 7fb1bfe00640 0 client.25801 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:23:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.918+0000 7fb1e8169740 1 -- 172.21.15.31:0/116026534 shutdown_connections 2024-04-25T01:23:54.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.918+0000 7fb1e8169740 1 -- 172.21.15.31:0/116026534 wait complete. 2024-04-25T01:23:54.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.919+0000 7fb1e8169740 20 client.25801 trim_cache size 0 max 16384 2024-04-25T01:23:54.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:23:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:23:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:23:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:23:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.920+0000 7fb1df200640 1 -- 172.21.15.31:0/1473637324 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb1c40a2b50 msgr2=0x7fb1c40a5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.920+0000 7fb1df200640 1 --2- 172.21.15.31:0/1473637324 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb1c40a2b50 0x7fb1c40a5010 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x5556173f4390 tx=0x7fb1cc005f90 comp rx=0 tx=0).stop 2024-04-25T01:23:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.920+0000 7fb1df200640 1 -- 172.21.15.31:0/1473637324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1e0101600 msgr2=0x7fb1e006bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:23:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.920+0000 7fb1df200640 1 --2- 172.21.15.31:0/1473637324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1e0101600 0x7fb1e006bb10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000d950 tx=0x7fb1d000de10 comp rx=0 tx=0).stop 2024-04-25T01:23:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.920+0000 7fb1df200640 1 -- 172.21.15.31:0/1473637324 shutdown_connections 2024-04-25T01:23:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.920+0000 7fb1df200640 1 --2- 172.21.15.31:0/1473637324 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fb1c40a2b50 0x7fb1c40a5010 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.920+0000 7fb1df200640 1 --2- 172.21.15.31:0/1473637324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1e01031b0 0x7fb1e00730c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.920+0000 7fb1df200640 1 --2- 172.21.15.31:0/1473637324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1e01027f0 0x7fb1e006c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.920+0000 7fb1df200640 1 --2- 172.21.15.31:0/1473637324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1e0101600 0x7fb1e006bb10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:23:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.920+0000 7fb1df200640 1 -- 172.21.15.31:0/1473637324 >> 172.21.15.31:0/1473637324 conn(0x7fb1e00fcb70 msgr2=0x7fb1e0105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:23:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.921+0000 7fb1df200640 1 -- 172.21.15.31:0/1473637324 shutdown_connections 2024-04-25T01:23:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:23:54.921+0000 7fb1df200640 1 -- 172.21.15.31:0/1473637324 wait complete. 2024-04-25T01:23:55.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:55 smithi031 ceph-mon[28053]: pgmap v630: 97 pgs: 97 active+clean; 13 GiB data, 117 GiB used, 956 GiB / 1.0 TiB avail; 22 KiB/s rd, 115 MiB/s wr, 18.27k op/s 2024-04-25T01:23:55.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:55 smithi169 ceph-mon[32759]: pgmap v630: 97 pgs: 97 active+clean; 13 GiB data, 117 GiB used, 956 GiB / 1.0 TiB avail; 22 KiB/s rd, 115 MiB/s wr, 18.27k op/s 2024-04-25T01:23:55.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:55 smithi040 ceph-mon[32518]: pgmap v630: 97 pgs: 97 active+clean; 13 GiB data, 117 GiB used, 956 GiB / 1.0 TiB avail; 22 KiB/s rd, 115 MiB/s wr, 18.27k op/s 2024-04-25T01:23:57.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:57 smithi169 ceph-mon[32759]: pgmap v631: 97 pgs: 97 active+clean; 13 GiB data, 117 GiB used, 956 GiB / 1.0 TiB avail; 21 KiB/s rd, 105 MiB/s wr, 16.77k op/s 2024-04-25T01:23:57.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:57 smithi040 ceph-mon[32518]: pgmap v631: 97 pgs: 97 active+clean; 13 GiB data, 117 GiB used, 956 GiB / 1.0 TiB avail; 21 KiB/s rd, 105 MiB/s wr, 16.77k op/s 2024-04-25T01:23:57.999 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:57 smithi031 ceph-mon[28053]: pgmap v631: 97 pgs: 97 active+clean; 13 GiB data, 117 GiB used, 956 GiB / 1.0 TiB avail; 21 KiB/s rd, 105 MiB/s wr, 16.77k op/s 2024-04-25T01:23:59.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:23:59 smithi169 ceph-mon[32759]: pgmap v632: 97 pgs: 97 active+clean; 13 GiB data, 118 GiB used, 955 GiB / 1.0 TiB avail; 9.4 KiB/s rd, 128 MiB/s wr, 21.06k op/s 2024-04-25T01:23:59.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:23:59 smithi040 ceph-mon[32518]: pgmap v632: 97 pgs: 97 active+clean; 13 GiB data, 118 GiB used, 955 GiB / 1.0 TiB avail; 9.4 KiB/s rd, 128 MiB/s wr, 21.06k op/s 2024-04-25T01:24:00.000 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:23:59 smithi031 ceph-mon[28053]: pgmap v632: 97 pgs: 97 active+clean; 13 GiB data, 118 GiB used, 955 GiB / 1.0 TiB avail; 9.4 KiB/s rd, 128 MiB/s wr, 21.06k op/s 2024-04-25T01:24:01.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:01 smithi169 ceph-mon[32759]: pgmap v633: 97 pgs: 97 active+clean; 13 GiB data, 118 GiB used, 955 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 106 MiB/s wr, 17.83k op/s 2024-04-25T01:24:01.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:01 smithi040 ceph-mon[32518]: pgmap v633: 97 pgs: 97 active+clean; 13 GiB data, 118 GiB used, 955 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 106 MiB/s wr, 17.83k op/s 2024-04-25T01:24:02.000 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:01 smithi031 ceph-mon[28053]: pgmap v633: 97 pgs: 97 active+clean; 13 GiB data, 118 GiB used, 955 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 106 MiB/s wr, 17.83k op/s 2024-04-25T01:24:03.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:03 smithi169 ceph-mon[32759]: pgmap v634: 97 pgs: 97 active+clean; 13 GiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 8.4 KiB/s rd, 109 MiB/s wr, 18.69k op/s 2024-04-25T01:24:03.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:03 smithi040 ceph-mon[32518]: pgmap v634: 97 pgs: 97 active+clean; 13 GiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 8.4 KiB/s rd, 109 MiB/s wr, 18.69k op/s 2024-04-25T01:24:04.000 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:03 smithi031 ceph-mon[28053]: pgmap v634: 97 pgs: 97 active+clean; 13 GiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 8.4 KiB/s rd, 109 MiB/s wr, 18.69k op/s 2024-04-25T01:24:05.802 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:05 smithi031 ceph-mon[28053]: pgmap v635: 97 pgs: 97 active+clean; 13 GiB data, 120 GiB used, 953 GiB / 1.0 TiB avail; 7.0 KiB/s rd, 111 MiB/s wr, 19.62k op/s 2024-04-25T01:24:05.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:05 smithi169 ceph-mon[32759]: pgmap v635: 97 pgs: 97 active+clean; 13 GiB data, 120 GiB used, 953 GiB / 1.0 TiB avail; 7.0 KiB/s rd, 111 MiB/s wr, 19.62k op/s 2024-04-25T01:24:05.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:05 smithi040 ceph-mon[32518]: pgmap v635: 97 pgs: 97 active+clean; 13 GiB data, 120 GiB used, 953 GiB / 1.0 TiB avail; 7.0 KiB/s rd, 111 MiB/s wr, 19.62k op/s 2024-04-25T01:24:07.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:07 smithi169 ceph-mon[32759]: pgmap v636: 97 pgs: 97 active+clean; 13 GiB data, 120 GiB used, 953 GiB / 1.0 TiB avail; 8.5 KiB/s rd, 88 MiB/s wr, 15.91k op/s 2024-04-25T01:24:07.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:07 smithi040 ceph-mon[32518]: pgmap v636: 97 pgs: 97 active+clean; 13 GiB data, 120 GiB used, 953 GiB / 1.0 TiB avail; 8.5 KiB/s rd, 88 MiB/s wr, 15.91k op/s 2024-04-25T01:24:08.000 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:07 smithi031 ceph-mon[28053]: pgmap v636: 97 pgs: 97 active+clean; 13 GiB data, 120 GiB used, 953 GiB / 1.0 TiB avail; 8.5 KiB/s rd, 88 MiB/s wr, 15.91k op/s 2024-04-25T01:24:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.137+0000 7f61a3600640 1 -- 172.21.15.31:0/3095928322 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619c077ce0 msgr2=0x7f619c078580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.137+0000 7f61a3600640 1 --2- 172.21.15.31:0/3095928322 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619c077ce0 0x7f619c078580 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6194009ca0 tx=0x7f619402a8c0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.137+0000 7f61a3600640 1 -- 172.21.15.31:0/3095928322 shutdown_connections 2024-04-25T01:24:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.137+0000 7f61a3600640 1 --2- 172.21.15.31:0/3095928322 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619c077ce0 0x7f619c078580 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.137+0000 7f61a3600640 1 --2- 172.21.15.31:0/3095928322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f619c077340 0x7f619c0777a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.137+0000 7f61a3600640 1 --2- 172.21.15.31:0/3095928322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f619c076100 0x7f619c076520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.137+0000 7f61a3600640 1 -- 172.21.15.31:0/3095928322 >> 172.21.15.31:0/3095928322 conn(0x7f619c0fef60 msgr2=0x7f619c101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:24:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.137+0000 7f61a3600640 1 -- 172.21.15.31:0/3095928322 shutdown_connections 2024-04-25T01:24:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.137+0000 7f61a3600640 1 -- 172.21.15.31:0/3095928322 wait complete. 2024-04-25T01:24:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.139+0000 7f61a3600640 1 Processor -- start 2024-04-25T01:24:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.139+0000 7f61a3600640 1 -- start start 2024-04-25T01:24:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.140+0000 7f61a3600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f619c076100 0x7f619c10e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.140+0000 7f61a3600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619c077340 0x7f619c10e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.140+0000 7f61a3600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f619c077ce0 0x7f619c1ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.140+0000 7f61a3600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f619c113aa0 con 0x7f619c077ce0 2024-04-25T01:24:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.140+0000 7f61a3600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f619c113920 con 0x7f619c077340 2024-04-25T01:24:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.140+0000 7f61a3600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f619c113c20 con 0x7f619c076100 2024-04-25T01:24:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.141+0000 7f61a0c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f619c076100 0x7f619c10e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.141+0000 7f61a0c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f619c076100 0x7f619c10e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:32796/0 (socket says 172.21.15.31:32796) 2024-04-25T01:24:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.141+0000 7f61a0c00640 1 -- 172.21.15.31:0/1988312234 learned_addr learned my addr 172.21.15.31:0/1988312234 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:24:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.142+0000 7f61a0c00640 1 -- 172.21.15.31:0/1988312234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619c077340 msgr2=0x7f619c10e7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T01:24:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.142+0000 7f61a0c00640 1 --2- 172.21.15.31:0/1988312234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619c077340 0x7f619c10e7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.142+0000 7f61a0c00640 1 -- 172.21.15.31:0/1988312234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f619c077ce0 msgr2=0x7f619c1ae170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:24:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.142+0000 7f61a0c00640 1 --2- 172.21.15.31:0/1988312234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f619c077ce0 0x7f619c1ae170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.142+0000 7f61a0c00640 1 -- 172.21.15.31:0/1988312234 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619c111910 con 0x7f619c076100 2024-04-25T01:24:08.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.143+0000 7f61a0c00640 1 --2- 172.21.15.31:0/1988312234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f619c076100 0x7f619c10e2a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f619000e9f0 tx=0x7f619000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:24:08.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.143+0000 7f6199600640 1 -- 172.21.15.31:0/1988312234 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619000cd80 con 0x7f619c076100 2024-04-25T01:24:08.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.144+0000 7f6199600640 1 -- 172.21.15.31:0/1988312234 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6190010910 con 0x7f619c076100 2024-04-25T01:24:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.145+0000 7f6199600640 1 -- 172.21.15.31:0/1988312234 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6190018990 con 0x7f619c076100 2024-04-25T01:24:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.145+0000 7f61a3600640 1 -- 172.21.15.31:0/1988312234 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f619c1ae6b0 con 0x7f619c076100 2024-04-25T01:24:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.145+0000 7f61a3600640 1 -- 172.21.15.31:0/1988312234 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f619c1aeb80 con 0x7f619c076100 2024-04-25T01:24:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.155+0000 7f6199600640 1 -- 172.21.15.31:0/1988312234 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6190018b30 con 0x7f619c076100 2024-04-25T01:24:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.157+0000 7f61a6441740 1 Processor -- start 2024-04-25T01:24:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.158+0000 7f61a6441740 1 -- start start 2024-04-25T01:24:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.158+0000 7f61a6441740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db84cc9ca0 0x55db84cca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.158+0000 7f61a6441740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db84cca690 0x55db84ccf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.158+0000 7f61a6441740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db84ccf800 0x55db84cd1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.158+0000 7f61a6441740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55db84c2cab0 con 0x55db84cca690 2024-04-25T01:24:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.158+0000 7f61a6441740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55db84bd5640 con 0x55db84cc9ca0 2024-04-25T01:24:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.158+0000 7f61a6441740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55db84c28a70 con 0x55db84ccf800 2024-04-25T01:24:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.159+0000 7f61a1600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db84cc9ca0 0x55db84cca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.159+0000 7f61a1600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db84cc9ca0 0x55db84cca0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:42884/0 (socket says 172.21.15.31:42884) 2024-04-25T01:24:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.159+0000 7f61a1600640 1 -- 172.21.15.31:0/1217650286 learned_addr learned my addr 172.21.15.31:0/1217650286 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:24:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.159+0000 7f619be00640 1 --2- 172.21.15.31:0/1217650286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db84ccf800 0x55db84cd1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.159+0000 7f61a0c00640 1 --2- 172.21.15.31:0/1217650286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db84cca690 0x55db84ccf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.161+0000 7f61a1600640 1 -- 172.21.15.31:0/1217650286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db84ccf800 msgr2=0x55db84cd1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:24:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.161+0000 7f61a1600640 1 --2- 172.21.15.31:0/1217650286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db84ccf800 0x55db84cd1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.161+0000 7f61a1600640 1 -- 172.21.15.31:0/1217650286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db84cca690 msgr2=0x55db84ccf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:24:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.161+0000 7f61a1600640 1 --2- 172.21.15.31:0/1217650286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db84cca690 0x55db84ccf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.161+0000 7f61a1600640 1 -- 172.21.15.31:0/1217650286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db84941210 con 0x55db84cc9ca0 2024-04-25T01:24:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.161+0000 7f619be00640 1 --2- 172.21.15.31:0/1217650286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db84ccf800 0x55db84cd1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:24:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.162+0000 7f61a1600640 1 --2- 172.21.15.31:0/1217650286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db84cc9ca0 0x55db84cca0c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f619402ade0 tx=0x7f6194005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6b8c4df3e29bdaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:24:08.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.163+0000 7f6199600640 1 --2- 172.21.15.31:0/1988312234 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f61840a2c30 0x7f61840a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.164+0000 7f6199600640 1 -- 172.21.15.31:0/1988312234 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f61900cf420 con 0x7f619c076100 2024-04-25T01:24:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.165+0000 7f618dc00640 1 -- 172.21.15.31:0/1217650286 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6194031450 con 0x55db84cc9ca0 2024-04-25T01:24:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.165+0000 7f618dc00640 1 -- 172.21.15.31:0/1217650286 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f619403f070 con 0x55db84cc9ca0 2024-04-25T01:24:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.165+0000 7f618dc00640 1 -- 172.21.15.31:0/1217650286 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61940046a0 con 0x55db84cc9ca0 2024-04-25T01:24:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.167+0000 7f61a0c00640 1 --2- 172.21.15.31:0/1988312234 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f61840a2c30 0x7f61840a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.167+0000 7f61a6441740 1 -- 172.21.15.31:0/1217650286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db84cc9ca0 msgr2=0x55db84cca0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.167+0000 7f61a6441740 1 --2- 172.21.15.31:0/1217650286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db84cc9ca0 0x55db84cca0c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f619402ade0 tx=0x7f6194005ef0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.168+0000 7f61a6441740 1 -- 172.21.15.31:0/1217650286 shutdown_connections 2024-04-25T01:24:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.168+0000 7f61a6441740 1 --2- 172.21.15.31:0/1217650286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db84ccf800 0x55db84cd1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.168+0000 7f61a6441740 1 --2- 172.21.15.31:0/1217650286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db84cca690 0x55db84ccf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.168+0000 7f61a6441740 1 --2- 172.21.15.31:0/1217650286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db84cc9ca0 0x55db84cca0c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.169+0000 7f61a6441740 1 -- 172.21.15.31:0/1217650286 >> 172.21.15.31:0/1217650286 conn(0x55db84cc5310 msgr2=0x55db84cc5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:24:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.169+0000 7f61a6441740 1 -- 172.21.15.31:0/1217650286 shutdown_connections 2024-04-25T01:24:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.169+0000 7f61a6441740 1 -- 172.21.15.31:0/1217650286 wait complete. 2024-04-25T01:24:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.170+0000 7f61a6441740 1 -- start start 2024-04-25T01:24:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.171+0000 7f61a0c00640 1 --2- 172.21.15.31:0/1988312234 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f61840a2c30 0x7f61840a50f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55db84ccaa90 tx=0x7f6190000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:24:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.171+0000 7f61a6441740 1 Processor -- start 2024-04-25T01:24:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.172+0000 7f61a6441740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db84cc9ca0 0x55db84cd3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.172+0000 7f61a6441740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db84cca690 0x55db84cd4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.173+0000 7f61a1600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db84cc9ca0 0x55db84cd3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.173+0000 7f61a1600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db84cc9ca0 0x55db84cd3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:32814/0 (socket says 172.21.15.31:32814) 2024-04-25T01:24:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.173+0000 7f619be00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db84cca690 0x55db84cd4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.173+0000 7f61a1600640 1 -- 172.21.15.31:0/2448716397 learned_addr learned my addr 172.21.15.31:0/2448716397 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:24:08.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.174+0000 7f61a6441740 1 --2- 172.21.15.31:0/2448716397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db84ccf800 0x55db84cdb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:08.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.174+0000 7f61a6441740 1 -- 172.21.15.31:0/2448716397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55db84cd2750 con 0x55db84cca690 2024-04-25T01:24:08.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.174+0000 7f61a6441740 1 -- 172.21.15.31:0/2448716397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55db84be8990 con 0x55db84ccf800 2024-04-25T01:24:08.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.174+0000 7f61a6441740 1 -- 172.21.15.31:0/2448716397 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55db84cd28d0 con 0x55db84cc9ca0 2024-04-25T01:24:08.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.174+0000 7f61a6441740 10 client.0 _ll_register_callbacks cb 0x55db84bea950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:24:08.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.175+0000 7f61a1600640 1 -- 172.21.15.31:0/2448716397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db84ccf800 msgr2=0x55db84cdb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:24:08.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.175+0000 7f61a1600640 1 --2- 172.21.15.31:0/2448716397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db84ccf800 0x55db84cdb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.175+0000 7f61a1600640 1 -- 172.21.15.31:0/2448716397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db84cca690 msgr2=0x55db84cd4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:24:08.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.175+0000 7f61a1600640 1 --2- 172.21.15.31:0/2448716397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db84cca690 0x55db84cd4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.175+0000 7f61a1600640 1 -- 172.21.15.31:0/2448716397 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db84a0d620 con 0x55db84cc9ca0 2024-04-25T01:24:08.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.176+0000 7f619be00640 1 --2- 172.21.15.31:0/2448716397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db84cca690 0x55db84cd4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:24:08.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.176+0000 7f61a1600640 1 --2- 172.21.15.31:0/2448716397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db84cc9ca0 0x55db84cd3ee0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f619402aeb0 tx=0x7f619402f3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dd67806ff6bffef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:24:08.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.176+0000 7f6182000640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-25T01:24:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.177+0000 7f6182000640 1 -- 172.21.15.31:0/2448716397 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6194050020 con 0x55db84cc9ca0 2024-04-25T01:24:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.177+0000 7f6182000640 1 -- 172.21.15.31:0/2448716397 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f6194003970 con 0x55db84cc9ca0 2024-04-25T01:24:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.178+0000 7f61a6441740 1 -- 172.21.15.31:0/2448716397 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f619403f070 con 0x55db84cc9ca0 2024-04-25T01:24:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.178+0000 7f6182000640 1 -- 172.21.15.31:0/2448716397 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6194003b70 con 0x55db84cc9ca0 2024-04-25T01:24:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.178+0000 7f61a6441740 10 client.25813 fetch_fsmap learned FSMap version 32 2024-04-25T01:24:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.179+0000 7f61a6441740 1 -- 172.21.15.31:0/2448716397 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55db84cd2750 con 0x55db84cc9ca0 2024-04-25T01:24:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.179+0000 7f6182000640 1 -- 172.21.15.31:0/2448716397 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f619403f070 con 0x55db84cc9ca0 2024-04-25T01:24:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.180+0000 7f6182000640 1 -- 172.21.15.31:0/2448716397 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f61940305c0 con 0x55db84cc9ca0 2024-04-25T01:24:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.181+0000 7f61a6441740 10 client.25813 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T01:24:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.181+0000 7f61a6441740 10 client.25813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:24:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.181+0000 7f61a6441740 20 client.25813 populate_metadata read hostname 'smithi031' 2024-04-25T01:24:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.181+0000 7f61a6441740 1 --2- 172.21.15.31:0/2448716397 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55db84cdf660 0x55db84ce1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.181+0000 7f61a6441740 4 client.25813 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:24:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.181+0000 7f61a6441740 1 -- 172.21.15.31:0/2448716397 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db84c39ff0 con 0x55db84cdf660 2024-04-25T01:24:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.181+0000 7f619be00640 1 --2- 172.21.15.31:0/2448716397 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55db84cdf660 0x55db84ce1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.182+0000 7f619be00640 1 --2- 172.21.15.31:0/2448716397 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55db84cdf660 0x55db84ce1a50 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c293d1b3630c8ed1 server_cookie=67e66dbc72c4e0cf in_seq=0 out_seq=0 2024-04-25T01:24:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.182+0000 7f6182000640 10 client.25813 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:24:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.183+0000 7f6182000640 1 -- 172.21.15.31:0/2448716397 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55db84c39ff0 con 0x55db84cdf660 2024-04-25T01:24:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.183+0000 7f6182000640 10 client.25813 handle_command_reply: tid=0 2024-04-25T01:24:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.183+0000 7f61a6441740 1 client.25813 shutdown 2024-04-25T01:24:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.184+0000 7f61a6441740 1 -- 172.21.15.31:0/2448716397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db84cc9ca0 msgr2=0x55db84cd3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.184+0000 7f61a6441740 1 --2- 172.21.15.31:0/2448716397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db84cc9ca0 0x55db84cd3ee0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f619402aeb0 tx=0x7f619402f3f0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.184+0000 7f61a6441740 1 -- 172.21.15.31:0/2448716397 shutdown_connections 2024-04-25T01:24:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.185+0000 7f61a6441740 1 --2- 172.21.15.31:0/2448716397 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55db84cdf660 0x55db84ce1a50 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.185+0000 7f6182000640 0 client.25813 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:24:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.185+0000 7f61a6441740 1 --2- 172.21.15.31:0/2448716397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db84ccf800 0x55db84cdb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.185+0000 7f61a6441740 1 --2- 172.21.15.31:0/2448716397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db84cca690 0x55db84cd4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.185+0000 7f61a6441740 1 --2- 172.21.15.31:0/2448716397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db84cc9ca0 0x55db84cd3ee0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.185+0000 7f61a6441740 1 -- 172.21.15.31:0/2448716397 >> 172.21.15.31:0/2448716397 conn(0x55db84cc5310 msgr2=0x55db84cc4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:24:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.185+0000 7f61a6441740 1 -- 172.21.15.31:0/2448716397 shutdown_connections 2024-04-25T01:24:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.185+0000 7f61a6441740 1 -- 172.21.15.31:0/2448716397 wait complete. 2024-04-25T01:24:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.186+0000 7f61a6441740 20 client.25813 trim_cache size 0 max 16384 2024-04-25T01:24:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.218+0000 7f61a6441740 1 Processor -- start 2024-04-25T01:24:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.218+0000 7f61a6441740 1 -- start start 2024-04-25T01:24:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.219+0000 7f61a6441740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db84cc5310 0x55db84cc8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.219+0000 7f61a6441740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db84cc9ca0 0x55db84cdb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.219+0000 7f61a6441740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db84cca690 0x55db84cc4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.219+0000 7f61a6441740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f619c1ba060 con 0x55db84cca690 2024-04-25T01:24:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.219+0000 7f61a6441740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f619c1b7930 con 0x55db84cc9ca0 2024-04-25T01:24:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.219+0000 7f61a6441740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f619c1b5830 con 0x55db84cc5310 2024-04-25T01:24:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.220+0000 7f61a1600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db84cca690 0x55db84cc4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.220+0000 7f61a1600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db84cca690 0x55db84cc4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43936/0 (socket says 172.21.15.31:43936) 2024-04-25T01:24:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.220+0000 7f619be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db84cc9ca0 0x55db84cdb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.220+0000 7f619be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db84cc9ca0 0x55db84cdb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:42898/0 (socket says 172.21.15.31:42898) 2024-04-25T01:24:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.220+0000 7f619be00640 1 -- 172.21.15.31:0/3309210861 learned_addr learned my addr 172.21.15.31:0/3309210861 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:24:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.220+0000 7f61a1600640 1 --2- 172.21.15.31:0/3309210861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db84cc5310 0x55db84cc8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.221+0000 7f619be00640 1 -- 172.21.15.31:0/3309210861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db84cc5310 msgr2=0x55db84cc8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:24:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.221+0000 7f619be00640 1 --2- 172.21.15.31:0/3309210861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db84cc5310 0x55db84cc8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.221+0000 7f619be00640 1 -- 172.21.15.31:0/3309210861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db84cca690 msgr2=0x55db84cc4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:24:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.221+0000 7f619be00640 1 --2- 172.21.15.31:0/3309210861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db84cca690 0x55db84cc4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.221+0000 7f619be00640 1 -- 172.21.15.31:0/3309210861 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db84cc98e0 con 0x55db84cc9ca0 2024-04-25T01:24:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.222+0000 7f619be00640 1 --2- 172.21.15.31:0/3309210861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db84cc9ca0 0x55db84cdb3f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55db84cc61e0 tx=0x7f6188010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98fd491d55a3a380 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:24:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.222+0000 7f618d200640 1 -- 172.21.15.31:0/3309210861 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188004230 con 0x55db84cc9ca0 2024-04-25T01:24:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.223+0000 7f618d200640 1 -- 172.21.15.31:0/3309210861 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f618801d900 con 0x55db84cc9ca0 2024-04-25T01:24:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.223+0000 7f618d200640 1 -- 172.21.15.31:0/3309210861 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188004230 con 0x55db84cc9ca0 2024-04-25T01:24:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.224+0000 7f61a6441740 1 -- 172.21.15.31:0/3309210861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db84cc9ca0 msgr2=0x55db84cdb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.224+0000 7f61a6441740 1 --2- 172.21.15.31:0/3309210861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db84cc9ca0 0x55db84cdb3f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55db84cc61e0 tx=0x7f6188010800 comp rx=0 tx=0).stop 2024-04-25T01:24:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.224+0000 7f61a6441740 1 -- 172.21.15.31:0/3309210861 shutdown_connections 2024-04-25T01:24:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.224+0000 7f61a6441740 1 --2- 172.21.15.31:0/3309210861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db84cca690 0x55db84cc4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.225+0000 7f61a6441740 1 --2- 172.21.15.31:0/3309210861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db84cc9ca0 0x55db84cdb3f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.225+0000 7f61a6441740 1 --2- 172.21.15.31:0/3309210861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db84cc5310 0x55db84cc8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.225+0000 7f61a6441740 1 -- 172.21.15.31:0/3309210861 >> 172.21.15.31:0/3309210861 conn(0x55db84ccf2d0 msgr2=0x55db84cc56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:24:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.225+0000 7f61a6441740 1 -- 172.21.15.31:0/3309210861 shutdown_connections 2024-04-25T01:24:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.225+0000 7f61a6441740 1 -- 172.21.15.31:0/3309210861 wait complete. 2024-04-25T01:24:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.225+0000 7f61a6441740 1 -- start start 2024-04-25T01:24:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.226+0000 7f61a6441740 1 Processor -- start 2024-04-25T01:24:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.227+0000 7f61a6441740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db84cc5310 0x55db84cd8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.227+0000 7f61a6441740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db84cc9ca0 0x55db84cd8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.227+0000 7f61a1600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db84cc5310 0x55db84cd8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.227+0000 7f61a1600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db84cc5310 0x55db84cd8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:42908/0 (socket says 172.21.15.31:42908) 2024-04-25T01:24:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.227+0000 7f61a1600640 1 -- 172.21.15.31:0/1653443119 learned_addr learned my addr 172.21.15.31:0/1653443119 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:24:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.228+0000 7f61a6441740 1 --2- 172.21.15.31:0/1653443119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db84cca690 0x55db84cd9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.228+0000 7f61a6441740 1 -- 172.21.15.31:0/1653443119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f619c1b5df0 con 0x55db84cca690 2024-04-25T01:24:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.228+0000 7f61a6441740 1 -- 172.21.15.31:0/1653443119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f619c1b7260 con 0x55db84cc5310 2024-04-25T01:24:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.228+0000 7f61a6441740 1 -- 172.21.15.31:0/1653443119 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f619c1b64c0 con 0x55db84cc9ca0 2024-04-25T01:24:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.228+0000 7f61a6441740 10 client.0 _ll_register_callbacks cb 0x7f61880093f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:24:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.228+0000 7f619be00640 1 --2- 172.21.15.31:0/1653443119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db84cc9ca0 0x55db84cd8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.229+0000 7f61a1600640 1 -- 172.21.15.31:0/1653443119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db84cc9ca0 msgr2=0x55db84cd8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:24:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.229+0000 7f61a1600640 1 --2- 172.21.15.31:0/1653443119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db84cc9ca0 0x55db84cd8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.229+0000 7f61a1600640 1 -- 172.21.15.31:0/1653443119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db84cca690 msgr2=0x55db84cd9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:24:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.229+0000 7f61a1600640 1 --2- 172.21.15.31:0/1653443119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db84cca690 0x55db84cd9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.229+0000 7f61a1600640 1 -- 172.21.15.31:0/1653443119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db84cc5140 con 0x55db84cc5310 2024-04-25T01:24:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.229+0000 7f61a1600640 1 --2- 172.21.15.31:0/1653443119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db84cc5310 0x55db84cd8810 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f619402ade0 tx=0x7f6194005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aa262eecb91430f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:24:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.230+0000 7f6182000640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-25T01:24:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.230+0000 7f6182000640 1 -- 172.21.15.31:0/1653443119 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6194050040 con 0x55db84cc5310 2024-04-25T01:24:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.231+0000 7f6182000640 1 -- 172.21.15.31:0/1653443119 <== mon.2 v2:172.21.15.40:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f6194030e00 con 0x55db84cc5310 2024-04-25T01:24:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.231+0000 7f61a6441740 1 -- 172.21.15.31:0/1653443119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f618801d900 con 0x55db84cc5310 2024-04-25T01:24:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.231+0000 7f6182000640 1 -- 172.21.15.31:0/1653443119 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6194004b40 con 0x55db84cc5310 2024-04-25T01:24:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.231+0000 7f61a6441740 10 client.25769 fetch_fsmap learned FSMap version 32 2024-04-25T01:24:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.231+0000 7f61a6441740 1 -- 172.21.15.31:0/1653443119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55db84cdb440 con 0x55db84cc5310 2024-04-25T01:24:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.232+0000 7f6182000640 1 -- 172.21.15.31:0/1653443119 <== mon.2 v2:172.21.15.40:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f618801d900 con 0x55db84cc5310 2024-04-25T01:24:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.233+0000 7f6182000640 1 -- 172.21.15.31:0/1653443119 <== mon.2 v2:172.21.15.40:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f619402f8e0 con 0x55db84cc5310 2024-04-25T01:24:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.234+0000 7f61a6441740 10 client.25769 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T01:24:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.234+0000 7f61a6441740 10 client.25769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T01:24:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.234+0000 7f61a6441740 20 client.25769 populate_metadata read hostname 'smithi031' 2024-04-25T01:24:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.234+0000 7f61a6441740 1 --2- 172.21.15.31:0/1653443119 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55db84ccf800 0x55db84d47dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.234+0000 7f61a6441740 4 client.25769 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T01:24:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.234+0000 7f61a6441740 1 -- 172.21.15.31:0/1653443119 --> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55db84c34340 con 0x55db84ccf800 2024-04-25T01:24:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.235+0000 7f619be00640 1 --2- 172.21.15.31:0/1653443119 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55db84ccf800 0x55db84d47dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.236+0000 7f619be00640 1 --2- 172.21.15.31:0/1653443119 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55db84ccf800 0x55db84d47dc0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f605a88182e0c86d server_cookie=61a44f534e195bb in_seq=0 out_seq=0 2024-04-25T01:24:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.236+0000 7f6182000640 10 client.25769 ms_handle_connect on v2:172.21.15.40:6838/927194004 2024-04-25T01:24:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.403+0000 7f6182000640 1 -- 172.21.15.31:0/1653443119 <== mds.0 v2:172.21.15.40:6838/927194004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55db84c34340 con 0x55db84ccf800 2024-04-25T01:24:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.403+0000 7f6182000640 10 client.25769 handle_command_reply: tid=0 2024-04-25T01:24:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.404+0000 7f61a6441740 1 client.25769 shutdown 2024-04-25T01:24:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.404+0000 7f61a6441740 1 -- 172.21.15.31:0/1653443119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db84cc5310 msgr2=0x55db84cd8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.404+0000 7f61a6441740 1 --2- 172.21.15.31:0/1653443119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db84cc5310 0x55db84cd8810 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f619402ade0 tx=0x7f6194005bd0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.404+0000 7f61a6441740 1 -- 172.21.15.31:0/1653443119 shutdown_connections 2024-04-25T01:24:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.404+0000 7f61a6441740 1 --2- 172.21.15.31:0/1653443119 >> [v2:172.21.15.40:6838/927194004,v1:172.21.15.40:6839/927194004] conn(0x55db84ccf800 0x55db84d47dc0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.404+0000 7f6182000640 0 client.25769 ms_handle_reset on v2:172.21.15.40:6838/927194004 2024-04-25T01:24:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.404+0000 7f61a6441740 1 --2- 172.21.15.31:0/1653443119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db84cca690 0x55db84cd9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.404+0000 7f61a6441740 1 --2- 172.21.15.31:0/1653443119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55db84cc9ca0 0x55db84cd8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.404+0000 7f61a6441740 1 --2- 172.21.15.31:0/1653443119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55db84cc5310 0x55db84cd8810 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.404+0000 7f61a6441740 1 -- 172.21.15.31:0/1653443119 >> 172.21.15.31:0/1653443119 conn(0x55db84ccf2d0 msgr2=0x55db84cd3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:24:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.405+0000 7f61a6441740 1 -- 172.21.15.31:0/1653443119 shutdown_connections 2024-04-25T01:24:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.405+0000 7f61a6441740 1 -- 172.21.15.31:0/1653443119 wait complete. 2024-04-25T01:24:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.405+0000 7f61a6441740 20 client.25769 trim_cache size 0 max 16384 2024-04-25T01:24:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-04-25T01:24:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-04-25T01:24:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-04-25T01:24:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-04-25T01:24:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.406+0000 7f61a3600640 1 -- 172.21.15.31:0/1988312234 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f61840a2c30 msgr2=0x7f61840a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.406+0000 7f61a3600640 1 --2- 172.21.15.31:0/1988312234 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f61840a2c30 0x7f61840a50f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55db84ccaa90 tx=0x7f6190000bc0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.406+0000 7f61a3600640 1 -- 172.21.15.31:0/1988312234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f619c076100 msgr2=0x7f619c10e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.406+0000 7f61a3600640 1 --2- 172.21.15.31:0/1988312234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f619c076100 0x7f619c10e2a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f619000e9f0 tx=0x7f619000eeb0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.407+0000 7f61a3600640 1 -- 172.21.15.31:0/1988312234 shutdown_connections 2024-04-25T01:24:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.407+0000 7f61a3600640 1 --2- 172.21.15.31:0/1988312234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f619c077ce0 0x7f619c1ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.407+0000 7f61a3600640 1 --2- 172.21.15.31:0/1988312234 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f61840a2c30 0x7f61840a50f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.407+0000 7f61a3600640 1 --2- 172.21.15.31:0/1988312234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619c077340 0x7f619c10e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.407+0000 7f61a3600640 1 --2- 172.21.15.31:0/1988312234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f619c076100 0x7f619c10e2a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.407+0000 7f61a3600640 1 -- 172.21.15.31:0/1988312234 >> 172.21.15.31:0/1988312234 conn(0x7f619c0fef60 msgr2=0x7f619c100a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:24:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.407+0000 7f61a3600640 1 -- 172.21.15.31:0/1988312234 shutdown_connections 2024-04-25T01:24:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-04-25T01:24:08.407+0000 7f61a3600640 1 -- 172.21.15.31:0/1988312234 wait complete. 2024-04-25T01:24:09.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:09 smithi169 ceph-mon[32759]: pgmap v637: 97 pgs: 97 active+clean; 13 GiB data, 121 GiB used, 952 GiB / 1.0 TiB avail; 10 KiB/s rd, 104 MiB/s wr, 19.20k op/s 2024-04-25T01:24:09.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:09 smithi040 ceph-mon[32518]: pgmap v637: 97 pgs: 97 active+clean; 13 GiB data, 121 GiB used, 952 GiB / 1.0 TiB avail; 10 KiB/s rd, 104 MiB/s wr, 19.20k op/s 2024-04-25T01:24:09.999 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:09 smithi031 ceph-mon[28053]: pgmap v637: 97 pgs: 97 active+clean; 13 GiB data, 121 GiB used, 952 GiB / 1.0 TiB avail; 10 KiB/s rd, 104 MiB/s wr, 19.20k op/s 2024-04-25T01:24:11.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:11 smithi169 ceph-mon[32759]: pgmap v638: 97 pgs: 97 active+clean; 13 GiB data, 121 GiB used, 952 GiB / 1.0 TiB avail; 11 KiB/s rd, 82 MiB/s wr, 15.29k op/s 2024-04-25T01:24:11.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:11 smithi040 ceph-mon[32518]: pgmap v638: 97 pgs: 97 active+clean; 13 GiB data, 121 GiB used, 952 GiB / 1.0 TiB avail; 11 KiB/s rd, 82 MiB/s wr, 15.29k op/s 2024-04-25T01:24:11.999 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:11 smithi031 ceph-mon[28053]: pgmap v638: 97 pgs: 97 active+clean; 13 GiB data, 121 GiB used, 952 GiB / 1.0 TiB avail; 11 KiB/s rd, 82 MiB/s wr, 15.29k op/s 2024-04-25T01:24:12.698 INFO:tasks.workunit.client.0.smithi031.stdout:Syncing()...29 sec 2024-04-25T01:24:12.698 INFO:tasks.workunit.client.0.smithi031.stdout:FFSB benchmark finished at: Thu Apr 25 01:24:12 2024 2024-04-25T01:24:12.699 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-04-25T01:24:12.699 INFO:tasks.workunit.client.0.smithi031.stdout:Results: 2024-04-25T01:24:12.699 INFO:tasks.workunit.client.0.smithi031.stdout:Benchmark took 333.41 sec 2024-04-25T01:24:12.699 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-04-25T01:24:12.699 INFO:tasks.workunit.client.0.smithi031.stdout:Total Results 2024-04-25T01:24:12.699 INFO:tasks.workunit.client.0.smithi031.stdout:=============== 2024-04-25T01:24:12.699 INFO:tasks.workunit.client.0.smithi031.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2024-04-25T01:24:12.699 INFO:tasks.workunit.client.0.smithi031.stdout: ======= ============ ========= ======= =========== ========== 2024-04-25T01:24:12.699 INFO:tasks.workunit.client.0.smithi031.stdout: write : 14019840 42050.39 100.000% 100.000% 164MB/sec 2024-04-25T01:24:12.699 INFO:tasks.workunit.client.0.smithi031.stdout:- 2024-04-25T01:24:12.699 INFO:tasks.workunit.client.0.smithi031.stdout:42050.39 Transactions per Second 2024-04-25T01:24:12.699 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-04-25T01:24:12.699 INFO:tasks.workunit.client.0.smithi031.stdout:Throughput Results 2024-04-25T01:24:12.700 INFO:tasks.workunit.client.0.smithi031.stdout:=================== 2024-04-25T01:24:12.700 INFO:tasks.workunit.client.0.smithi031.stdout:Write Throughput: 164MB/sec 2024-04-25T01:24:12.841 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-04-25T01:24:12.841 INFO:tasks.workunit.client.0.smithi031.stdout:System Call Latency statistics in millisecs 2024-04-25T01:24:12.841 INFO:tasks.workunit.client.0.smithi031.stdout:===== 2024-04-25T01:24:12.841 INFO:tasks.workunit.client.0.smithi031.stdout: Min Avg Max Total Calls 2024-04-25T01:24:12.841 INFO:tasks.workunit.client.0.smithi031.stdout: ======== ======== ======== ============ 2024-04-25T01:24:12.841 INFO:tasks.workunit.client.0.smithi031.stdout:[ open] 0.018000 0.481919 1542.022949 10953 2024-04-25T01:24:12.841 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-04-25T01:24:12.841 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-04-25T01:24:12.841 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[2] 1000.000000 - 2000.000000 : 1 2024-04-25T01:24:12.841 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[3] 500.000000 - 1000.000000 : 1 2024-04-25T01:24:12.841 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-04-25T01:24:12.842 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[5] 100.000000 - 200.000000 : 1 2024-04-25T01:24:12.842 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[6] 50.000000 - 100.000000 : 2 2024-04-25T01:24:12.842 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[7] 20.000000 - 50.000000 : 3 2024-04-25T01:24:12.842 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[8] 10.000000 - 20.000000 : 25 2024-04-25T01:24:12.842 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[9] 5.000000 - 10.000000 : 88 2024-04-25T01:24:12.842 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[10] 2.000000 - 5.000000 : 235 2024-04-25T01:24:12.842 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[11] 1.000000 - 2.000000 : 118 2024-04-25T01:24:12.842 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[12] 0.500000 - 1.000000 : 108 2024-04-25T01:24:12.842 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[13] 0.200000 - 0.500000 : 170 2024-04-25T01:24:12.842 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[14] 0.100000 - 0.200000 : 1245 2024-04-25T01:24:12.842 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[15] 0.050000 - 0.100000 : 919 2024-04-25T01:24:12.842 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[16] 0.020000 - 0.050000 : 8032 2024-04-25T01:24:12.842 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[17] 0.010000 - 0.020000 : 5 2024-04-25T01:24:12.843 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-04-25T01:24:12.843 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-04-25T01:24:12.843 INFO:tasks.workunit.client.0.smithi031.stdout:[ write] 0.003000 0.687971 16515.511719 14019840 2024-04-25T01:24:12.843 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[0] 5000.000000 - 10000.000000 : 448 2024-04-25T01:24:12.843 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[1] 2000.000000 - 5000.000000 : 1060 2024-04-25T01:24:12.843 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[2] 1000.000000 - 2000.000000 : 777 2024-04-25T01:24:12.843 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[3] 500.000000 - 1000.000000 : 659 2024-04-25T01:24:12.843 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[4] 200.000000 - 500.000000 : 880 2024-04-25T01:24:12.843 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[5] 100.000000 - 200.000000 : 613 2024-04-25T01:24:12.843 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[6] 50.000000 - 100.000000 : 625 2024-04-25T01:24:12.843 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[7] 20.000000 - 50.000000 : 1388 2024-04-25T01:24:12.843 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[8] 10.000000 - 20.000000 : 3696 2024-04-25T01:24:12.843 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[9] 5.000000 - 10.000000 : 5662 2024-04-25T01:24:12.843 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[10] 2.000000 - 5.000000 : 9266 2024-04-25T01:24:12.844 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[11] 1.000000 - 2.000000 : 4536 2024-04-25T01:24:12.844 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[12] 0.500000 - 1.000000 : 3893 2024-04-25T01:24:12.844 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[13] 0.200000 - 0.500000 : 7471 2024-04-25T01:24:12.844 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[14] 0.100000 - 0.200000 : 14955 2024-04-25T01:24:12.844 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[15] 0.050000 - 0.100000 : 50171 2024-04-25T01:24:12.844 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[16] 0.020000 - 0.050000 : 162355 2024-04-25T01:24:12.844 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[17] 0.010000 - 0.020000 : 5940496 2024-04-25T01:24:12.844 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[18] 0.000000 - 0.010000 : 7810818 2024-04-25T01:24:12.844 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-04-25T01:24:12.844 INFO:tasks.workunit.client.0.smithi031.stdout:[ lseek] 0.000000 0.002115 84.732002 14019840 2024-04-25T01:24:12.844 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-04-25T01:24:12.844 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-04-25T01:24:12.844 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-04-25T01:24:12.844 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-04-25T01:24:12.845 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-04-25T01:24:12.845 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-04-25T01:24:12.845 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[6] 50.000000 - 100.000000 : 5 2024-04-25T01:24:12.845 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[7] 20.000000 - 50.000000 : 50 2024-04-25T01:24:12.845 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[8] 10.000000 - 20.000000 : 291 2024-04-25T01:24:12.845 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[9] 5.000000 - 10.000000 : 509 2024-04-25T01:24:12.845 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[10] 2.000000 - 5.000000 : 977 2024-04-25T01:24:12.845 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[11] 1.000000 - 2.000000 : 451 2024-04-25T01:24:12.845 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[12] 0.500000 - 1.000000 : 411 2024-04-25T01:24:12.845 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[13] 0.200000 - 0.500000 : 626 2024-04-25T01:24:12.845 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[14] 0.100000 - 0.200000 : 922 2024-04-25T01:24:12.845 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[15] 0.050000 - 0.100000 : 2931 2024-04-25T01:24:12.845 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[16] 0.020000 - 0.050000 : 13159 2024-04-25T01:24:12.845 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[17] 0.010000 - 0.020000 : 9340 2024-04-25T01:24:12.845 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[18] 0.000000 - 0.010000 : 13990168 2024-04-25T01:24:12.846 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-04-25T01:24:12.846 INFO:tasks.workunit.client.0.smithi031.stdout:[ close] 0.004000 0.021099 24.900999 10953 2024-04-25T01:24:12.846 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-04-25T01:24:12.846 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-04-25T01:24:12.846 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-04-25T01:24:12.846 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-04-25T01:24:12.846 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-04-25T01:24:12.846 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-04-25T01:24:12.846 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-04-25T01:24:12.846 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[7] 20.000000 - 50.000000 : 1 2024-04-25T01:24:12.846 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[8] 10.000000 - 20.000000 : 3 2024-04-25T01:24:12.846 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[9] 5.000000 - 10.000000 : 4 2024-04-25T01:24:12.846 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[10] 2.000000 - 5.000000 : 8 2024-04-25T01:24:12.846 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[11] 1.000000 - 2.000000 : 2 2024-04-25T01:24:12.847 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[12] 0.500000 - 1.000000 : 5 2024-04-25T01:24:12.847 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[13] 0.200000 - 0.500000 : 2 2024-04-25T01:24:12.847 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[14] 0.100000 - 0.200000 : 6 2024-04-25T01:24:12.847 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[15] 0.050000 - 0.100000 : 34 2024-04-25T01:24:12.847 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[16] 0.020000 - 0.050000 : 105 2024-04-25T01:24:12.847 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[17] 0.010000 - 0.020000 : 6351 2024-04-25T01:24:12.847 INFO:tasks.workunit.client.0.smithi031.stdout: msec_range[18] 0.000000 - 0.010000 : 4432 2024-04-25T01:24:12.847 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-04-25T01:24:12.847 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-04-25T01:24:12.847 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-04-25T01:24:12.847 INFO:tasks.workunit.client.0.smithi031.stdout: 2024-04-25T01:24:12.847 INFO:tasks.workunit.client.0.smithi031.stdout:3.0% User Time 2024-04-25T01:24:12.847 INFO:tasks.workunit.client.0.smithi031.stdout:44.3% System Time 2024-04-25T01:24:12.847 INFO:tasks.workunit.client.0.smithi031.stdout:47.3% CPU Utilization 2024-04-25T01:24:12.872 INFO:tasks.workunit.client.0.smithi031.stderr:+ cd .. 2024-04-25T01:24:12.876 INFO:tasks.workunit.client.0.smithi031.stderr:+ rm -r tmp ffsb 2024-04-25T01:24:13.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:13 smithi169 ceph-mon[32759]: pgmap v639: 97 pgs: 97 active+clean; 13 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 18 KiB/s rd, 82 MiB/s wr, 15.46k op/s 2024-04-25T01:24:13.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:13 smithi040 ceph-mon[32518]: pgmap v639: 97 pgs: 97 active+clean; 13 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 18 KiB/s rd, 82 MiB/s wr, 15.46k op/s 2024-04-25T01:24:14.000 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:13 smithi031 ceph-mon[28053]: pgmap v639: 97 pgs: 97 active+clean; 13 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 18 KiB/s rd, 82 MiB/s wr, 15.46k op/s 2024-04-25T01:24:15.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:15 smithi169 ceph-mon[32759]: pgmap v640: 97 pgs: 97 active+clean; 13 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 13 KiB/s rd, 66 MiB/s wr, 12.61k op/s 2024-04-25T01:24:15.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:15 smithi040 ceph-mon[32518]: pgmap v640: 97 pgs: 97 active+clean; 13 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 13 KiB/s rd, 66 MiB/s wr, 12.61k op/s 2024-04-25T01:24:15.999 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:15 smithi031 ceph-mon[28053]: pgmap v640: 97 pgs: 97 active+clean; 13 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 13 KiB/s rd, 66 MiB/s wr, 12.61k op/s 2024-04-25T01:24:17.809 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T01:24:17.809 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T01:24:17.928 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:17 smithi031 ceph-mon[28053]: pgmap v641: 97 pgs: 97 active+clean; 12 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 25 KiB/s rd, 46 MiB/s wr, 9.04k op/s 2024-04-25T01:24:17.938 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:17 smithi169 ceph-mon[32759]: pgmap v641: 97 pgs: 97 active+clean; 12 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 25 KiB/s rd, 46 MiB/s wr, 9.04k op/s 2024-04-25T01:24:17.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:17 smithi040 ceph-mon[32518]: pgmap v641: 97 pgs: 97 active+clean; 12 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 25 KiB/s rd, 46 MiB/s wr, 9.04k op/s 2024-04-25T01:24:18.005 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-04-25T01:24:18.005 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T01:24:19.577 DEBUG:teuthology.parallel:result is None 2024-04-25T01:24:19.577 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T01:24:19.660 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T01:24:19.660 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T01:24:19.672 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T01:24:19.697 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T01:24:19.707 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:24:19.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.954+0000 7f04d2a00640 1 -- 172.21.15.31:0/2597263601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04cc100790 msgr2=0x7f04cc100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:19.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.954+0000 7f04d2a00640 1 --2- 172.21.15.31:0/2597263601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04cc100790 0x7f04cc100bf0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009860 tx=0x7f04b802a340 comp rx=0 tx=0).stop 2024-04-25T01:24:19.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.955+0000 7f04d2a00640 1 -- 172.21.15.31:0/2597263601 shutdown_connections 2024-04-25T01:24:19.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.955+0000 7f04d2a00640 1 --2- 172.21.15.31:0/2597263601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04cc101130 0x7f04cc1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:19.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.955+0000 7f04d2a00640 1 --2- 172.21.15.31:0/2597263601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04cc100790 0x7f04cc100bf0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:19.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.955+0000 7f04d2a00640 1 --2- 172.21.15.31:0/2597263601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04cc0ffe30 0x7f04cc100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:19.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.955+0000 7f04d2a00640 1 -- 172.21.15.31:0/2597263601 >> 172.21.15.31:0/2597263601 conn(0x7f04cc0fba10 msgr2=0x7f04cc0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:24:19.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.955+0000 7f04d2a00640 1 -- 172.21.15.31:0/2597263601 shutdown_connections 2024-04-25T01:24:19.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.955+0000 7f04d2a00640 1 -- 172.21.15.31:0/2597263601 wait complete. 2024-04-25T01:24:19.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.956+0000 7f04d2a00640 1 Processor -- start 2024-04-25T01:24:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.956+0000 7f04d2a00640 1 -- start start 2024-04-25T01:24:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.956+0000 7f04d2a00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04cc0ffe30 0x7f04cc199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.956+0000 7f04d2a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04cc100790 0x7f04cc199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.956+0000 7f04d2a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04cc101130 0x7f04cc1a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.956+0000 7f04d2a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc10f420 con 0x7f04cc101130 2024-04-25T01:24:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.956+0000 7f04d2a00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc10f2a0 con 0x7f04cc0ffe30 2024-04-25T01:24:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.956+0000 7f04d2a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc10f5a0 con 0x7f04cc100790 2024-04-25T01:24:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.956+0000 7f04d1600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04cc0ffe30 0x7f04cc199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.956+0000 7f04d1600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04cc0ffe30 0x7f04cc199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:59452/0 (socket says 172.21.15.31:59452) 2024-04-25T01:24:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.956+0000 7f04d1600640 1 -- 172.21.15.31:0/1128299280 learned_addr learned my addr 172.21.15.31:0/1128299280 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:24:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.956+0000 7f04d0c00640 1 --2- 172.21.15.31:0/1128299280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04cc100790 0x7f04cc199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.956+0000 7f04d2000640 1 --2- 172.21.15.31:0/1128299280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04cc101130 0x7f04cc1a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.957+0000 7f04d1600640 1 -- 172.21.15.31:0/1128299280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04cc100790 msgr2=0x7f04cc199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.957+0000 7f04d1600640 1 --2- 172.21.15.31:0/1128299280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04cc100790 0x7f04cc199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.957+0000 7f04d1600640 1 -- 172.21.15.31:0/1128299280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04cc101130 msgr2=0x7f04cc1a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.957+0000 7f04d1600640 1 --2- 172.21.15.31:0/1128299280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04cc101130 0x7f04cc1a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.957+0000 7f04d1600640 1 -- 172.21.15.31:0/1128299280 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04cc1a16c0 con 0x7f04cc0ffe30 2024-04-25T01:24:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.957+0000 7f04d0c00640 1 --2- 172.21.15.31:0/1128299280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04cc100790 0x7f04cc199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:24:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.957+0000 7f04d1600640 1 --2- 172.21.15.31:0/1128299280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04cc0ffe30 0x7f04cc199aa0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f04c000e9f0 tx=0x7f04c000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:24:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.957+0000 7f04ca000640 1 -- 172.21.15.31:0/1128299280 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c000cd80 con 0x7f04cc0ffe30 2024-04-25T01:24:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.957+0000 7f04ca000640 1 -- 172.21.15.31:0/1128299280 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c0005500 con 0x7f04cc0ffe30 2024-04-25T01:24:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.957+0000 7f04d2a00640 1 -- 172.21.15.31:0/1128299280 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04cc1a19b0 con 0x7f04cc0ffe30 2024-04-25T01:24:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.957+0000 7f04ca000640 1 -- 172.21.15.31:0/1128299280 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c0010690 con 0x7f04cc0ffe30 2024-04-25T01:24:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.957+0000 7f04d2a00640 1 -- 172.21.15.31:0/1128299280 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04cc1a1ed0 con 0x7f04cc0ffe30 2024-04-25T01:24:19.940 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:19 smithi169 ceph-mon[32759]: pgmap v642: 97 pgs: 97 active+clean; 8.3 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 81 KiB/s rd, 43 MiB/s wr, 8.46k op/s 2024-04-25T01:24:19.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.959+0000 7f04d2a00640 1 -- 172.21.15.31:0/1128299280 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04cc10ab80 con 0x7f04cc0ffe30 2024-04-25T01:24:19.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.961+0000 7f04ca000640 1 -- 172.21.15.31:0/1128299280 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f04c00046c0 con 0x7f04cc0ffe30 2024-04-25T01:24:19.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:19 smithi040 ceph-mon[32518]: pgmap v642: 97 pgs: 97 active+clean; 8.3 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 81 KiB/s rd, 43 MiB/s wr, 8.46k op/s 2024-04-25T01:24:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.962+0000 7f04ca000640 1 --2- 172.21.15.31:0/1128299280 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f04b40a2c10 0x7f04b40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.962+0000 7f04ca000640 1 -- 172.21.15.31:0/1128299280 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f04c00cf3c0 con 0x7f04cc0ffe30 2024-04-25T01:24:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.962+0000 7f04d0c00640 1 --2- 172.21.15.31:0/1128299280 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f04b40a2c10 0x7f04b40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:19.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.963+0000 7f04d0c00640 1 --2- 172.21.15.31:0/1128299280 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f04b40a2c10 0x7f04b40a50d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009830 tx=0x7f04b8030550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:24:19.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:19.964+0000 7f04ca000640 1 -- 172.21.15.31:0/1128299280 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f04c009c310 con 0x7f04cc0ffe30 2024-04-25T01:24:19.999 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:19 smithi031 ceph-mon[28053]: pgmap v642: 97 pgs: 97 active+clean; 8.3 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 81 KiB/s rd, 43 MiB/s wr, 8.46k op/s 2024-04-25T01:24:20.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.179+0000 7f04d2a00640 1 -- 172.21.15.31:0/1128299280 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04cc062370 con 0x7f04cc0ffe30 2024-04-25T01:24:20.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.181+0000 7f04ca000640 1 -- 172.21.15.31:0/1128299280 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22530 (secure 0 0 0) 0x7f04c009bc10 con 0x7f04cc0ffe30 2024-04-25T01:24:20.161 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:24:20.162 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":89,"fsid":"95849b94-029f-11ef-bc93-c7b262605968","created":"2024-04-25T01:04:23.310856+0000","modified":"2024-04-25T01:14:33.793638+0000","last_up_change":"2024-04-25T01:12:29.393907+0000","last_in_change":"2024-04-25T01:12:08.978148+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:18.662391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:13:29.258771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:13:30.480996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa08e4c-fe6b-4116-bdf5-1adf5b465d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6803","nonce":788174091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6805","nonce":788174091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6809","nonce":788174091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6807","nonce":788174091}]},"public_addr":"172.21.15.31:6803/788174091","cluster_addr":"172.21.15.31:6805/788174091","heartbeat_back_addr":"172.21.15.31:6809/788174091","heartbeat_front_addr":"172.21.15.31:6807/788174091","state":["exists","up"]},{"osd":1,"uuid":"85339fe7-0dbc-4485-9334-2b039993c1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6801","nonce":3196394509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6803","nonce":3196394509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6807","nonce":3196394509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6805","nonce":3196394509}]},"public_addr":"172.21.15.40:6801/3196394509","cluster_addr":"172.21.15.40:6803/3196394509","heartbeat_back_addr":"172.21.15.40:6807/3196394509","heartbeat_front_addr":"172.21.15.40:6805/3196394509","state":["exists","up"]},{"osd":2,"uuid":"ab62ecb2-8aae-4987-9e2e-589e0f2fee5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6801","nonce":1955211197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6803","nonce":1955211197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6807","nonce":1955211197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6805","nonce":1955211197}]},"public_addr":"172.21.15.169:6801/1955211197","cluster_addr":"172.21.15.169:6803/1955211197","heartbeat_back_addr":"172.21.15.169:6807/1955211197","heartbeat_front_addr":"172.21.15.169:6805/1955211197","state":["exists","up"]},{"osd":3,"uuid":"8b5a7881-3401-495c-8ca4-c90685919d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6811","nonce":1228688643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6813","nonce":1228688643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6817","nonce":1228688643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6815","nonce":1228688643}]},"public_addr":"172.21.15.31:6811/1228688643","cluster_addr":"172.21.15.31:6813/1228688643","heartbeat_back_addr":"172.21.15.31:6817/1228688643","heartbeat_front_addr":"172.21.15.31:6815/1228688643","state":["exists","up"]},{"osd":4,"uuid":"56deb61d-a334-4cf2-8ac7-fbb6e6947092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6809","nonce":1913779091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6811","nonce":1913779091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6815","nonce":1913779091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6813","nonce":1913779091}]},"public_addr":"172.21.15.40:6809/1913779091","cluster_addr":"172.21.15.40:6811/1913779091","heartbeat_back_addr":"172.21.15.40:6815/1913779091","heartbeat_front_addr":"172.21.15.40:6813/1913779091","state":["exists","up"]},{"osd":5,"uuid":"8da55693-f9c7-49a7-99e0-165078f8001b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6809","nonce":2549650781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6811","nonce":2549650781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6815","nonce":2549650781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6813","nonce":2549650781}]},"public_addr":"172.21.15.169:6809/2549650781","cluster_addr":"172.21.15.169:6811/2549650781","heartbeat_back_addr":"172.21.15.169:6815/2549650781","heartbeat_front_addr":"172.21.15.169:6813/2549650781","state":["exists","up"]},{"osd":6,"uuid":"fc8350d2-1ad6-43cb-89f4-d336afc99ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6819","nonce":611507624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6821","nonce":611507624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6825","nonce":611507624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6823","nonce":611507624}]},"public_addr":"172.21.15.31:6819/611507624","cluster_addr":"172.21.15.31:6821/611507624","heartbeat_back_addr":"172.21.15.31:6825/611507624","heartbeat_front_addr":"172.21.15.31:6823/611507624","state":["exists","up"]},{"osd":7,"uuid":"3121b58b-8c94-40b5-9d48-f56552d7c39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6817","nonce":3776085849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6819","nonce":3776085849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6823","nonce":3776085849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6821","nonce":3776085849}]},"public_addr":"172.21.15.40:6817/3776085849","cluster_addr":"172.21.15.40:6819/3776085849","heartbeat_back_addr":"172.21.15.40:6823/3776085849","heartbeat_front_addr":"172.21.15.40:6821/3776085849","state":["exists","up"]},{"osd":8,"uuid":"f60407df-91f5-477a-9702-ddaef24faf82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6817","nonce":2736546122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6819","nonce":2736546122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6823","nonce":2736546122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6821","nonce":2736546122}]},"public_addr":"172.21.15.169:6817/2736546122","cluster_addr":"172.21.15.169:6819/2736546122","heartbeat_back_addr":"172.21.15.169:6823/2736546122","heartbeat_front_addr":"172.21.15.169:6821/2736546122","state":["exists","up"]},{"osd":9,"uuid":"7506a407-1960-49bc-9e38-7f2a8bf996ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6827","nonce":1187207880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6829","nonce":1187207880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6833","nonce":1187207880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6831","nonce":1187207880}]},"public_addr":"172.21.15.31:6827/1187207880","cluster_addr":"172.21.15.31:6829/1187207880","heartbeat_back_addr":"172.21.15.31:6833/1187207880","heartbeat_front_addr":"172.21.15.31:6831/1187207880","state":["exists","up"]},{"osd":10,"uuid":"831c48e2-b0d0-451f-959b-228e927542cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6825","nonce":3050148001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6827","nonce":3050148001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6831","nonce":3050148001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6829","nonce":3050148001}]},"public_addr":"172.21.15.40:6825/3050148001","cluster_addr":"172.21.15.40:6827/3050148001","heartbeat_back_addr":"172.21.15.40:6831/3050148001","heartbeat_front_addr":"172.21.15.40:6829/3050148001","state":["exists","up"]},{"osd":11,"uuid":"144fe0aa-f757-4b3c-90ad-99775e17d57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6825","nonce":262401944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6827","nonce":262401944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6831","nonce":262401944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6829","nonce":262401944}]},"public_addr":"172.21.15.169:6825/262401944","cluster_addr":"172.21.15.169:6827/262401944","heartbeat_back_addr":"172.21.15.169:6831/262401944","heartbeat_front_addr":"172.21.15.169:6829/262401944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:25.640995+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:51.036909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.635633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:48.467912+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:14.454971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:37.609315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:12.441807+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:38.239215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:02.230758+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:35.762553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:02.051405+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:26.569593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6801/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4004014066":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/1930101646":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/2638309002":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2238638505":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2902841144":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4190696323":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/480558521":"2024-04-26T01:04:57.149272+0000","172.21.15.31:6801/2638309002":"2024-04-26T01:04:57.149272+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:24:20.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.183+0000 7f04d2a00640 1 -- 172.21.15.31:0/1128299280 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f04b40a2c10 msgr2=0x7f04b40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:20.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.183+0000 7f04d2a00640 1 --2- 172.21.15.31:0/1128299280 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f04b40a2c10 0x7f04b40a50d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009830 tx=0x7f04b8030550 comp rx=0 tx=0).stop 2024-04-25T01:24:20.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.184+0000 7f04d2a00640 1 -- 172.21.15.31:0/1128299280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04cc0ffe30 msgr2=0x7f04cc199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:20.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.184+0000 7f04d2a00640 1 --2- 172.21.15.31:0/1128299280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04cc0ffe30 0x7f04cc199aa0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f04c000e9f0 tx=0x7f04c000edf0 comp rx=0 tx=0).stop 2024-04-25T01:24:20.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.184+0000 7f04d2a00640 1 -- 172.21.15.31:0/1128299280 shutdown_connections 2024-04-25T01:24:20.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.184+0000 7f04d2a00640 1 --2- 172.21.15.31:0/1128299280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04cc101130 0x7f04cc1a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:20.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.184+0000 7f04d2a00640 1 --2- 172.21.15.31:0/1128299280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04cc100790 0x7f04cc199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:20.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.184+0000 7f04d2a00640 1 --2- 172.21.15.31:0/1128299280 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f04b40a2c10 0x7f04b40a50d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:20.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.184+0000 7f04d2a00640 1 --2- 172.21.15.31:0/1128299280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04cc0ffe30 0x7f04cc199aa0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:20.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.184+0000 7f04d2a00640 1 -- 172.21.15.31:0/1128299280 >> 172.21.15.31:0/1128299280 conn(0x7f04cc0fba10 msgr2=0x7f04cc0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:24:20.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.184+0000 7f04d2a00640 1 -- 172.21.15.31:0/1128299280 shutdown_connections 2024-04-25T01:24:20.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.184+0000 7f04d2a00640 1 -- 172.21.15.31:0/1128299280 wait complete. 2024-04-25T01:24:20.181 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:18.662391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:13:29.258771+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:13:30.480996+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:24:20.182 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:24:20.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.372+0000 7f5be3e00640 1 -- 172.21.15.31:0/4141915685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5be4102860 msgr2=0x7f5be4102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:20.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.372+0000 7f5be3e00640 1 --2- 172.21.15.31:0/4141915685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5be4102860 0x7f5be4102ce0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0009900 tx=0x7f5bd002a410 comp rx=0 tx=0).stop 2024-04-25T01:24:20.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.372+0000 7f5be3e00640 1 -- 172.21.15.31:0/4141915685 shutdown_connections 2024-04-25T01:24:20.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.372+0000 7f5be3e00640 1 --2- 172.21.15.31:0/4141915685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5be4103220 0x7f5be4109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:20.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.372+0000 7f5be3e00640 1 --2- 172.21.15.31:0/4141915685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5be4102860 0x7f5be4102ce0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:20.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.372+0000 7f5be3e00640 1 --2- 172.21.15.31:0/4141915685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5be4101670 0x7f5be4101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:20.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.372+0000 7f5be3e00640 1 -- 172.21.15.31:0/4141915685 >> 172.21.15.31:0/4141915685 conn(0x7f5be40fcbe0 msgr2=0x7f5be40ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:24:20.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.372+0000 7f5be3e00640 1 -- 172.21.15.31:0/4141915685 shutdown_connections 2024-04-25T01:24:20.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.373+0000 7f5be3e00640 1 -- 172.21.15.31:0/4141915685 wait complete. 2024-04-25T01:24:20.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.373+0000 7f5be3e00640 1 Processor -- start 2024-04-25T01:24:20.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.374+0000 7f5be3e00640 1 -- start start 2024-04-25T01:24:20.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.374+0000 7f5be3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5be4101670 0x7f5be40754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:20.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.374+0000 7f5be3e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5be4102860 0x7f5be4073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:20.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.374+0000 7f5be3e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5be4103220 0x7f5be4074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.374+0000 7f5be3e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5be410d2e0 con 0x7f5be4102860 2024-04-25T01:24:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.374+0000 7f5be3e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5be410d160 con 0x7f5be4103220 2024-04-25T01:24:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.374+0000 7f5be3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5be410d460 con 0x7f5be4101670 2024-04-25T01:24:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.374+0000 7f5be2a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5be4101670 0x7f5be40754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.374+0000 7f5be2a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5be4101670 0x7f5be40754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:56422/0 (socket says 172.21.15.31:56422) 2024-04-25T01:24:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.374+0000 7f5be2a00640 1 -- 172.21.15.31:0/2740392354 learned_addr learned my addr 172.21.15.31:0/2740392354 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:24:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.374+0000 7f5be2000640 1 --2- 172.21.15.31:0/2740392354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5be4102860 0x7f5be4073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.374+0000 7f5be3400640 1 --2- 172.21.15.31:0/2740392354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5be4103220 0x7f5be4074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.375+0000 7f5be3400640 1 -- 172.21.15.31:0/2740392354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5be4101670 msgr2=0x7f5be40754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.375+0000 7f5be3400640 1 --2- 172.21.15.31:0/2740392354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5be4101670 0x7f5be40754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.375+0000 7f5be3400640 1 -- 172.21.15.31:0/2740392354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5be4102860 msgr2=0x7f5be4073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.375+0000 7f5be3400640 1 --2- 172.21.15.31:0/2740392354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5be4102860 0x7f5be4073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.375+0000 7f5be3400640 1 -- 172.21.15.31:0/2740392354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be4074900 con 0x7f5be4103220 2024-04-25T01:24:20.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.375+0000 7f5be2a00640 1 --2- 172.21.15.31:0/2740392354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5be4101670 0x7f5be40754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:24:20.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.375+0000 7f5be3400640 1 --2- 172.21.15.31:0/2740392354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5be4103220 0x7f5be4074070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc009fd0 tx=0x7f5bdc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:24:20.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.375+0000 7f5bdb400640 1 -- 172.21.15.31:0/2740392354 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc00ce20 con 0x7f5be4103220 2024-04-25T01:24:20.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.375+0000 7f5bdb400640 1 -- 172.21.15.31:0/2740392354 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bdc005020 con 0x7f5be4103220 2024-04-25T01:24:20.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.375+0000 7f5be3e00640 1 -- 172.21.15.31:0/2740392354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be41a1aa0 con 0x7f5be4103220 2024-04-25T01:24:20.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.375+0000 7f5bdb400640 1 -- 172.21.15.31:0/2740392354 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc010720 con 0x7f5be4103220 2024-04-25T01:24:20.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.375+0000 7f5be3e00640 1 -- 172.21.15.31:0/2740392354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be41a1fc0 con 0x7f5be4103220 2024-04-25T01:24:20.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.376+0000 7f5be3e00640 1 -- 172.21.15.31:0/2740392354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be4067bf0 con 0x7f5be4103220 2024-04-25T01:24:20.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.378+0000 7f5bdb400640 1 -- 172.21.15.31:0/2740392354 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5bdc005540 con 0x7f5be4103220 2024-04-25T01:24:20.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.380+0000 7f5bdb400640 1 --2- 172.21.15.31:0/2740392354 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5bc80a2c10 0x7f5bc80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:20.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.380+0000 7f5bdb400640 1 -- 172.21.15.31:0/2740392354 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f5bdc0cf710 con 0x7f5be4103220 2024-04-25T01:24:20.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.380+0000 7f5be2a00640 1 --2- 172.21.15.31:0/2740392354 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5bc80a2c10 0x7f5bc80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:20.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.381+0000 7f5be2a00640 1 --2- 172.21.15.31:0/2740392354 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5bc80a2c10 0x7f5bc80a50d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40058c0 tx=0x7f5bd4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:24:20.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.383+0000 7f5bdb400640 1 -- 172.21.15.31:0/2740392354 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5bdc09c660 con 0x7f5be4103220 2024-04-25T01:24:20.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.592+0000 7f5be3e00640 1 -- 172.21.15.31:0/2740392354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5be4062340 con 0x7f5be4103220 2024-04-25T01:24:20.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.592+0000 7f5bdb400640 1 -- 172.21.15.31:0/2740392354 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f5bdc09bf60 con 0x7f5be4103220 2024-04-25T01:24:20.572 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T01:24:20.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.595+0000 7f5be3e00640 1 -- 172.21.15.31:0/2740392354 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5bc80a2c10 msgr2=0x7f5bc80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:20.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.595+0000 7f5be3e00640 1 --2- 172.21.15.31:0/2740392354 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5bc80a2c10 0x7f5bc80a50d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40058c0 tx=0x7f5bd4005f90 comp rx=0 tx=0).stop 2024-04-25T01:24:20.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.595+0000 7f5be3e00640 1 -- 172.21.15.31:0/2740392354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5be4103220 msgr2=0x7f5be4074070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:20.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.595+0000 7f5be3e00640 1 --2- 172.21.15.31:0/2740392354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5be4103220 0x7f5be4074070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc009fd0 tx=0x7f5bdc00ef10 comp rx=0 tx=0).stop 2024-04-25T01:24:20.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.595+0000 7f5be3e00640 1 -- 172.21.15.31:0/2740392354 shutdown_connections 2024-04-25T01:24:20.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.595+0000 7f5be3e00640 1 --2- 172.21.15.31:0/2740392354 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5bc80a2c10 0x7f5bc80a50d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:20.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.595+0000 7f5be3e00640 1 --2- 172.21.15.31:0/2740392354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5be4103220 0x7f5be4074070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:20.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.595+0000 7f5be3e00640 1 --2- 172.21.15.31:0/2740392354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5be4102860 0x7f5be4073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:20.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.595+0000 7f5be3e00640 1 --2- 172.21.15.31:0/2740392354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5be4101670 0x7f5be40754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:20.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.595+0000 7f5be3e00640 1 -- 172.21.15.31:0/2740392354 >> 172.21.15.31:0/2740392354 conn(0x7f5be40fcbe0 msgr2=0x7f5be4105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:24:20.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.596+0000 7f5be3e00640 1 -- 172.21.15.31:0/2740392354 shutdown_connections 2024-04-25T01:24:20.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.596+0000 7f5be3e00640 1 -- 172.21.15.31:0/2740392354 wait complete. 2024-04-25T01:24:20.594 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:24:20.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.809+0000 7f8b39c00640 1 -- 172.21.15.31:0/474781485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b34102970 msgr2=0x7f8b34102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:20.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.809+0000 7f8b39c00640 1 --2- 172.21.15.31:0/474781485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b34102970 0x7f8b34102df0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8b20009900 tx=0x7f8b2002a3f0 comp rx=0 tx=0).stop 2024-04-25T01:24:20.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.809+0000 7f8b39c00640 1 -- 172.21.15.31:0/474781485 shutdown_connections 2024-04-25T01:24:20.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.809+0000 7f8b39c00640 1 --2- 172.21.15.31:0/474781485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b34103330 0x7f8b34109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:20.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.809+0000 7f8b39c00640 1 --2- 172.21.15.31:0/474781485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b34102970 0x7f8b34102df0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:20.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.809+0000 7f8b39c00640 1 --2- 172.21.15.31:0/474781485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b34101780 0x7f8b34101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:20.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.809+0000 7f8b39c00640 1 -- 172.21.15.31:0/474781485 >> 172.21.15.31:0/474781485 conn(0x7f8b340fccf0 msgr2=0x7f8b340ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:24:20.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.809+0000 7f8b39c00640 1 -- 172.21.15.31:0/474781485 shutdown_connections 2024-04-25T01:24:20.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.809+0000 7f8b39c00640 1 -- 172.21.15.31:0/474781485 wait complete. 2024-04-25T01:24:20.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.810+0000 7f8b39c00640 1 Processor -- start 2024-04-25T01:24:20.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.810+0000 7f8b39c00640 1 -- start start 2024-04-25T01:24:20.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.811+0000 7f8b39c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b34101780 0x7f8b34199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:20.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.811+0000 7f8b39c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b34102970 0x7f8b3419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:20.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.811+0000 7f8b39c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b34103330 0x7f8b341a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:20.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.811+0000 7f8b39c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3410d450 con 0x7f8b34101780 2024-04-25T01:24:20.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.811+0000 7f8b39c00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3410d2d0 con 0x7f8b34103330 2024-04-25T01:24:20.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.811+0000 7f8b39c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3410d5d0 con 0x7f8b34102970 2024-04-25T01:24:20.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.811+0000 7f8b32a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b34101780 0x7f8b34199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:20.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.811+0000 7f8b32a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b34101780 0x7f8b34199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41006/0 (socket says 172.21.15.31:41006) 2024-04-25T01:24:20.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.811+0000 7f8b33400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b34103330 0x7f8b341a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:20.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.811+0000 7f8b32a00640 1 -- 172.21.15.31:0/2459327139 learned_addr learned my addr 172.21.15.31:0/2459327139 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:24:20.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.811+0000 7f8b32000640 1 --2- 172.21.15.31:0/2459327139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b34102970 0x7f8b3419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:20.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.812+0000 7f8b33400640 1 -- 172.21.15.31:0/2459327139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b34102970 msgr2=0x7f8b3419a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:20.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.812+0000 7f8b33400640 1 --2- 172.21.15.31:0/2459327139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b34102970 0x7f8b3419a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:20.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.812+0000 7f8b33400640 1 -- 172.21.15.31:0/2459327139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b34101780 msgr2=0x7f8b34199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:20.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.812+0000 7f8b33400640 1 --2- 172.21.15.31:0/2459327139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b34101780 0x7f8b34199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:20.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.812+0000 7f8b33400640 1 -- 172.21.15.31:0/2459327139 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b341a1910 con 0x7f8b34103330 2024-04-25T01:24:20.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.812+0000 7f8b33400640 1 --2- 172.21.15.31:0/2459327139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b34103330 0x7f8b341a1270 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c009fd0 tx=0x7f8b2c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:24:20.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.812+0000 7f8b2b400640 1 -- 172.21.15.31:0/2459327139 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c00ce20 con 0x7f8b34103330 2024-04-25T01:24:20.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.812+0000 7f8b2b400640 1 -- 172.21.15.31:0/2459327139 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b2c005020 con 0x7f8b34103330 2024-04-25T01:24:20.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.812+0000 7f8b39c00640 1 -- 172.21.15.31:0/2459327139 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b341a1c00 con 0x7f8b34103330 2024-04-25T01:24:20.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.812+0000 7f8b2b400640 1 -- 172.21.15.31:0/2459327139 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c010720 con 0x7f8b34103330 2024-04-25T01:24:20.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.812+0000 7f8b39c00640 1 -- 172.21.15.31:0/2459327139 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b341a2100 con 0x7f8b34103330 2024-04-25T01:24:20.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.813+0000 7f8b39c00640 1 -- 172.21.15.31:0/2459327139 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b341a22d0 con 0x7f8b34103330 2024-04-25T01:24:20.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.815+0000 7f8b2b400640 1 -- 172.21.15.31:0/2459327139 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8b2c005540 con 0x7f8b34103330 2024-04-25T01:24:20.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.816+0000 7f8b2b400640 1 --2- 172.21.15.31:0/2459327139 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f8b180a2b90 0x7f8b180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:20.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.817+0000 7f8b2b400640 1 -- 172.21.15.31:0/2459327139 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f8b2c0cf7f0 con 0x7f8b34103330 2024-04-25T01:24:20.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.817+0000 7f8b32a00640 1 --2- 172.21.15.31:0/2459327139 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f8b180a2b90 0x7f8b180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:20.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.817+0000 7f8b32a00640 1 --2- 172.21.15.31:0/2459327139 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f8b180a2b90 0x7f8b180a5050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8b3419b1d0 tx=0x7f8b24005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:24:20.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:20.821+0000 7f8b2b400640 1 -- 172.21.15.31:0/2459327139 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8b2c09c740 con 0x7f8b34103330 2024-04-25T01:24:20.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:20 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1128299280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:24:20.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:20 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2740392354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:24:20.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:20 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1128299280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:24:20.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:20 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2740392354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:24:21.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:20 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1128299280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:24:21.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:20 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2740392354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:24:21.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.127+0000 7f8b39c00640 1 -- 172.21.15.31:0/2459327139 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8b341a24a0 con 0x7f8b34103330 2024-04-25T01:24:21.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.127+0000 7f8b2b400640 1 -- 172.21.15.31:0/2459327139 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f8b2c09c040 con 0x7f8b34103330 2024-04-25T01:24:21.108 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-25T01:24:21.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.131+0000 7f8b39c00640 1 -- 172.21.15.31:0/2459327139 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f8b180a2b90 msgr2=0x7f8b180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:21.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.131+0000 7f8b39c00640 1 --2- 172.21.15.31:0/2459327139 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f8b180a2b90 0x7f8b180a5050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8b3419b1d0 tx=0x7f8b24005f90 comp rx=0 tx=0).stop 2024-04-25T01:24:21.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.131+0000 7f8b39c00640 1 -- 172.21.15.31:0/2459327139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b34103330 msgr2=0x7f8b341a1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:21.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.131+0000 7f8b39c00640 1 --2- 172.21.15.31:0/2459327139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b34103330 0x7f8b341a1270 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c009fd0 tx=0x7f8b2c00ef10 comp rx=0 tx=0).stop 2024-04-25T01:24:21.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.131+0000 7f8b39c00640 1 -- 172.21.15.31:0/2459327139 shutdown_connections 2024-04-25T01:24:21.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.131+0000 7f8b39c00640 1 --2- 172.21.15.31:0/2459327139 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f8b180a2b90 0x7f8b180a5050 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:21.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.131+0000 7f8b39c00640 1 --2- 172.21.15.31:0/2459327139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b34103330 0x7f8b341a1270 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:21.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.131+0000 7f8b39c00640 1 --2- 172.21.15.31:0/2459327139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b34102970 0x7f8b3419a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:21.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.131+0000 7f8b39c00640 1 --2- 172.21.15.31:0/2459327139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b34101780 0x7f8b34199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:21.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.131+0000 7f8b39c00640 1 -- 172.21.15.31:0/2459327139 >> 172.21.15.31:0/2459327139 conn(0x7f8b340fccf0 msgr2=0x7f8b34073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:24:21.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.131+0000 7f8b39c00640 1 -- 172.21.15.31:0/2459327139 shutdown_connections 2024-04-25T01:24:21.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.131+0000 7f8b39c00640 1 -- 172.21.15.31:0/2459327139 wait complete. 2024-04-25T01:24:21.140 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:24:21.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.400+0000 7f6579200640 1 -- 172.21.15.31:0/3654823760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6574102980 msgr2=0x7f6574102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:21.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.400+0000 7f6579200640 1 --2- 172.21.15.31:0/3654823760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6574102980 0x7f6574102e00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6560009900 tx=0x7f656002a400 comp rx=0 tx=0).stop 2024-04-25T01:24:21.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.400+0000 7f6579200640 1 -- 172.21.15.31:0/3654823760 shutdown_connections 2024-04-25T01:24:21.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.400+0000 7f6579200640 1 --2- 172.21.15.31:0/3654823760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6574103340 0x7f6574109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:21.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.400+0000 7f6579200640 1 --2- 172.21.15.31:0/3654823760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6574102980 0x7f6574102e00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:21.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.400+0000 7f6579200640 1 --2- 172.21.15.31:0/3654823760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6574101790 0x7f6574101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:21.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.400+0000 7f6579200640 1 -- 172.21.15.31:0/3654823760 >> 172.21.15.31:0/3654823760 conn(0x7f65740fcd20 msgr2=0x7f65740ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:24:21.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.400+0000 7f6579200640 1 -- 172.21.15.31:0/3654823760 shutdown_connections 2024-04-25T01:24:21.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.400+0000 7f6579200640 1 -- 172.21.15.31:0/3654823760 wait complete. 2024-04-25T01:24:21.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.402+0000 7f6579200640 1 Processor -- start 2024-04-25T01:24:21.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.402+0000 7f6579200640 1 -- start start 2024-04-25T01:24:21.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.402+0000 7f6579200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6574101790 0x7f6574110b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:21.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.402+0000 7f6579200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6574102980 0x7f65741110d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:21.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.402+0000 7f6579200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6574103340 0x7f6574118140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:21.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.402+0000 7f6579200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f65740751c0 con 0x7f6574103340 2024-04-25T01:24:21.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.402+0000 7f6579200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6574075040 con 0x7f6574102980 2024-04-25T01:24:21.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.402+0000 7f6579200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6574075340 con 0x7f6574101790 2024-04-25T01:24:21.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.404+0000 7f6571e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6574101790 0x7f6574110b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:21.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.404+0000 7f6571e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6574101790 0x7f6574110b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:56454/0 (socket says 172.21.15.31:56454) 2024-04-25T01:24:21.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.404+0000 7f6571e00640 1 -- 172.21.15.31:0/3117342106 learned_addr learned my addr 172.21.15.31:0/3117342106 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:24:21.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.404+0000 7f6571400640 1 --2- 172.21.15.31:0/3117342106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6574102980 0x7f65741110d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:21.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.404+0000 7f6571e00640 1 -- 172.21.15.31:0/3117342106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6574102980 msgr2=0x7f65741110d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:21.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.404+0000 7f6572800640 1 --2- 172.21.15.31:0/3117342106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6574103340 0x7f6574118140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:21.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.404+0000 7f6571e00640 1 --2- 172.21.15.31:0/3117342106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6574102980 0x7f65741110d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:21.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.404+0000 7f6571e00640 1 -- 172.21.15.31:0/3117342106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6574103340 msgr2=0x7f6574118140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:21.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.404+0000 7f6571e00640 1 --2- 172.21.15.31:0/3117342106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6574103340 0x7f6574118140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:21.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.404+0000 7f6571e00640 1 -- 172.21.15.31:0/3117342106 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65741ae550 con 0x7f6574101790 2024-04-25T01:24:21.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.405+0000 7f6571e00640 1 --2- 172.21.15.31:0/3117342106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6574101790 0x7f6574110b90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f656400e9c0 tx=0x7f656400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:24:21.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.405+0000 7f656aa00640 1 -- 172.21.15.31:0/3117342106 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656400cd80 con 0x7f6574101790 2024-04-25T01:24:21.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.405+0000 7f656aa00640 1 -- 172.21.15.31:0/3117342106 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65640035f0 con 0x7f6574101790 2024-04-25T01:24:21.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.405+0000 7f656aa00640 1 -- 172.21.15.31:0/3117342106 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6564010690 con 0x7f6574101790 2024-04-25T01:24:21.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.405+0000 7f6579200640 1 -- 172.21.15.31:0/3117342106 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65741ae7e0 con 0x7f6574101790 2024-04-25T01:24:21.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.405+0000 7f6579200640 1 -- 172.21.15.31:0/3117342106 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65741aece0 con 0x7f6574101790 2024-04-25T01:24:21.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.407+0000 7f6579200640 1 -- 172.21.15.31:0/3117342106 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65741089e0 con 0x7f6574101790 2024-04-25T01:24:21.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.409+0000 7f656aa00640 1 -- 172.21.15.31:0/3117342106 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f65640037a0 con 0x7f6574101790 2024-04-25T01:24:21.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.411+0000 7f656aa00640 1 --2- 172.21.15.31:0/3117342106 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f65580a2c10 0x7f65580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:21.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.411+0000 7f656aa00640 1 -- 172.21.15.31:0/3117342106 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f65640cf510 con 0x7f6574101790 2024-04-25T01:24:21.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.412+0000 7f6571400640 1 --2- 172.21.15.31:0/3117342106 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f65580a2c10 0x7f65580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:21.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.413+0000 7f6571400640 1 --2- 172.21.15.31:0/3117342106 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f65580a2c10 0x7f65580a50d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6560031610 tx=0x7f6560005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:24:21.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.417+0000 7f656aa00640 1 -- 172.21.15.31:0/3117342106 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f656409c460 con 0x7f6574101790 2024-04-25T01:24:21.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.685+0000 7f6579200640 1 -- 172.21.15.31:0/3117342106 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f65740623e0 con 0x7f6574101790 2024-04-25T01:24:21.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.686+0000 7f656aa00640 1 -- 172.21.15.31:0/3117342106 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f656409bd60 con 0x7f6574101790 2024-04-25T01:24:21.666 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-04-25T01:24:21.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.689+0000 7f6579200640 1 -- 172.21.15.31:0/3117342106 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f65580a2c10 msgr2=0x7f65580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:21.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.689+0000 7f6579200640 1 --2- 172.21.15.31:0/3117342106 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f65580a2c10 0x7f65580a50d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6560031610 tx=0x7f6560005fd0 comp rx=0 tx=0).stop 2024-04-25T01:24:21.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.689+0000 7f6579200640 1 -- 172.21.15.31:0/3117342106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6574101790 msgr2=0x7f6574110b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:21.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.689+0000 7f6579200640 1 --2- 172.21.15.31:0/3117342106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6574101790 0x7f6574110b90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f656400e9c0 tx=0x7f656400ee80 comp rx=0 tx=0).stop 2024-04-25T01:24:21.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.689+0000 7f6579200640 1 -- 172.21.15.31:0/3117342106 shutdown_connections 2024-04-25T01:24:21.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.689+0000 7f6579200640 1 --2- 172.21.15.31:0/3117342106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6574103340 0x7f6574118140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:21.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.689+0000 7f6579200640 1 --2- 172.21.15.31:0/3117342106 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f65580a2c10 0x7f65580a50d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:21.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.689+0000 7f6579200640 1 --2- 172.21.15.31:0/3117342106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6574102980 0x7f65741110d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:21.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.689+0000 7f6579200640 1 --2- 172.21.15.31:0/3117342106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6574101790 0x7f6574110b90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:21.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.689+0000 7f6579200640 1 -- 172.21.15.31:0/3117342106 >> 172.21.15.31:0/3117342106 conn(0x7f65740fcd20 msgr2=0x7f6574105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:24:21.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.689+0000 7f6579200640 1 -- 172.21.15.31:0/3117342106 shutdown_connections 2024-04-25T01:24:21.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.689+0000 7f6579200640 1 -- 172.21.15.31:0/3117342106 wait complete. 2024-04-25T01:24:21.687 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T01:24:21.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.887+0000 7f21eec00640 1 -- 172.21.15.31:0/1531590508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21e8105370 msgr2=0x7f21e810bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:21.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.887+0000 7f21eec00640 1 --2- 172.21.15.31:0/1531590508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21e8105370 0x7f21e810bc00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009be0 tx=0x7f21e002a6c0 comp rx=0 tx=0).stop 2024-04-25T01:24:21.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.887+0000 7f21eec00640 1 -- 172.21.15.31:0/1531590508 shutdown_connections 2024-04-25T01:24:21.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.887+0000 7f21eec00640 1 --2- 172.21.15.31:0/1531590508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21e8105370 0x7f21e810bc00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:21.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.887+0000 7f21eec00640 1 --2- 172.21.15.31:0/1531590508 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21e81049b0 0x7f21e8104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:21.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.887+0000 7f21eec00640 1 --2- 172.21.15.31:0/1531590508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e81037c0 0x7f21e8103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:21.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.887+0000 7f21eec00640 1 -- 172.21.15.31:0/1531590508 >> 172.21.15.31:0/1531590508 conn(0x7f21e80fed50 msgr2=0x7f21e8101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:24:21.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.887+0000 7f21eec00640 1 -- 172.21.15.31:0/1531590508 shutdown_connections 2024-04-25T01:24:21.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.887+0000 7f21eec00640 1 -- 172.21.15.31:0/1531590508 wait complete. 2024-04-25T01:24:21.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.888+0000 7f21eec00640 1 Processor -- start 2024-04-25T01:24:21.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.888+0000 7f21eec00640 1 -- start start 2024-04-25T01:24:21.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.888+0000 7f21eec00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21e81037c0 0x7f21e8199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:21.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.888+0000 7f21eec00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21e81049b0 0x7f21e8199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:21.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.888+0000 7f21eec00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e8105370 0x7f21e81a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:21.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.888+0000 7f21eec00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f21e810f440 con 0x7f21e81037c0 2024-04-25T01:24:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.888+0000 7f21eec00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f21e810f2c0 con 0x7f21e8105370 2024-04-25T01:24:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.888+0000 7f21eec00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f21e810f5c0 con 0x7f21e81049b0 2024-04-25T01:24:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.889+0000 7f21ee200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e8105370 0x7f21e81a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.889+0000 7f21ece00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21e81049b0 0x7f21e8199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.889+0000 7f21ee200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e8105370 0x7f21e81a1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:59522/0 (socket says 172.21.15.31:59522) 2024-04-25T01:24:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.889+0000 7f21ee200640 1 -- 172.21.15.31:0/3676795655 learned_addr learned my addr 172.21.15.31:0/3676795655 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:24:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.889+0000 7f21ed800640 1 --2- 172.21.15.31:0/3676795655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21e81037c0 0x7f21e8199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.889+0000 7f21ee200640 1 -- 172.21.15.31:0/3676795655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21e81049b0 msgr2=0x7f21e8199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.889+0000 7f21ee200640 1 --2- 172.21.15.31:0/3676795655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21e81049b0 0x7f21e8199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.889+0000 7f21ee200640 1 -- 172.21.15.31:0/3676795655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21e81037c0 msgr2=0x7f21e8199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.889+0000 7f21ee200640 1 --2- 172.21.15.31:0/3676795655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21e81037c0 0x7f21e8199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.889+0000 7f21ee200640 1 -- 172.21.15.31:0/3676795655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e81a1680 con 0x7f21e8105370 2024-04-25T01:24:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.889+0000 7f21ece00640 1 --2- 172.21.15.31:0/3676795655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21e81049b0 0x7f21e8199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:24:21.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.890+0000 7f21ee200640 1 --2- 172.21.15.31:0/3676795655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e8105370 0x7f21e81a1010 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009b30 tx=0x7f21e00305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:24:21.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.890+0000 7f21e6000640 1 -- 172.21.15.31:0/3676795655 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0041020 con 0x7f21e8105370 2024-04-25T01:24:21.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.890+0000 7f21e6000640 1 -- 172.21.15.31:0/3676795655 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21e0030740 con 0x7f21e8105370 2024-04-25T01:24:21.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.890+0000 7f21eec00640 1 -- 172.21.15.31:0/3676795655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e81a1910 con 0x7f21e8105370 2024-04-25T01:24:21.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.890+0000 7f21e6000640 1 -- 172.21.15.31:0/3676795655 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0036040 con 0x7f21e8105370 2024-04-25T01:24:21.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.890+0000 7f21eec00640 1 -- 172.21.15.31:0/3676795655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e81a1e50 con 0x7f21e8105370 2024-04-25T01:24:21.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.891+0000 7f21eec00640 1 -- 172.21.15.31:0/3676795655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21e810aae0 con 0x7f21e8105370 2024-04-25T01:24:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.893+0000 7f21e6000640 1 -- 172.21.15.31:0/3676795655 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f21e0044020 con 0x7f21e8105370 2024-04-25T01:24:21.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.894+0000 7f21e6000640 1 --2- 172.21.15.31:0/3676795655 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f21cc0a2c10 0x7f21cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:21.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.894+0000 7f21e6000640 1 -- 172.21.15.31:0/3676795655 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f21e00efcb0 con 0x7f21e8105370 2024-04-25T01:24:21.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.894+0000 7f21ed800640 1 --2- 172.21.15.31:0/3676795655 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f21cc0a2c10 0x7f21cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:21.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.895+0000 7f21ed800640 1 --2- 172.21.15.31:0/3676795655 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f21cc0a2c10 0x7f21cc0a50d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f21e8104810 tx=0x7f21d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:24:21.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:21.898+0000 7f21e6000640 1 -- 172.21.15.31:0/3676795655 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f21e004b020 con 0x7f21e8105370 2024-04-25T01:24:21.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:21 smithi169 ceph-mon[32759]: pgmap v643: 97 pgs: 97 active+clean; 5.0 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 95 KiB/s rd, 22 MiB/s wr, 4.39k op/s 2024-04-25T01:24:21.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:21 smithi169 ceph-mon[32759]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:24:21.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:21 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2459327139' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:24:21.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:21 smithi040 ceph-mon[32518]: pgmap v643: 97 pgs: 97 active+clean; 5.0 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 95 KiB/s rd, 22 MiB/s wr, 4.39k op/s 2024-04-25T01:24:21.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:21 smithi040 ceph-mon[32518]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:24:21.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:21 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2459327139' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:24:21.999 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:21 smithi031 ceph-mon[28053]: pgmap v643: 97 pgs: 97 active+clean; 5.0 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 95 KiB/s rd, 22 MiB/s wr, 4.39k op/s 2024-04-25T01:24:22.000 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:21 smithi031 ceph-mon[28053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:24:22.000 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:21 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2459327139' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:24:22.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:22.145+0000 7f21eec00640 1 -- 172.21.15.31:0/3676795655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f21e8062340 con 0x7f21e8105370 2024-04-25T01:24:22.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:22.156+0000 7f21e6000640 1 -- 172.21.15.31:0/3676795655 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+740369 (secure 0 0 0) 0x7f21e00bcc00 con 0x7f21e8105370 2024-04-25T01:24:22.149 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:24:22.149 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T01:24:22.150 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 18, 2024-04-25T01:24:22.150 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.150 INFO:teuthology.orchestra.run.smithi031.stdout: "active_gid": 14152, 2024-04-25T01:24:22.150 INFO:teuthology.orchestra.run.smithi031.stdout: "active_name": "x", 2024-04-25T01:24:22.150 INFO:teuthology.orchestra.run.smithi031.stdout: "active_addrs": { 2024-04-25T01:24:22.150 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-04-25T01:24:22.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.151 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-04-25T01:24:22.151 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-04-25T01:24:22.151 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874594057 2024-04-25T01:24:22.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.151 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-04-25T01:24:22.151 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-04-25T01:24:22.151 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2874594057 2024-04-25T01:24:22.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.151 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T01:24:22.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.151 INFO:teuthology.orchestra.run.smithi031.stdout: "active_addr": "172.21.15.31:6801/2874594057", 2024-04-25T01:24:22.152 INFO:teuthology.orchestra.run.smithi031.stdout: "active_change": "2024-04-25T01:05:31.932435+0000", 2024-04-25T01:24:22.152 INFO:teuthology.orchestra.run.smithi031.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T01:24:22.152 INFO:teuthology.orchestra.run.smithi031.stdout: "available": true, 2024-04-25T01:24:22.152 INFO:teuthology.orchestra.run.smithi031.stdout: "standbys": [ 2024-04-25T01:24:22.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.152 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 24109, 2024-04-25T01:24:22.152 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "z", 2024-04-25T01:24:22.152 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_features": 4540701547738038271, 2024-04-25T01:24:22.152 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2024-04-25T01:24:22.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.152 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2024-04-25T01:24:22.152 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.152 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.153 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.153 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T01:24:22.153 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T01:24:22.153 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.153 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.153 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.153 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T01:24:22.153 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.153 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.153 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.153 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T01:24:22.153 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.153 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.154 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.154 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.154 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.154 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.154 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.154 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.154 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.154 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.154 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.155 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.155 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.155 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.155 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.155 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.155 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.155 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.155 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.155 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.155 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.155 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.155 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.156 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.156 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.156 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.156 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.156 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.156 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.156 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.156 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.156 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.156 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.156 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.156 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.156 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.159 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.159 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.159 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.159 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2024-04-25T01:24:22.159 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2024-04-25T01:24:22.159 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.159 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.159 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.159 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.159 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.159 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.159 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2024-04-25T01:24:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2024-04-25T01:24:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2024-04-25T01:24:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2024-04-25T01:24:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2024-04-25T01:24:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2024-04-25T01:24:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2024-04-25T01:24:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2024-04-25T01:24:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2024-04-25T01:24:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2024-04-25T01:24:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2024-04-25T01:24:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2024-04-25T01:24:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2024-04-25T01:24:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2024-04-25T01:24:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2024-04-25T01:24:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2024-04-25T01:24:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2024-04-25T01:24:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2024-04-25T01:24:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2024-04-25T01:24:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2024-04-25T01:24:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T01:24:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2024-04-25T01:24:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2024-04-25T01:24:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2024-04-25T01:24:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2024-04-25T01:24:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2024-04-25T01:24:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2024-04-25T01:24:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T01:24:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2024-04-25T01:24:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2024-04-25T01:24:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2024-04-25T01:24:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T01:24:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T01:24:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2024-04-25T01:24:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2024-04-25T01:24:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T01:24:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T01:24:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-04-25T01:24:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-04-25T01:24:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T01:24:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T01:24:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2024-04-25T01:24:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2024-04-25T01:24:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T01:24:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2024-04-25T01:24:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T01:24:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2024-04-25T01:24:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T01:24:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2024-04-25T01:24:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2024-04-25T01:24:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T01:24:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T01:24:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T01:24:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2024-04-25T01:24:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2024-04-25T01:24:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T01:24:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2024-04-25T01:24:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2024-04-25T01:24:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2024-04-25T01:24:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2024-04-25T01:24:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T01:24:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2024-04-25T01:24:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2024-04-25T01:24:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2024-04-25T01:24:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T01:24:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T01:24:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2024-04-25T01:24:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2024-04-25T01:24:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T01:24:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T01:24:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-04-25T01:24:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-04-25T01:24:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T01:24:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T01:24:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2024-04-25T01:24:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2024-04-25T01:24:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T01:24:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T01:24:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T01:24:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-04-25T01:24:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-04-25T01:24:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2024-04-25T01:24:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2024-04-25T01:24:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2024-04-25T01:24:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "read", 2024-04-25T01:24:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap", 2024-04-25T01:24:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap-read" 2024-04-25T01:24:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2024-04-25T01:24:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2024-04-25T01:24:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2024-04-25T01:24:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T01:24:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T01:24:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T01:24:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T01:24:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T01:24:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2024-04-25T01:24:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2024-04-25T01:24:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T01:24:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T01:24:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T01:24:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T01:24:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2024-04-25T01:24:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2024-04-25T01:24:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T01:24:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T01:24:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T01:24:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2024-04-25T01:24:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2024-04-25T01:24:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2024-04-25T01:24:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T01:24:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-04-25T01:24:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T01:24:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2024-04-25T01:24:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2024-04-25T01:24:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2024-04-25T01:24:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T01:24:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2024-04-25T01:24:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2024-04-25T01:24:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2024-04-25T01:24:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T01:24:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2024-04-25T01:24:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2024-04-25T01:24:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T01:24:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T01:24:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2024-04-25T01:24:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2024-04-25T01:24:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T01:24:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T01:24:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2024-04-25T01:24:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T01:24:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T01:24:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2024-04-25T01:24:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T01:24:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2024-04-25T01:24:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2024-04-25T01:24:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2024-04-25T01:24:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2024-04-25T01:24:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2024-04-25T01:24:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T01:24:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2024-04-25T01:24:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2024-04-25T01:24:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T01:24:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2024-04-25T01:24:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2024-04-25T01:24:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T01:24:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2024-04-25T01:24:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2024-04-25T01:24:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T01:24:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2024-04-25T01:24:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2024-04-25T01:24:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T01:24:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T01:24:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2024-04-25T01:24:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2024-04-25T01:24:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T01:24:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2024-04-25T01:24:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2024-04-25T01:24:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2024-04-25T01:24:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T01:24:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2024-04-25T01:24:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2024-04-25T01:24:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2024-04-25T01:24:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T01:24:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T01:24:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2024-04-25T01:24:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2024-04-25T01:24:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T01:24:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2024-04-25T01:24:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2024-04-25T01:24:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2024-04-25T01:24:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T01:24:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2024-04-25T01:24:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2024-04-25T01:24:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2024-04-25T01:24:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T01:24:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2024-04-25T01:24:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2024-04-25T01:24:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2024-04-25T01:24:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T01:24:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2024-04-25T01:24:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2024-04-25T01:24:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2024-04-25T01:24:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T01:24:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2024-04-25T01:24:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2024-04-25T01:24:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2024-04-25T01:24:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T01:24:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2024-04-25T01:24:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2024-04-25T01:24:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2024-04-25T01:24:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T01:24:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T01:24:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2024-04-25T01:24:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2024-04-25T01:24:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T01:24:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2024-04-25T01:24:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2024-04-25T01:24:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2024-04-25T01:24:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T01:24:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T01:24:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2024-04-25T01:24:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2024-04-25T01:24:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T01:24:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2024-04-25T01:24:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba": { 2024-04-25T01:24:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba", 2024-04-25T01:24:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T01:24:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB container image", 2024-04-25T01:24:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2024-04-25T01:24:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2024-04-25T01:24:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T01:24:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2024-04-25T01:24:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2024-04-25T01:24:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2024-04-25T01:24:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T01:24:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2024-04-25T01:24:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2024-04-25T01:24:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T01:24:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T01:24:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2024-04-25T01:24:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T01:24:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2024-04-25T01:24:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T01:24:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2024-04-25T01:24:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2024-04-25T01:24:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2024-04-25T01:24:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T01:24:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2024-04-25T01:24:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2024-04-25T01:24:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2024-04-25T01:24:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2024-04-25T01:24:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2024-04-25T01:24:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2024-04-25T01:24:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T01:24:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2024-04-25T01:24:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2024-04-25T01:24:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T01:24:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2024-04-25T01:24:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "grafana_dashboards_path": { 2024-04-25T01:24:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "grafana_dashboards_path", 2024-04-25T01:24:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T01:24:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T01:24:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2024-04-25T01:24:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2024-04-25T01:24:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T01:24:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2024-04-25T01:24:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "hw_monitoring": { 2024-04-25T01:24:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hw_monitoring", 2024-04-25T01:24:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T01:24:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2024-04-25T01:24:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2024-04-25T01:24:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T01:24:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2024-04-25T01:24:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2024-04-25T01:24:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T01:24:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T01:24:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T01:24:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T01:24:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T01:24:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T01:24:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T01:24:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2024-04-25T01:24:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T01:24:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2024-04-25T01:24:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2024-04-25T01:24:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T01:24:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2024-04-25T01:24:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2024-04-25T01:24:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2024-04-25T01:24:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T01:24:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T01:24:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2024-04-25T01:24:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2024-04-25T01:24:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2024-04-25T01:24:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-04-25T01:24:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-04-25T01:24:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2024-04-25T01:24:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2024-04-25T01:24:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2024-04-25T01:24:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T01:24:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "oob_default_addr": { 2024-04-25T01:24:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "oob_default_addr", 2024-04-25T01:24:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "169.254.1.1", 2024-04-25T01:24:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T01:24:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2024-04-25T01:24:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2024-04-25T01:24:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T01:24:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T01:24:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2024-04-25T01:24:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2024-04-25T01:24:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T01:24:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2024-04-25T01:24:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2024-04-25T01:24:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T01:24:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2024-04-25T01:24:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2024-04-25T01:24:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T01:24:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2024-04-25T01:24:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2024-04-25T01:24:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T01:24:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2024-04-25T01:24:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2024-04-25T01:24:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T01:24:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2024-04-25T01:24:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2024-04-25T01:24:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2024-04-25T01:24:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2024-04-25T01:24:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2024-04-25T01:24:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2024-04-25T01:24:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T01:24:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2024-04-25T01:24:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2024-04-25T01:24:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T01:24:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2024-04-25T01:24:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2024-04-25T01:24:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T01:24:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2024-04-25T01:24:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2024-04-25T01:24:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T01:24:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2024-04-25T01:24:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2024-04-25T01:24:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T01:24:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2024-04-25T01:24:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2024-04-25T01:24:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T01:24:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2024-04-25T01:24:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2024-04-25T01:24:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2024-04-25T01:24:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2024-04-25T01:24:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T01:24:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2024-04-25T01:24:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2024-04-25T01:24:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2024-04-25T01:24:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.258 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T01:24:22.258 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.258 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.258 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.258 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.258 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.258 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2024-04-25T01:24:22.258 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.258 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.258 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.258 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T01:24:22.258 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T01:24:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T01:24:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T01:24:22.259 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T01:24:22.260 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.260 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.260 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.260 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.260 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.260 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.260 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.260 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.260 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.260 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.260 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.260 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T01:24:22.260 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T01:24:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2024-04-25T01:24:22.261 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T01:24:22.262 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.262 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.262 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.262 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.262 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.262 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.262 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.262 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.262 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.262 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.262 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.262 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T01:24:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T01:24:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.263 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T01:24:22.264 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T01:24:22.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.264 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.264 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.264 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.264 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.264 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.264 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.264 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.264 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.264 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.265 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T01:24:22.265 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T01:24:22.265 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.265 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.265 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.265 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.265 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.265 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.265 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.265 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.265 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.265 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.266 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T01:24:22.266 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T01:24:22.266 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.266 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.266 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.266 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.266 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.266 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.266 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.266 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.266 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.266 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T01:24:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T01:24:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.267 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T01:24:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T01:24:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.268 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T01:24:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T01:24:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.270 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T01:24:22.270 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T01:24:22.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.270 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.270 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.270 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.270 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.270 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.270 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.270 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.270 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.270 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.270 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T01:24:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T01:24:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T01:24:22.271 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T01:24:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T01:24:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T01:24:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T01:24:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T01:24:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T01:24:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.273 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2024-04-25T01:24:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2024-04-25T01:24:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T01:24:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T01:24:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T01:24:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T01:24:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T01:24:22.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T01:24:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T01:24:22.277 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.278 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.278 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.278 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.278 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.278 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.278 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.278 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.278 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.278 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.278 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T01:24:22.278 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T01:24:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T01:24:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T01:24:22.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T01:24:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T01:24:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T01:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T01:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T01:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T01:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T01:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T01:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T01:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T01:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T01:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T01:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T01:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T01:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T01:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T01:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T01:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T01:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T01:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T01:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T01:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T01:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T01:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T01:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T01:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T01:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T01:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T01:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2024-04-25T01:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T01:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T01:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2024-04-25T01:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T01:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T01:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T01:24:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T01:24:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T01:24:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T01:24:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T01:24:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T01:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T01:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T01:24:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T01:24:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T01:24:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T01:24:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T01:24:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T01:24:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T01:24:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T01:24:22.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T01:24:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2024-04-25T01:24:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2024-04-25T01:24:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.301 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2024-04-25T01:24:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2024-04-25T01:24:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.302 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.303 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.303 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.303 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.303 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.303 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.303 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.303 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.303 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.303 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.303 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2024-04-25T01:24:22.303 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2024-04-25T01:24:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2024-04-25T01:24:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2024-04-25T01:24:22.304 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2024-04-25T01:24:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2024-04-25T01:24:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-04-25T01:24:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-04-25T01:24:22.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.306 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.307 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.309 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.310 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.310 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.310 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.310 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.310 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.310 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.310 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.310 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.310 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.310 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.310 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.310 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.310 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2024-04-25T01:24:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2024-04-25T01:24:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.311 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2024-04-25T01:24:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2024-04-25T01:24:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T01:24:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.312 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T01:24:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T01:24:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-04-25T01:24:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.313 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T01:24:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T01:24:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2024-04-25T01:24:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.314 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-04-25T01:24:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-04-25T01:24:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.315 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2024-04-25T01:24:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2024-04-25T01:24:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2024-04-25T01:24:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-04-25T01:24:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-04-25T01:24:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2024-04-25T01:24:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2024-04-25T01:24:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.317 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-04-25T01:24:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-04-25T01:24:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-04-25T01:24:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-04-25T01:24:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-04-25T01:24:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2024-04-25T01:24:22.318 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2024-04-25T01:24:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.320 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-04-25T01:24:22.320 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.320 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.320 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.320 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2024-04-25T01:24:22.320 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2024-04-25T01:24:22.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.320 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.320 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.320 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.320 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.320 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.320 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2024-04-25T01:24:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.321 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.322 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2024-04-25T01:24:22.325 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2024-04-25T01:24:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2024-04-25T01:24:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T01:24:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2024-04-25T01:24:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2024-04-25T01:24:22.326 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2024-04-25T01:24:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T01:24:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2024-04-25T01:24:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2024-04-25T01:24:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.327 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2024-04-25T01:24:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2024-04-25T01:24:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2024-04-25T01:24:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2024-04-25T01:24:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-04-25T01:24:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T01:24:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2024-04-25T01:24:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2024-04-25T01:24:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.329 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T01:24:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T01:24:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T01:24:22.330 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T01:24:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T01:24:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2024-04-25T01:24:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2024-04-25T01:24:22.331 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2024-04-25T01:24:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T01:24:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2024-04-25T01:24:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.333 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.334 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.334 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.334 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.334 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.334 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.334 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.334 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.334 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.334 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.334 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.334 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.334 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.335 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.336 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.337 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2024-04-25T01:24:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2024-04-25T01:24:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-04-25T01:24:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2024-04-25T01:24:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2024-04-25T01:24:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2024-04-25T01:24:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T01:24:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T01:24:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T01:24:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2024-04-25T01:24:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2024-04-25T01:24:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2024-04-25T01:24:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2024-04-25T01:24:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2024-04-25T01:24:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2024-04-25T01:24:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T01:24:22.341 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2024-04-25T01:24:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2024-04-25T01:24:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2024-04-25T01:24:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.342 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T01:24:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2024-04-25T01:24:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2024-04-25T01:24:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.343 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2024-04-25T01:24:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T01:24:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T01:24:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2024-04-25T01:24:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2024-04-25T01:24:22.344 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T01:24:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.345 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.348 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.349 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2024-04-25T01:24:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2024-04-25T01:24:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.350 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2024-04-25T01:24:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2024-04-25T01:24:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2024-04-25T01:24:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2024-04-25T01:24:22.351 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2024-04-25T01:24:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-04-25T01:24:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-04-25T01:24:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2024-04-25T01:24:22.352 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T01:24:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2024-04-25T01:24:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2024-04-25T01:24:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T01:24:22.353 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T01:24:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2024-04-25T01:24:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T01:24:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2024-04-25T01:24:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2024-04-25T01:24:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2024-04-25T01:24:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2024-04-25T01:24:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2024-04-25T01:24:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.355 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2024-04-25T01:24:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T01:24:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2024-04-25T01:24:22.356 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.357 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.358 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.359 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.360 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2024-04-25T01:24:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.363 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.364 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.365 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.366 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2024-04-25T01:24:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.367 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2024-04-25T01:24:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2024-04-25T01:24:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-04-25T01:24:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T01:24:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2024-04-25T01:24:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2024-04-25T01:24:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T01:24:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2024-04-25T01:24:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T01:24:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.370 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.371 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.373 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2024-04-25T01:24:22.374 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2024-04-25T01:24:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2024-04-25T01:24:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2024-04-25T01:24:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2024-04-25T01:24:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T01:24:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.375 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.376 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.377 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2024-04-25T01:24:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2024-04-25T01:24:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.380 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2024-04-25T01:24:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T01:24:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2024-04-25T01:24:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2024-04-25T01:24:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-04-25T01:24:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.381 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2024-04-25T01:24:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2024-04-25T01:24:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2024-04-25T01:24:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2024-04-25T01:24:22.382 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T01:24:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2024-04-25T01:24:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2024-04-25T01:24:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.383 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2024-04-25T01:24:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2024-04-25T01:24:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2024-04-25T01:24:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.384 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2024-04-25T01:24:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T01:24:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T01:24:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2024-04-25T01:24:22.385 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.386 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.387 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.388 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.389 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.390 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2024-04-25T01:24:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.391 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.392 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.393 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.394 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.395 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2024-04-25T01:24:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.397 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.398 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.399 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.400 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.401 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-04-25T01:24:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.402 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2024-04-25T01:24:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2024-04-25T01:24:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T01:24:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.404 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.405 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2024-04-25T01:24:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-04-25T01:24:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-04-25T01:24:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2024-04-25T01:24:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2024-04-25T01:24:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2024-04-25T01:24:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2024-04-25T01:24:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.414 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2024-04-25T01:24:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2024-04-25T01:24:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T01:24:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T01:24:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T01:24:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2024-04-25T01:24:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2024-04-25T01:24:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T01:24:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-04-25T01:24:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2024-04-25T01:24:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2024-04-25T01:24:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T01:24:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2024-04-25T01:24:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2024-04-25T01:24:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2024-04-25T01:24:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T01:24:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-04-25T01:24:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-04-25T01:24:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2024-04-25T01:24:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2024-04-25T01:24:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2024-04-25T01:24:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2024-04-25T01:24:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T01:24:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T01:24:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T01:24:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2024-04-25T01:24:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2024-04-25T01:24:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2024-04-25T01:24:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2024-04-25T01:24:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2024-04-25T01:24:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2024-04-25T01:24:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T01:24:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T01:24:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2024-04-25T01:24:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2024-04-25T01:24:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T01:24:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T01:24:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2024-04-25T01:24:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2024-04-25T01:24:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2024-04-25T01:24:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T01:24:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2024-04-25T01:24:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T01:24:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T01:24:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-04-25T01:24:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T01:24:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T01:24:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T01:24:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2024-04-25T01:24:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T01:24:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2024-04-25T01:24:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2024-04-25T01:24:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2024-04-25T01:24:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-04-25T01:24:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-04-25T01:24:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2024-04-25T01:24:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2024-04-25T01:24:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2024-04-25T01:24:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-04-25T01:24:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-04-25T01:24:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-04-25T01:24:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-04-25T01:24:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-04-25T01:24:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-04-25T01:24:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2024-04-25T01:24:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2024-04-25T01:24:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T01:24:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2024-04-25T01:24:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T01:24:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2024-04-25T01:24:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2024-04-25T01:24:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2024-04-25T01:24:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2024-04-25T01:24:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2024-04-25T01:24:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-04-25T01:24:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-04-25T01:24:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T01:24:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T01:24:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T01:24:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T01:24:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2024-04-25T01:24:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2024-04-25T01:24:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2024-04-25T01:24:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2024-04-25T01:24:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2024-04-25T01:24:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T01:24:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2024-04-25T01:24:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2024-04-25T01:24:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2024-04-25T01:24:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2024-04-25T01:24:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2024-04-25T01:24:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2024-04-25T01:24:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2024-04-25T01:24:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2024-04-25T01:24:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2024-04-25T01:24:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2024-04-25T01:24:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2024-04-25T01:24:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2024-04-25T01:24:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T01:24:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2024-04-25T01:24:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2024-04-25T01:24:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2024-04-25T01:24:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2024-04-25T01:24:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2024-04-25T01:24:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2024-04-25T01:24:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2024-04-25T01:24:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2024-04-25T01:24:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T01:24:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2024-04-25T01:24:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2024-04-25T01:24:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2024-04-25T01:24:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2024-04-25T01:24:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2024-04-25T01:24:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2024-04-25T01:24:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2024-04-25T01:24:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-04-25T01:24:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2024-04-25T01:24:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2024-04-25T01:24:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T01:24:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2024-04-25T01:24:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2024-04-25T01:24:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2024-04-25T01:24:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2024-04-25T01:24:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2024-04-25T01:24:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2024-04-25T01:24:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2024-04-25T01:24:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2024-04-25T01:24:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T01:24:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T01:24:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T01:24:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2024-04-25T01:24:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2024-04-25T01:24:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2024-04-25T01:24:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2024-04-25T01:24:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T01:24:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2024-04-25T01:24:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2024-04-25T01:24:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T01:24:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2024-04-25T01:24:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2024-04-25T01:24:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T01:24:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2024-04-25T01:24:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2024-04-25T01:24:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T01:24:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2024-04-25T01:24:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2024-04-25T01:24:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T01:24:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2024-04-25T01:24:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2024-04-25T01:24:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2024-04-25T01:24:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2024-04-25T01:24:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2024-04-25T01:24:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2024-04-25T01:24:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T01:24:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-04-25T01:24:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-04-25T01:24:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T01:24:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T01:24:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2024-04-25T01:24:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2024-04-25T01:24:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2024-04-25T01:24:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2024-04-25T01:24:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2024-04-25T01:24:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2024-04-25T01:24:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2024-04-25T01:24:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2024-04-25T01:24:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2024-04-25T01:24:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.545 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2024-04-25T01:24:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2024-04-25T01:24:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2024-04-25T01:24:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2024-04-25T01:24:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.547 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.548 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.548 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.548 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.548 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.548 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.548 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.548 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.548 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2024-04-25T01:24:22.548 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2024-04-25T01:24:22.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.548 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T01:24:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2024-04-25T01:24:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.549 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.550 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.550 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.550 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.550 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.550 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.550 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.550 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.550 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.550 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.550 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.550 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.550 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.551 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.552 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.552 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.552 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.552 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.552 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.552 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.552 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.552 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.552 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.552 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.552 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.552 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.553 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.554 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.554 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.554 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.554 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.554 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.554 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.554 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.554 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.554 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.554 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.554 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-04-25T01:24:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.555 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.556 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.556 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.556 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.556 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.556 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.556 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.556 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.556 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.556 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.556 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.556 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.556 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.556 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.557 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2024-04-25T01:24:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2024-04-25T01:24:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.560 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2024-04-25T01:24:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T01:24:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2024-04-25T01:24:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2024-04-25T01:24:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.561 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.562 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.562 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.562 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.562 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2024-04-25T01:24:22.562 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.562 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.562 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.562 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2024-04-25T01:24:22.562 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2024-04-25T01:24:22.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.562 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.563 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T01:24:22.563 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.563 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.563 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.563 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T01:24:22.563 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.563 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.563 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_no_wait": { 2024-04-25T01:24:22.563 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T01:24:22.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.563 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T01:24:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.564 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2024-04-25T01:24:22.565 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.565 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.565 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.565 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2024-04-25T01:24:22.565 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2024-04-25T01:24:22.565 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T01:24:22.565 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.565 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.565 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2024-04-25T01:24:22.565 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.565 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.565 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.565 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.565 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2024-04-25T01:24:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2024-04-25T01:24:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.566 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T01:24:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T01:24:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T01:24:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.567 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.568 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.570 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.570 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.570 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.570 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.570 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.570 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.570 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.570 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.570 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.570 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.570 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.570 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.571 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.572 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.572 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.572 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.572 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.572 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.572 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.572 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.572 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.572 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.572 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.572 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.572 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.572 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2024-04-25T01:24:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2024-04-25T01:24:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.573 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2024-04-25T01:24:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2024-04-25T01:24:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2024-04-25T01:24:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.574 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2024-04-25T01:24:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2024-04-25T01:24:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T01:24:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.575 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T01:24:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 24110, 2024-04-25T01:24:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "y", 2024-04-25T01:24:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_features": 4540701547738038271, 2024-04-25T01:24:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2024-04-25T01:24:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.576 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2024-04-25T01:24:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T01:24:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T01:24:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T01:24:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T01:24:22.577 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.578 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.578 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.578 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.578 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.578 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.578 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.578 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.578 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.578 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.578 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.578 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.578 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.578 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.579 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.580 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.581 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.582 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.582 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.582 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.582 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.582 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.582 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.582 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.582 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.582 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.582 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.582 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.582 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2024-04-25T01:24:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2024-04-25T01:24:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.583 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2024-04-25T01:24:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2024-04-25T01:24:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2024-04-25T01:24:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2024-04-25T01:24:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2024-04-25T01:24:22.584 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2024-04-25T01:24:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2024-04-25T01:24:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2024-04-25T01:24:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.585 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2024-04-25T01:24:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2024-04-25T01:24:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2024-04-25T01:24:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2024-04-25T01:24:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2024-04-25T01:24:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2024-04-25T01:24:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2024-04-25T01:24:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2024-04-25T01:24:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2024-04-25T01:24:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2024-04-25T01:24:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.588 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2024-04-25T01:24:22.589 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2024-04-25T01:24:22.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.589 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.589 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.589 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.589 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.589 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.589 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.589 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T01:24:22.589 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.589 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.589 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.589 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2024-04-25T01:24:22.589 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2024-04-25T01:24:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2024-04-25T01:24:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.591 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2024-04-25T01:24:22.591 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.591 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.591 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.591 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2024-04-25T01:24:22.591 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2024-04-25T01:24:22.591 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.591 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.591 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.591 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.591 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.591 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T01:24:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2024-04-25T01:24:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2024-04-25T01:24:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2024-04-25T01:24:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.592 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.593 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.593 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T01:24:22.593 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T01:24:22.593 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.593 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.593 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2024-04-25T01:24:22.593 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2024-04-25T01:24:22.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T01:24:22.593 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.593 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.593 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T01:24:22.593 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-04-25T01:24:22.593 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-04-25T01:24:22.594 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.594 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T01:24:22.594 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T01:24:22.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.594 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.594 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2024-04-25T01:24:22.594 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2024-04-25T01:24:22.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T01:24:22.594 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.594 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.594 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2024-04-25T01:24:22.594 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T01:24:22.595 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2024-04-25T01:24:22.595 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.595 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T01:24:22.595 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.595 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.595 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.595 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2024-04-25T01:24:22.595 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2024-04-25T01:24:22.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.595 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.595 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T01:24:22.596 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.596 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.596 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.596 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T01:24:22.596 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T01:24:22.596 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.596 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.596 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2024-04-25T01:24:22.596 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2024-04-25T01:24:22.596 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T01:24:22.596 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.596 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2024-04-25T01:24:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2024-04-25T01:24:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2024-04-25T01:24:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2024-04-25T01:24:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T01:24:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2024-04-25T01:24:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2024-04-25T01:24:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.597 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2024-04-25T01:24:22.598 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.598 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.598 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.598 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T01:24:22.598 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T01:24:22.598 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.598 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.598 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2024-04-25T01:24:22.598 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2024-04-25T01:24:22.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T01:24:22.598 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.598 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.599 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T01:24:22.599 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-04-25T01:24:22.599 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-04-25T01:24:22.599 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.599 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T01:24:22.599 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T01:24:22.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.599 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.599 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.599 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.599 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.600 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.601 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.602 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.602 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.602 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.602 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.602 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.602 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.602 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.602 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.602 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.602 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.602 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.603 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.604 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.604 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.604 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.604 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.604 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.604 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.604 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.604 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.604 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.604 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2024-04-25T01:24:22.604 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2024-04-25T01:24:22.604 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T01:24:22.604 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.605 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.605 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T01:24:22.605 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.605 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.605 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.605 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T01:24:22.605 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.605 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.605 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.605 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-04-25T01:24:22.605 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-04-25T01:24:22.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.606 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.606 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.606 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2024-04-25T01:24:22.606 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.606 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.606 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.606 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2024-04-25T01:24:22.606 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2024-04-25T01:24:22.606 INFO:teuthology.orchestra.run.smithi031.stdout: "read", 2024-04-25T01:24:22.606 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap", 2024-04-25T01:24:22.606 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap-read" 2024-04-25T01:24:22.606 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.606 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2024-04-25T01:24:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2024-04-25T01:24:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2024-04-25T01:24:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.607 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T01:24:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T01:24:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T01:24:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T01:24:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T01:24:22.608 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2024-04-25T01:24:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2024-04-25T01:24:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T01:24:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T01:24:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T01:24:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T01:24:22.609 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2024-04-25T01:24:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2024-04-25T01:24:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T01:24:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T01:24:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T01:24:22.610 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2024-04-25T01:24:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2024-04-25T01:24:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2024-04-25T01:24:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T01:24:22.611 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-04-25T01:24:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T01:24:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2024-04-25T01:24:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2024-04-25T01:24:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.612 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2024-04-25T01:24:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T01:24:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2024-04-25T01:24:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2024-04-25T01:24:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.613 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.614 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.614 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2024-04-25T01:24:22.614 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.614 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.614 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.614 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T01:24:22.614 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.614 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.614 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.614 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2024-04-25T01:24:22.614 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2024-04-25T01:24:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T01:24:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T01:24:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2024-04-25T01:24:22.615 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2024-04-25T01:24:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T01:24:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T01:24:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2024-04-25T01:24:22.616 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T01:24:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T01:24:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2024-04-25T01:24:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T01:24:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2024-04-25T01:24:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2024-04-25T01:24:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.617 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2024-04-25T01:24:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2024-04-25T01:24:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2024-04-25T01:24:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T01:24:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2024-04-25T01:24:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2024-04-25T01:24:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T01:24:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.619 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2024-04-25T01:24:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2024-04-25T01:24:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T01:24:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2024-04-25T01:24:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2024-04-25T01:24:22.620 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T01:24:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2024-04-25T01:24:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2024-04-25T01:24:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.621 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T01:24:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T01:24:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2024-04-25T01:24:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2024-04-25T01:24:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.622 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T01:24:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2024-04-25T01:24:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2024-04-25T01:24:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2024-04-25T01:24:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.623 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T01:24:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2024-04-25T01:24:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2024-04-25T01:24:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2024-04-25T01:24:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.624 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T01:24:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T01:24:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2024-04-25T01:24:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2024-04-25T01:24:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T01:24:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.625 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.626 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.626 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2024-04-25T01:24:22.626 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.626 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.626 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.626 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2024-04-25T01:24:22.626 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2024-04-25T01:24:22.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.626 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.626 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.626 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T01:24:22.626 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.627 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.627 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.627 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2024-04-25T01:24:22.627 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.627 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.627 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.627 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2024-04-25T01:24:22.627 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2024-04-25T01:24:22.627 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.627 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.627 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.627 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T01:24:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2024-04-25T01:24:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2024-04-25T01:24:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2024-04-25T01:24:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T01:24:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2024-04-25T01:24:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2024-04-25T01:24:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2024-04-25T01:24:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T01:24:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2024-04-25T01:24:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2024-04-25T01:24:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2024-04-25T01:24:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T01:24:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2024-04-25T01:24:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2024-04-25T01:24:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2024-04-25T01:24:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.631 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T01:24:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T01:24:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2024-04-25T01:24:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2024-04-25T01:24:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T01:24:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.632 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2024-04-25T01:24:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2024-04-25T01:24:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2024-04-25T01:24:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T01:24:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.633 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.634 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T01:24:22.634 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.634 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.634 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.634 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2024-04-25T01:24:22.634 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2024-04-25T01:24:22.634 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.634 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.634 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.634 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T01:24:22.634 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.634 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.634 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2024-04-25T01:24:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba": { 2024-04-25T01:24:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba", 2024-04-25T01:24:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T01:24:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB container image", 2024-04-25T01:24:22.635 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.636 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.636 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.636 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2024-04-25T01:24:22.636 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2024-04-25T01:24:22.636 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.636 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.636 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.636 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T01:24:22.636 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.636 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.636 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.636 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2024-04-25T01:24:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2024-04-25T01:24:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2024-04-25T01:24:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T01:24:22.637 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2024-04-25T01:24:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2024-04-25T01:24:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T01:24:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T01:24:22.638 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2024-04-25T01:24:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T01:24:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2024-04-25T01:24:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T01:24:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.639 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2024-04-25T01:24:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2024-04-25T01:24:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2024-04-25T01:24:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T01:24:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2024-04-25T01:24:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2024-04-25T01:24:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2024-04-25T01:24:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2024-04-25T01:24:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2024-04-25T01:24:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2024-04-25T01:24:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T01:24:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2024-04-25T01:24:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2024-04-25T01:24:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T01:24:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2024-04-25T01:24:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.644 INFO:teuthology.orchestra.run.smithi031.stdout: "grafana_dashboards_path": { 2024-04-25T01:24:22.644 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "grafana_dashboards_path", 2024-04-25T01:24:22.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.644 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.644 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.644 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T01:24:22.644 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.644 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.644 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.644 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T01:24:22.644 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.644 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.644 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2024-04-25T01:24:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2024-04-25T01:24:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T01:24:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2024-04-25T01:24:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.645 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.646 INFO:teuthology.orchestra.run.smithi031.stdout: "hw_monitoring": { 2024-04-25T01:24:22.646 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hw_monitoring", 2024-04-25T01:24:22.646 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.646 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.646 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.646 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.646 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.646 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.646 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.646 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T01:24:22.646 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.646 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.646 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2024-04-25T01:24:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2024-04-25T01:24:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T01:24:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.647 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.648 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.648 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.648 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.648 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.648 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.648 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.648 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.648 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.648 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.648 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.648 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2024-04-25T01:24:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2024-04-25T01:24:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.649 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.650 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.650 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.650 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.650 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.650 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.650 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T01:24:22.650 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.650 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.650 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.650 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.650 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T01:24:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.651 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.652 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.652 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.652 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.652 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.652 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.652 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.652 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.652 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.652 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.652 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.652 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.653 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.653 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.653 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.653 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.653 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.653 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.653 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.653 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.653 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.653 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.654 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.654 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.654 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.654 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.654 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.654 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.654 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.654 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T01:24:22.654 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T01:24:22.654 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.654 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.654 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.654 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T01:24:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T01:24:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T01:24:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.655 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2024-04-25T01:24:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T01:24:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2024-04-25T01:24:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2024-04-25T01:24:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T01:24:22.656 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2024-04-25T01:24:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2024-04-25T01:24:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2024-04-25T01:24:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T01:24:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.657 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T01:24:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2024-04-25T01:24:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2024-04-25T01:24:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2024-04-25T01:24:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-04-25T01:24:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-04-25T01:24:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2024-04-25T01:24:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.659 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2024-04-25T01:24:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2024-04-25T01:24:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T01:24:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: "oob_default_addr": { 2024-04-25T01:24:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "oob_default_addr", 2024-04-25T01:24:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "169.254.1.1", 2024-04-25T01:24:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.660 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T01:24:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2024-04-25T01:24:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2024-04-25T01:24:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T01:24:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.661 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T01:24:22.662 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.662 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.662 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.662 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2024-04-25T01:24:22.662 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2024-04-25T01:24:22.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.662 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.662 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.662 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.662 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.662 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.662 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.663 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T01:24:22.663 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.663 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.663 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.663 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2024-04-25T01:24:22.663 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2024-04-25T01:24:22.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.663 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.663 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.663 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.663 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.663 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T01:24:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2024-04-25T01:24:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2024-04-25T01:24:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.664 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T01:24:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2024-04-25T01:24:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2024-04-25T01:24:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.665 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.666 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.666 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T01:24:22.666 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.666 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.666 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.666 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2024-04-25T01:24:22.666 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2024-04-25T01:24:22.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.666 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.666 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.666 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.666 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.666 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T01:24:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2024-04-25T01:24:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2024-04-25T01:24:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2024-04-25T01:24:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.667 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2024-04-25T01:24:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2024-04-25T01:24:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2024-04-25T01:24:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.668 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.669 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T01:24:22.669 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.669 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.669 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.669 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2024-04-25T01:24:22.669 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2024-04-25T01:24:22.669 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.669 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.669 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.669 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.669 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.669 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.669 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.669 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T01:24:22.669 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2024-04-25T01:24:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2024-04-25T01:24:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T01:24:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.671 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.671 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.671 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2024-04-25T01:24:22.671 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2024-04-25T01:24:22.671 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.671 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.671 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.671 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.671 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.671 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.671 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.671 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T01:24:22.672 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.672 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.672 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.672 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2024-04-25T01:24:22.672 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2024-04-25T01:24:22.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.672 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.672 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.672 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.672 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.672 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.672 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T01:24:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2024-04-25T01:24:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2024-04-25T01:24:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.673 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T01:24:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2024-04-25T01:24:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.674 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.675 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.675 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.675 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.675 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.675 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.675 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.675 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.675 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.675 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.675 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.675 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.675 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.675 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.675 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.676 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.678 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.678 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.678 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.678 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.678 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.678 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.678 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.678 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.678 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.678 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.678 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.678 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.679 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2024-04-25T01:24:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2024-04-25T01:24:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2024-04-25T01:24:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T01:24:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.680 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2024-04-25T01:24:22.681 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2024-04-25T01:24:22.681 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.681 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.681 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.681 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2024-04-25T01:24:22.681 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.681 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.681 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.681 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T01:24:22.681 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.681 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.681 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.682 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.682 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2024-04-25T01:24:22.682 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.682 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.682 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.682 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T01:24:22.682 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T01:24:22.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.682 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.682 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.682 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T01:24:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T01:24:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T01:24:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.683 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.684 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.684 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.684 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.684 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.684 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.684 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.684 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.684 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T01:24:22.684 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T01:24:22.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.684 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.684 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2024-04-25T01:24:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T01:24:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.685 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.686 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.686 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.686 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.686 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.686 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.686 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.686 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.686 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T01:24:22.686 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T01:24:22.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.686 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.686 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T01:24:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T01:24:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.687 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T01:24:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T01:24:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.688 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T01:24:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T01:24:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.689 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.690 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.690 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.690 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.690 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.690 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.690 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.690 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.690 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T01:24:22.690 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T01:24:22.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.690 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.690 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.690 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T01:24:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T01:24:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.691 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T01:24:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T01:24:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T01:24:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T01:24:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.693 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.694 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.694 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.694 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.694 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.694 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.694 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T01:24:22.694 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T01:24:22.694 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.694 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.694 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.694 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.694 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.695 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.695 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.695 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.695 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.695 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.695 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.695 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T01:24:22.695 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T01:24:22.695 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.695 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.695 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.695 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T01:24:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T01:24:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.696 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T01:24:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T01:24:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T01:24:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2024-04-25T01:24:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2024-04-25T01:24:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T01:24:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T01:24:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.699 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T01:24:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T01:24:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T01:24:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.700 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T01:24:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T01:24:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.701 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.702 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.702 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.702 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.702 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.702 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.702 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.702 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.702 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.702 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T01:24:22.702 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T01:24:22.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T01:24:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T01:24:22.703 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.704 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.704 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.704 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.704 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.704 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.704 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.704 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.704 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.704 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.704 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.704 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T01:24:22.704 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T01:24:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T01:24:22.705 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T01:24:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T01:24:22.706 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T01:24:22.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.707 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.707 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.707 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.707 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.707 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.707 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.707 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.707 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.707 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.707 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.707 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T01:24:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T01:24:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.708 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T01:24:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T01:24:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.709 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T01:24:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T01:24:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T01:24:22.710 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T01:24:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T01:24:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T01:24:22.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T01:24:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T01:24:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T01:24:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T01:24:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.713 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T01:24:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T01:24:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.714 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T01:24:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T01:24:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T01:24:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.715 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T01:24:22.716 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.716 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.716 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.716 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.716 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.716 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.716 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.716 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T01:24:22.716 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T01:24:22.716 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.716 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.716 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2024-04-25T01:24:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T01:24:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T01:24:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.717 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2024-04-25T01:24:22.718 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.718 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.718 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.718 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.718 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.718 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.718 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T01:24:22.718 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T01:24:22.718 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.718 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.719 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.719 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.719 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.719 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.719 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.719 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.719 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.719 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.719 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.719 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T01:24:22.719 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T01:24:22.719 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.719 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.720 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.720 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T01:24:22.720 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.720 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.720 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.720 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.720 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.720 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.720 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.720 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T01:24:22.720 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T01:24:22.720 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.721 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.721 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.721 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.721 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.721 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.721 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.721 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.721 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.721 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.721 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.721 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T01:24:22.721 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T01:24:22.721 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.722 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.722 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.722 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.722 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.722 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.722 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.722 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.722 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.722 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.722 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.722 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T01:24:22.722 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T01:24:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T01:24:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.723 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T01:24:22.724 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T01:24:22.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.724 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.724 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.724 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T01:24:22.724 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.724 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.724 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.724 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.724 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.724 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.724 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.725 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T01:24:22.725 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T01:24:22.725 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.725 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.725 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.725 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T01:24:22.725 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.725 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.725 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.725 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.725 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.725 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.725 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.726 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2024-04-25T01:24:22.726 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2024-04-25T01:24:22.726 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.726 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.726 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.726 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.726 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.726 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.726 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.726 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.726 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.726 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2024-04-25T01:24:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2024-04-25T01:24:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.727 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.728 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.728 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2024-04-25T01:24:22.728 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2024-04-25T01:24:22.728 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.728 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.728 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.728 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.728 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.728 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.728 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.728 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2024-04-25T01:24:22.728 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2024-04-25T01:24:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2024-04-25T01:24:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2024-04-25T01:24:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.730 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.730 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.730 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.730 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-04-25T01:24:22.730 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-04-25T01:24:22.730 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.730 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.730 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.730 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.730 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.730 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.730 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.731 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.731 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.731 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.731 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.731 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.731 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.731 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.731 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.731 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.731 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.731 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.731 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.733 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.733 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.733 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.733 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.733 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.733 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.733 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.733 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.733 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.733 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.733 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.733 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.733 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.734 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.734 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.734 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.734 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.734 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.734 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.734 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.734 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.734 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.734 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.734 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.734 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.734 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.735 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2024-04-25T01:24:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2024-04-25T01:24:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.736 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.737 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.737 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2024-04-25T01:24:22.737 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.737 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.737 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.737 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2024-04-25T01:24:22.737 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T01:24:22.737 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.737 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.737 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.737 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.737 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.738 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.738 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.738 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.738 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.738 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.738 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T01:24:22.738 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T01:24:22.738 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.738 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.738 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-04-25T01:24:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T01:24:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T01:24:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.739 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2024-04-25T01:24:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-04-25T01:24:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-04-25T01:24:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2024-04-25T01:24:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2024-04-25T01:24:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.741 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2024-04-25T01:24:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-04-25T01:24:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-04-25T01:24:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.742 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2024-04-25T01:24:22.743 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.743 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.743 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.743 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.743 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2024-04-25T01:24:22.743 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.743 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.743 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.743 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.743 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.743 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-04-25T01:24:22.743 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-04-25T01:24:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-04-25T01:24:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-04-25T01:24:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-04-25T01:24:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2024-04-25T01:24:22.744 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2024-04-25T01:24:22.745 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.745 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.745 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.745 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.745 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.745 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.745 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.745 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.745 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.745 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.745 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.746 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-04-25T01:24:22.746 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.746 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.746 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.746 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2024-04-25T01:24:22.746 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2024-04-25T01:24:22.746 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.746 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.746 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.746 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.746 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.746 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2024-04-25T01:24:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.747 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.748 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.748 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.748 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.748 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.748 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.748 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.748 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.748 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.748 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.748 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.748 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.748 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.749 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.750 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.751 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.751 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.751 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.751 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.751 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.751 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.751 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.751 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.751 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.751 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.751 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.751 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2024-04-25T01:24:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2024-04-25T01:24:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2024-04-25T01:24:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T01:24:22.752 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2024-04-25T01:24:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2024-04-25T01:24:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2024-04-25T01:24:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T01:24:22.753 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2024-04-25T01:24:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2024-04-25T01:24:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2024-04-25T01:24:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2024-04-25T01:24:22.754 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2024-04-25T01:24:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2024-04-25T01:24:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-04-25T01:24:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T01:24:22.755 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2024-04-25T01:24:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2024-04-25T01:24:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T01:24:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.756 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T01:24:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T01:24:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T01:24:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T01:24:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.757 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2024-04-25T01:24:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2024-04-25T01:24:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2024-04-25T01:24:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T01:24:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2024-04-25T01:24:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.759 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.760 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.761 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.761 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.761 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.761 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.761 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.761 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.761 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.761 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.761 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.761 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.761 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.762 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.762 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.762 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.762 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.762 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.762 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.762 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.762 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.762 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.762 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.762 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2024-04-25T01:24:22.764 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2024-04-25T01:24:22.765 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.765 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.765 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.765 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-04-25T01:24:22.765 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.765 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.765 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.765 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.765 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.765 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.765 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.765 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2024-04-25T01:24:22.765 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2024-04-25T01:24:22.765 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.766 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.766 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.766 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2024-04-25T01:24:22.766 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.766 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.766 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.766 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.766 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.766 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.766 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.766 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T01:24:22.766 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T01:24:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T01:24:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.768 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2024-04-25T01:24:22.768 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2024-04-25T01:24:22.768 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2024-04-25T01:24:22.768 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.768 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2024-04-25T01:24:22.768 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2024-04-25T01:24:22.768 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.768 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.768 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.768 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2024-04-25T01:24:22.768 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.768 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.768 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T01:24:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2024-04-25T01:24:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2024-04-25T01:24:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2024-04-25T01:24:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.769 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.770 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T01:24:22.770 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.770 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.770 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.770 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2024-04-25T01:24:22.770 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2024-04-25T01:24:22.770 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.770 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.770 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.770 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.770 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2024-04-25T01:24:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T01:24:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T01:24:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2024-04-25T01:24:22.771 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2024-04-25T01:24:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T01:24:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.772 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.773 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.774 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.774 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.774 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.774 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.774 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.774 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.774 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.774 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.774 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.774 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.774 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.774 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.776 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2024-04-25T01:24:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2024-04-25T01:24:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2024-04-25T01:24:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.778 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2024-04-25T01:24:22.778 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2024-04-25T01:24:22.778 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.778 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.778 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.778 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2024-04-25T01:24:22.778 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.778 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.778 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.778 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2024-04-25T01:24:22.778 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.778 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.778 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-04-25T01:24:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-04-25T01:24:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2024-04-25T01:24:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T01:24:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2024-04-25T01:24:22.779 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2024-04-25T01:24:22.780 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.780 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.780 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.780 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T01:24:22.780 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T01:24:22.780 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2024-04-25T01:24:22.780 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.780 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T01:24:22.780 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.780 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.780 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.780 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2024-04-25T01:24:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2024-04-25T01:24:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2024-04-25T01:24:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2024-04-25T01:24:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2024-04-25T01:24:22.781 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.782 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.782 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.782 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2024-04-25T01:24:22.782 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.782 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.782 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.782 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T01:24:22.782 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.782 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.782 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.783 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2024-04-25T01:24:22.783 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.783 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.783 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.783 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.783 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.783 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.783 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.783 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.783 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.783 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.783 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.783 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.783 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.784 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.785 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.786 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.787 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2024-04-25T01:24:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.788 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.789 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.790 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.791 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.792 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.792 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.792 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.792 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.792 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.792 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.792 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.792 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.792 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.792 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.792 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.792 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.792 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.793 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.793 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.793 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.793 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2024-04-25T01:24:22.793 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.793 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.793 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.793 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2024-04-25T01:24:22.793 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2024-04-25T01:24:22.793 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.794 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.794 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.794 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-04-25T01:24:22.794 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.794 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.794 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.794 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T01:24:22.794 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.794 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.794 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.794 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2024-04-25T01:24:22.794 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2024-04-25T01:24:22.795 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.795 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.795 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.795 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T01:24:22.795 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2024-04-25T01:24:22.795 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.795 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.795 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T01:24:22.795 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.795 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.795 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.796 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.796 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.796 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.796 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.796 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.796 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.796 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.796 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.796 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.796 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.796 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.796 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.796 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.797 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.797 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.797 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.797 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.797 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.797 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.797 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.797 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.797 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.797 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.797 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.797 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.798 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.798 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.798 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.798 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.798 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.798 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.798 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.798 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.798 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.798 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.798 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.798 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2024-04-25T01:24:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2024-04-25T01:24:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2024-04-25T01:24:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2024-04-25T01:24:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2024-04-25T01:24:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T01:24:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2024-04-25T01:24:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2024-04-25T01:24:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2024-04-25T01:24:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T01:24:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2024-04-25T01:24:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2024-04-25T01:24:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-04-25T01:24:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2024-04-25T01:24:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2024-04-25T01:24:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2024-04-25T01:24:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2024-04-25T01:24:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T01:24:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2024-04-25T01:24:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2024-04-25T01:24:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2024-04-25T01:24:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2024-04-25T01:24:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2024-04-25T01:24:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2024-04-25T01:24:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T01:24:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T01:24:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2024-04-25T01:24:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2024-04-25T01:24:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2024-04-25T01:24:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-04-25T01:24:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2024-04-25T01:24:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2024-04-25T01:24:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T01:24:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2024-04-25T01:24:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-04-25T01:24:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-04-25T01:24:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2024-04-25T01:24:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2024-04-25T01:24:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2024-04-25T01:24:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2024-04-25T01:24:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2024-04-25T01:24:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2024-04-25T01:24:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T01:24:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T01:24:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T01:24:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2024-04-25T01:24:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2024-04-25T01:24:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T01:24:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-04-25T01:24:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2024-04-25T01:24:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2024-04-25T01:24:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T01:24:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2024-04-25T01:24:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2024-04-25T01:24:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2024-04-25T01:24:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T01:24:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-04-25T01:24:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-04-25T01:24:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2024-04-25T01:24:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2024-04-25T01:24:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2024-04-25T01:24:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2024-04-25T01:24:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T01:24:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T01:24:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T01:24:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2024-04-25T01:24:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2024-04-25T01:24:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2024-04-25T01:24:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2024-04-25T01:24:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2024-04-25T01:24:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2024-04-25T01:24:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T01:24:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T01:24:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.868 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.870 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2024-04-25T01:24:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2024-04-25T01:24:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T01:24:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T01:24:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2024-04-25T01:24:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2024-04-25T01:24:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2024-04-25T01:24:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T01:24:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2024-04-25T01:24:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T01:24:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T01:24:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-04-25T01:24:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T01:24:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.874 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T01:24:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T01:24:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2024-04-25T01:24:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T01:24:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2024-04-25T01:24:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2024-04-25T01:24:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2024-04-25T01:24:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.876 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-04-25T01:24:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-04-25T01:24:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2024-04-25T01:24:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2024-04-25T01:24:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2024-04-25T01:24:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.877 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-04-25T01:24:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-04-25T01:24:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-04-25T01:24:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-04-25T01:24:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-04-25T01:24:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.878 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-04-25T01:24:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.879 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.880 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.881 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.883 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2024-04-25T01:24:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2024-04-25T01:24:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T01:24:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.884 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2024-04-25T01:24:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T01:24:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.885 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2024-04-25T01:24:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2024-04-25T01:24:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.886 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2024-04-25T01:24:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.887 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2024-04-25T01:24:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2024-04-25T01:24:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.888 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-04-25T01:24:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-04-25T01:24:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.889 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.890 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.891 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.892 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.893 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T01:24:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T01:24:22.894 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T01:24:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T01:24:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.895 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.896 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2024-04-25T01:24:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.897 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.899 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.900 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.901 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2024-04-25T01:24:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.902 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.903 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.904 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.905 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.906 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2024-04-25T01:24:22.907 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2024-04-25T01:24:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2024-04-25T01:24:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T01:24:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2024-04-25T01:24:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.909 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.910 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.911 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.912 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.913 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2024-04-25T01:24:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2024-04-25T01:24:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2024-04-25T01:24:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2024-04-25T01:24:22.915 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2024-04-25T01:24:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2024-04-25T01:24:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2024-04-25T01:24:22.917 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2024-04-25T01:24:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2024-04-25T01:24:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.919 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2024-04-25T01:24:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2024-04-25T01:24:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T01:24:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.920 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2024-04-25T01:24:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2024-04-25T01:24:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.921 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2024-04-25T01:24:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2024-04-25T01:24:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.922 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2024-04-25T01:24:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2024-04-25T01:24:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.923 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2024-04-25T01:24:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2024-04-25T01:24:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.924 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T01:24:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2024-04-25T01:24:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2024-04-25T01:24:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2024-04-25T01:24:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.925 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2024-04-25T01:24:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2024-04-25T01:24:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.926 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2024-04-25T01:24:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2024-04-25T01:24:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.927 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-04-25T01:24:22.928 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2024-04-25T01:24:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2024-04-25T01:24:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T01:24:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.929 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2024-04-25T01:24:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2024-04-25T01:24:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2024-04-25T01:24:22.930 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.931 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.932 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.933 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.934 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.935 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2024-04-25T01:24:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.936 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.937 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.938 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.939 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.940 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2024-04-25T01:24:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.942 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.943 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.944 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.945 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.946 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2024-04-25T01:24:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2024-04-25T01:24:22.947 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2024-04-25T01:24:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T01:24:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.948 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T01:24:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T01:24:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2024-04-25T01:24:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.949 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.950 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.951 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.952 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.953 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2024-04-25T01:24:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2024-04-25T01:24:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2024-04-25T01:24:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.955 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T01:24:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2024-04-25T01:24:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2024-04-25T01:24:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.956 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T01:24:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2024-04-25T01:24:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2024-04-25T01:24:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.957 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T01:24:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2024-04-25T01:24:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2024-04-25T01:24:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.958 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T01:24:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2024-04-25T01:24:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2024-04-25T01:24:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.959 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T01:24:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2024-04-25T01:24:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2024-04-25T01:24:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.960 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2024-04-25T01:24:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2024-04-25T01:24:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.961 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2024-04-25T01:24:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2024-04-25T01:24:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T01:24:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-04-25T01:24:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-04-25T01:24:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.963 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T01:24:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T01:24:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2024-04-25T01:24:22.964 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2024-04-25T01:24:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2024-04-25T01:24:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2024-04-25T01:24:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2024-04-25T01:24:22.965 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2024-04-25T01:24:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2024-04-25T01:24:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.966 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2024-04-25T01:24:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2024-04-25T01:24:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.967 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.968 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.970 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.971 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.972 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2024-04-25T01:24:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2024-04-25T01:24:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.973 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2024-04-25T01:24:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2024-04-25T01:24:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.974 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2024-04-25T01:24:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2024-04-25T01:24:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T01:24:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.975 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2024-04-25T01:24:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.976 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.977 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.978 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.979 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.980 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-04-25T01:24:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.982 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.983 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.984 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.985 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.986 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2024-04-25T01:24:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2024-04-25T01:24:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2024-04-25T01:24:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T01:24:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.987 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2024-04-25T01:24:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2024-04-25T01:24:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2024-04-25T01:24:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.988 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2024-04-25T01:24:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2024-04-25T01:24:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T01:24:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T01:24:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_no_wait": { 2024-04-25T01:24:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T01:24:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T01:24:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.990 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2024-04-25T01:24:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2024-04-25T01:24:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2024-04-25T01:24:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T01:24:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.991 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2024-04-25T01:24:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2024-04-25T01:24:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2024-04-25T01:24:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T01:24:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T01:24:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T01:24:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2024-04-25T01:24:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2024-04-25T01:24:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2024-04-25T01:24:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2024-04-25T01:24:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2024-04-25T01:24:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2024-04-25T01:24:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2024-04-25T01:24:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T01:24:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T01:24:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "modules": [ 2024-04-25T01:24:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-04-25T01:24:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "dashboard", 2024-04-25T01:24:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "iostat", 2024-04-25T01:24:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "nfs", 2024-04-25T01:24:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "restful" 2024-04-25T01:24:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2024-04-25T01:24:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2024-04-25T01:24:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T01:24:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T01:24:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T01:24:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T01:24:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2024-04-25T01:24:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2024-04-25T01:24:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2024-04-25T01:24:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2024-04-25T01:24:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2024-04-25T01:24:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2024-04-25T01:24:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2024-04-25T01:24:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2024-04-25T01:24:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2024-04-25T01:24:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2024-04-25T01:24:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2024-04-25T01:24:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2024-04-25T01:24:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2024-04-25T01:24:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2024-04-25T01:24:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2024-04-25T01:24:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2024-04-25T01:24:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2024-04-25T01:24:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2024-04-25T01:24:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2024-04-25T01:24:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2024-04-25T01:24:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2024-04-25T01:24:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2024-04-25T01:24:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T01:24:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2024-04-25T01:24:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2024-04-25T01:24:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2024-04-25T01:24:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2024-04-25T01:24:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2024-04-25T01:24:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2024-04-25T01:24:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T01:24:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2024-04-25T01:24:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2024-04-25T01:24:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2024-04-25T01:24:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T01:24:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T01:24:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2024-04-25T01:24:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2024-04-25T01:24:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T01:24:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T01:24:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-04-25T01:24:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-04-25T01:24:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T01:24:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T01:24:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2024-04-25T01:24:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2024-04-25T01:24:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T01:24:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2024-04-25T01:24:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T01:24:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2024-04-25T01:24:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T01:24:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2024-04-25T01:24:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2024-04-25T01:24:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T01:24:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T01:24:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T01:24:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2024-04-25T01:24:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2024-04-25T01:24:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T01:24:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2024-04-25T01:24:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2024-04-25T01:24:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2024-04-25T01:24:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2024-04-25T01:24:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T01:24:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2024-04-25T01:24:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2024-04-25T01:24:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2024-04-25T01:24:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T01:24:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T01:24:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2024-04-25T01:24:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2024-04-25T01:24:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T01:24:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T01:24:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-04-25T01:24:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-04-25T01:24:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T01:24:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T01:24:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2024-04-25T01:24:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2024-04-25T01:24:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T01:24:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T01:24:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T01:24:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-04-25T01:24:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-04-25T01:24:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2024-04-25T01:24:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2024-04-25T01:24:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2024-04-25T01:24:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "read", 2024-04-25T01:24:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap", 2024-04-25T01:24:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap-read" 2024-04-25T01:24:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2024-04-25T01:24:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2024-04-25T01:24:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2024-04-25T01:24:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T01:24:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T01:24:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T01:24:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T01:24:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T01:24:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2024-04-25T01:24:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2024-04-25T01:24:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T01:24:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T01:24:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T01:24:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T01:24:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2024-04-25T01:24:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2024-04-25T01:24:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T01:24:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T01:24:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T01:24:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2024-04-25T01:24:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2024-04-25T01:24:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2024-04-25T01:24:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T01:24:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-04-25T01:24:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T01:24:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2024-04-25T01:24:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2024-04-25T01:24:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2024-04-25T01:24:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T01:24:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2024-04-25T01:24:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2024-04-25T01:24:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2024-04-25T01:24:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T01:24:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2024-04-25T01:24:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2024-04-25T01:24:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T01:24:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T01:24:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2024-04-25T01:24:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2024-04-25T01:24:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T01:24:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T01:24:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2024-04-25T01:24:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T01:24:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T01:24:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2024-04-25T01:24:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T01:24:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2024-04-25T01:24:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2024-04-25T01:24:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2024-04-25T01:24:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2024-04-25T01:24:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2024-04-25T01:24:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T01:24:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2024-04-25T01:24:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2024-04-25T01:24:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T01:24:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2024-04-25T01:24:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2024-04-25T01:24:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T01:24:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2024-04-25T01:24:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2024-04-25T01:24:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T01:24:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2024-04-25T01:24:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2024-04-25T01:24:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T01:24:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T01:24:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2024-04-25T01:24:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2024-04-25T01:24:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T01:24:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2024-04-25T01:24:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2024-04-25T01:24:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2024-04-25T01:24:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T01:24:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2024-04-25T01:24:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2024-04-25T01:24:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2024-04-25T01:24:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T01:24:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T01:24:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2024-04-25T01:24:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2024-04-25T01:24:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T01:24:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2024-04-25T01:24:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2024-04-25T01:24:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2024-04-25T01:24:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T01:24:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2024-04-25T01:24:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2024-04-25T01:24:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2024-04-25T01:24:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T01:24:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2024-04-25T01:24:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2024-04-25T01:24:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2024-04-25T01:24:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T01:24:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2024-04-25T01:24:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2024-04-25T01:24:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2024-04-25T01:24:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T01:24:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2024-04-25T01:24:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2024-04-25T01:24:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2024-04-25T01:24:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T01:24:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2024-04-25T01:24:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2024-04-25T01:24:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2024-04-25T01:24:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T01:24:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T01:24:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2024-04-25T01:24:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2024-04-25T01:24:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T01:24:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2024-04-25T01:24:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2024-04-25T01:24:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2024-04-25T01:24:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T01:24:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T01:24:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2024-04-25T01:24:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2024-04-25T01:24:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T01:24:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2024-04-25T01:24:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba": { 2024-04-25T01:24:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba", 2024-04-25T01:24:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T01:24:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB container image", 2024-04-25T01:24:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2024-04-25T01:24:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2024-04-25T01:24:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T01:24:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2024-04-25T01:24:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2024-04-25T01:24:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2024-04-25T01:24:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T01:24:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2024-04-25T01:24:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2024-04-25T01:24:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T01:24:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T01:24:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2024-04-25T01:24:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T01:24:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2024-04-25T01:24:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T01:24:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2024-04-25T01:24:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2024-04-25T01:24:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2024-04-25T01:24:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T01:24:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2024-04-25T01:24:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2024-04-25T01:24:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2024-04-25T01:24:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2024-04-25T01:24:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2024-04-25T01:24:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2024-04-25T01:24:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T01:24:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2024-04-25T01:24:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2024-04-25T01:24:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T01:24:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2024-04-25T01:24:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "grafana_dashboards_path": { 2024-04-25T01:24:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "grafana_dashboards_path", 2024-04-25T01:24:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T01:24:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T01:24:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2024-04-25T01:24:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2024-04-25T01:24:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T01:24:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2024-04-25T01:24:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "hw_monitoring": { 2024-04-25T01:24:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hw_monitoring", 2024-04-25T01:24:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T01:24:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2024-04-25T01:24:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2024-04-25T01:24:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T01:24:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2024-04-25T01:24:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2024-04-25T01:24:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T01:24:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T01:24:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T01:24:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T01:24:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T01:24:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T01:24:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T01:24:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2024-04-25T01:24:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T01:24:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2024-04-25T01:24:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2024-04-25T01:24:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T01:24:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2024-04-25T01:24:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2024-04-25T01:24:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2024-04-25T01:24:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T01:24:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T01:24:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2024-04-25T01:24:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2024-04-25T01:24:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2024-04-25T01:24:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-04-25T01:24:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-04-25T01:24:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2024-04-25T01:24:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2024-04-25T01:24:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2024-04-25T01:24:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T01:24:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "oob_default_addr": { 2024-04-25T01:24:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "oob_default_addr", 2024-04-25T01:24:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "169.254.1.1", 2024-04-25T01:24:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T01:24:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2024-04-25T01:24:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2024-04-25T01:24:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T01:24:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T01:24:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2024-04-25T01:24:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2024-04-25T01:24:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T01:24:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2024-04-25T01:24:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2024-04-25T01:24:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T01:24:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2024-04-25T01:24:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2024-04-25T01:24:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T01:24:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2024-04-25T01:24:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2024-04-25T01:24:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T01:24:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2024-04-25T01:24:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2024-04-25T01:24:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T01:24:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2024-04-25T01:24:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2024-04-25T01:24:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2024-04-25T01:24:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2024-04-25T01:24:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2024-04-25T01:24:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2024-04-25T01:24:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T01:24:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2024-04-25T01:24:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2024-04-25T01:24:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T01:24:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2024-04-25T01:24:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2024-04-25T01:24:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T01:24:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2024-04-25T01:24:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2024-04-25T01:24:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T01:24:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2024-04-25T01:24:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2024-04-25T01:24:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T01:24:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2024-04-25T01:24:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2024-04-25T01:24:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T01:24:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2024-04-25T01:24:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2024-04-25T01:24:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2024-04-25T01:24:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2024-04-25T01:24:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T01:24:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2024-04-25T01:24:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2024-04-25T01:24:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2024-04-25T01:24:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T01:24:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2024-04-25T01:24:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T01:24:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T01:24:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T01:24:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T01:24:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T01:24:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T01:24:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T01:24:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2024-04-25T01:24:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T01:24:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T01:24:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T01:24:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T01:24:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T01:24:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T01:24:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T01:24:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T01:24:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T01:24:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T01:24:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T01:24:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T01:24:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T01:24:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T01:24:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T01:24:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T01:24:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T01:24:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T01:24:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T01:24:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T01:24:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T01:24:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T01:24:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T01:24:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T01:24:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T01:24:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T01:24:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2024-04-25T01:24:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2024-04-25T01:24:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T01:24:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T01:24:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T01:24:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T01:24:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T01:24:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T01:24:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T01:24:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T01:24:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T01:24:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T01:24:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T01:24:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T01:24:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T01:24:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T01:24:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T01:24:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T01:24:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T01:24:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T01:24:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T01:24:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T01:24:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T01:24:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T01:24:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T01:24:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T01:24:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T01:24:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T01:24:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T01:24:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T01:24:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T01:24:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T01:24:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T01:24:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T01:24:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T01:24:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T01:24:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T01:24:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T01:24:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T01:24:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T01:24:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T01:24:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2024-04-25T01:24:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T01:24:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T01:24:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2024-04-25T01:24:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T01:24:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T01:24:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T01:24:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T01:24:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T01:24:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T01:24:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T01:24:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T01:24:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T01:24:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T01:24:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T01:24:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T01:24:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T01:24:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T01:24:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T01:24:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T01:24:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T01:24:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T01:24:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2024-04-25T01:24:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2024-04-25T01:24:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2024-04-25T01:24:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2024-04-25T01:24:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2024-04-25T01:24:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2024-04-25T01:24:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2024-04-25T01:24:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2024-04-25T01:24:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2024-04-25T01:24:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2024-04-25T01:24:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-04-25T01:24:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-04-25T01:24:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2024-04-25T01:24:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2024-04-25T01:24:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.162 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2024-04-25T01:24:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2024-04-25T01:24:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T01:24:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.163 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T01:24:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T01:24:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.164 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-04-25T01:24:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T01:24:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T01:24:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2024-04-25T01:24:23.165 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-04-25T01:24:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-04-25T01:24:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.166 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2024-04-25T01:24:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2024-04-25T01:24:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2024-04-25T01:24:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.167 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-04-25T01:24:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-04-25T01:24:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2024-04-25T01:24:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.168 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2024-04-25T01:24:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-04-25T01:24:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-04-25T01:24:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.169 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-04-25T01:24:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-04-25T01:24:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-04-25T01:24:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2024-04-25T01:24:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2024-04-25T01:24:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-04-25T01:24:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2024-04-25T01:24:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2024-04-25T01:24:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2024-04-25T01:24:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2024-04-25T01:24:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2024-04-25T01:24:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2024-04-25T01:24:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T01:24:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2024-04-25T01:24:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2024-04-25T01:24:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2024-04-25T01:24:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T01:24:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2024-04-25T01:24:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2024-04-25T01:24:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2024-04-25T01:24:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2024-04-25T01:24:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2024-04-25T01:24:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2024-04-25T01:24:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-04-25T01:24:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T01:24:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2024-04-25T01:24:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2024-04-25T01:24:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T01:24:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T01:24:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T01:24:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T01:24:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T01:24:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2024-04-25T01:24:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2024-04-25T01:24:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2024-04-25T01:24:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T01:24:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2024-04-25T01:24:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.188 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2024-04-25T01:24:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2024-04-25T01:24:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-04-25T01:24:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.189 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2024-04-25T01:24:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2024-04-25T01:24:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2024-04-25T01:24:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T01:24:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T01:24:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T01:24:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.191 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2024-04-25T01:24:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2024-04-25T01:24:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2024-04-25T01:24:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2024-04-25T01:24:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2024-04-25T01:24:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.192 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2024-04-25T01:24:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T01:24:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2024-04-25T01:24:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2024-04-25T01:24:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.193 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2024-04-25T01:24:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T01:24:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2024-04-25T01:24:23.194 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2024-04-25T01:24:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2024-04-25T01:24:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.195 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T01:24:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T01:24:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2024-04-25T01:24:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2024-04-25T01:24:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T01:24:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.196 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.197 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.198 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.199 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.200 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2024-04-25T01:24:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2024-04-25T01:24:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2024-04-25T01:24:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2024-04-25T01:24:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2024-04-25T01:24:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2024-04-25T01:24:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2024-04-25T01:24:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.203 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-04-25T01:24:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-04-25T01:24:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2024-04-25T01:24:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T01:24:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2024-04-25T01:24:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2024-04-25T01:24:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T01:24:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T01:24:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2024-04-25T01:24:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T01:24:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.205 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2024-04-25T01:24:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2024-04-25T01:24:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2024-04-25T01:24:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2024-04-25T01:24:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2024-04-25T01:24:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2024-04-25T01:24:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T01:24:23.207 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2024-04-25T01:24:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.208 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.211 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2024-04-25T01:24:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.213 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.214 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.215 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.216 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.217 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2024-04-25T01:24:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.218 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2024-04-25T01:24:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2024-04-25T01:24:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-04-25T01:24:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T01:24:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2024-04-25T01:24:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2024-04-25T01:24:23.219 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T01:24:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2024-04-25T01:24:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T01:24:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.220 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.221 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.222 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.223 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.225 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.225 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.225 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.225 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.225 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.225 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.225 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.225 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.225 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2024-04-25T01:24:23.225 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.225 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.226 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.226 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2024-04-25T01:24:23.226 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2024-04-25T01:24:23.226 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.226 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.226 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.226 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2024-04-25T01:24:23.226 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.226 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.226 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.226 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2024-04-25T01:24:23.226 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T01:24:23.226 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.227 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.230 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.230 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.230 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.230 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.230 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.230 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.230 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.230 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.230 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.230 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.230 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.230 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.230 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.231 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2024-04-25T01:24:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2024-04-25T01:24:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2024-04-25T01:24:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T01:24:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.232 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.233 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2024-04-25T01:24:23.233 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2024-04-25T01:24:23.233 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.233 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.233 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.233 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-04-25T01:24:23.233 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.233 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.233 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.233 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2024-04-25T01:24:23.233 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2024-04-25T01:24:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2024-04-25T01:24:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2024-04-25T01:24:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T01:24:23.234 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2024-04-25T01:24:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2024-04-25T01:24:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2024-04-25T01:24:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.235 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2024-04-25T01:24:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2024-04-25T01:24:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2024-04-25T01:24:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T01:24:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T01:24:23.236 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.237 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.237 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2024-04-25T01:24:23.237 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.237 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.237 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.237 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.237 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.238 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.238 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.238 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.238 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.238 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.238 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.239 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.240 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.241 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.242 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2024-04-25T01:24:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.243 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.245 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.246 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.246 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.246 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.246 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.246 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.246 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.246 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.246 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.246 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.246 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.246 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.246 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.246 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.247 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2024-04-25T01:24:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.248 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.249 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.250 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.250 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.250 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.250 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.250 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.250 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.250 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.250 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.250 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.250 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.250 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.251 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-04-25T01:24:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2024-04-25T01:24:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2024-04-25T01:24:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T01:24:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.255 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2024-04-25T01:24:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-04-25T01:24:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-04-25T01:24:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2024-04-25T01:24:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2024-04-25T01:24:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2024-04-25T01:24:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2024-04-25T01:24:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2024-04-25T01:24:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2024-04-25T01:24:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T01:24:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T01:24:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T01:24:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2024-04-25T01:24:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2024-04-25T01:24:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T01:24:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-04-25T01:24:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2024-04-25T01:24:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2024-04-25T01:24:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T01:24:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2024-04-25T01:24:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2024-04-25T01:24:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2024-04-25T01:24:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T01:24:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-04-25T01:24:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-04-25T01:24:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2024-04-25T01:24:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2024-04-25T01:24:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2024-04-25T01:24:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2024-04-25T01:24:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T01:24:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T01:24:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T01:24:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2024-04-25T01:24:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2024-04-25T01:24:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2024-04-25T01:24:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2024-04-25T01:24:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2024-04-25T01:24:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2024-04-25T01:24:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T01:24:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T01:24:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2024-04-25T01:24:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2024-04-25T01:24:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T01:24:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T01:24:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2024-04-25T01:24:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2024-04-25T01:24:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2024-04-25T01:24:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T01:24:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2024-04-25T01:24:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T01:24:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T01:24:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-04-25T01:24:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T01:24:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T01:24:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T01:24:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2024-04-25T01:24:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T01:24:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2024-04-25T01:24:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2024-04-25T01:24:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2024-04-25T01:24:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-04-25T01:24:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-04-25T01:24:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2024-04-25T01:24:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2024-04-25T01:24:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2024-04-25T01:24:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-04-25T01:24:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-04-25T01:24:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-04-25T01:24:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-04-25T01:24:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-04-25T01:24:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-04-25T01:24:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2024-04-25T01:24:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2024-04-25T01:24:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T01:24:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2024-04-25T01:24:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T01:24:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2024-04-25T01:24:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2024-04-25T01:24:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2024-04-25T01:24:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2024-04-25T01:24:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2024-04-25T01:24:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-04-25T01:24:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-04-25T01:24:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T01:24:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T01:24:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T01:24:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T01:24:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2024-04-25T01:24:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2024-04-25T01:24:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2024-04-25T01:24:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2024-04-25T01:24:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2024-04-25T01:24:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T01:24:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2024-04-25T01:24:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2024-04-25T01:24:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2024-04-25T01:24:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2024-04-25T01:24:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2024-04-25T01:24:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2024-04-25T01:24:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2024-04-25T01:24:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2024-04-25T01:24:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2024-04-25T01:24:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2024-04-25T01:24:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2024-04-25T01:24:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2024-04-25T01:24:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T01:24:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2024-04-25T01:24:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2024-04-25T01:24:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2024-04-25T01:24:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2024-04-25T01:24:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2024-04-25T01:24:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2024-04-25T01:24:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2024-04-25T01:24:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2024-04-25T01:24:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T01:24:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2024-04-25T01:24:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2024-04-25T01:24:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2024-04-25T01:24:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2024-04-25T01:24:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2024-04-25T01:24:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2024-04-25T01:24:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2024-04-25T01:24:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-04-25T01:24:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2024-04-25T01:24:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2024-04-25T01:24:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T01:24:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2024-04-25T01:24:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2024-04-25T01:24:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2024-04-25T01:24:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2024-04-25T01:24:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.364 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.364 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.364 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.364 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.364 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.364 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.364 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.364 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.364 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.364 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.364 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.364 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.364 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.365 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.365 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.365 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.365 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.365 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.365 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.365 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.365 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.365 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.366 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.366 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.366 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.366 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.366 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2024-04-25T01:24:23.366 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.366 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.366 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.366 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.366 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.366 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.367 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.367 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.367 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.367 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.367 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.367 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.367 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.367 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.367 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.367 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.367 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.367 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.367 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.368 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.369 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.369 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.369 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.369 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.369 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.369 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.369 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.369 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.369 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.369 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.370 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.370 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.370 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.370 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.370 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.370 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.370 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.370 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.370 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.370 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.370 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.370 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.371 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2024-04-25T01:24:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2024-04-25T01:24:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2024-04-25T01:24:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T01:24:23.372 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T01:24:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T01:24:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2024-04-25T01:24:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.377 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.377 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.377 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.377 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.377 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.377 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.377 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.377 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.377 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.377 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.377 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.377 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.378 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.378 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.378 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.378 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.378 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.378 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.378 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.378 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.378 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.378 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.378 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.378 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.378 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.379 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.379 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.379 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.379 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.379 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.379 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.379 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.379 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.379 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2024-04-25T01:24:23.379 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.379 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.379 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2024-04-25T01:24:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2024-04-25T01:24:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T01:24:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2024-04-25T01:24:23.380 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2024-04-25T01:24:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T01:24:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2024-04-25T01:24:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2024-04-25T01:24:23.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.382 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.382 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.382 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.382 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.382 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.382 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.382 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T01:24:23.382 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.382 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.382 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.382 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2024-04-25T01:24:23.382 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2024-04-25T01:24:23.383 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.383 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.383 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.383 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.383 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.383 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.383 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.383 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T01:24:23.383 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.383 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.383 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.383 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2024-04-25T01:24:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2024-04-25T01:24:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T01:24:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2024-04-25T01:24:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2024-04-25T01:24:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.385 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2024-04-25T01:24:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2024-04-25T01:24:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2024-04-25T01:24:23.386 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2024-04-25T01:24:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T01:24:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-04-25T01:24:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-04-25T01:24:23.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T01:24:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T01:24:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.388 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2024-04-25T01:24:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2024-04-25T01:24:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2024-04-25T01:24:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2024-04-25T01:24:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2024-04-25T01:24:23.389 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2024-04-25T01:24:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2024-04-25T01:24:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.390 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2024-04-25T01:24:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2024-04-25T01:24:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.392 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.393 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.394 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.395 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.396 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.397 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2024-04-25T01:24:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2024-04-25T01:24:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2024-04-25T01:24:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2024-04-25T01:24:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.399 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2024-04-25T01:24:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2024-04-25T01:24:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T01:24:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2024-04-25T01:24:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.401 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.402 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.404 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.406 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-04-25T01:24:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.407 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.409 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.410 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.411 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2024-04-25T01:24:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2024-04-25T01:24:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2024-04-25T01:24:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T01:24:23.412 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2024-04-25T01:24:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2024-04-25T01:24:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2024-04-25T01:24:23.413 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2024-04-25T01:24:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2024-04-25T01:24:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T01:24:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T01:24:23.414 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_no_wait": { 2024-04-25T01:24:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T01:24:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T01:24:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.415 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T01:24:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2024-04-25T01:24:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T01:24:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T01:24:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T01:24:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2024-04-25T01:24:23.416 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2024-04-25T01:24:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T01:24:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2024-04-25T01:24:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2024-04-25T01:24:23.417 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2024-04-25T01:24:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T01:24:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T01:24:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T01:24:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T01:24:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T01:24:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T01:24:23.419 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.420 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T01:24:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T01:24:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.421 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T01:24:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T01:24:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T01:24:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T01:24:23.422 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T01:24:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T01:24:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T01:24:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T01:24:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T01:24:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T01:24:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T01:24:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T01:24:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T01:24:23.423 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T01:24:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T01:24:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2024-04-25T01:24:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2024-04-25T01:24:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.424 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T01:24:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2024-04-25T01:24:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2024-04-25T01:24:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T01:24:23.425 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2024-04-25T01:24:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2024-04-25T01:24:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2024-04-25T01:24:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T01:24:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T01:24:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T01:24:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T01:24:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T01:24:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T01:24:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T01:24:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T01:24:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T01:24:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T01:24:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T01:24:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.427 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: "services": { 2024-04-25T01:24:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: "dashboard": "https://172.21.15.31:8443/" 2024-04-25T01:24:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: "always_on_modules": { 2024-04-25T01:24:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: "octopus": [ 2024-04-25T01:24:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-04-25T01:24:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-25T01:24:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-04-25T01:24:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-04-25T01:24:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-04-25T01:24:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-04-25T01:24:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-04-25T01:24:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-04-25T01:24:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-04-25T01:24:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-04-25T01:24:23.428 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: "pacific": [ 2024-04-25T01:24:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-04-25T01:24:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-25T01:24:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-04-25T01:24:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-04-25T01:24:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-04-25T01:24:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-04-25T01:24:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-04-25T01:24:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-04-25T01:24:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-04-25T01:24:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-04-25T01:24:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: "quincy": [ 2024-04-25T01:24:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-04-25T01:24:23.429 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-25T01:24:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-04-25T01:24:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-04-25T01:24:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-04-25T01:24:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-04-25T01:24:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-04-25T01:24:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-04-25T01:24:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-04-25T01:24:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-04-25T01:24:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: "reef": [ 2024-04-25T01:24:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-04-25T01:24:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-25T01:24:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-04-25T01:24:23.430 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-04-25T01:24:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-04-25T01:24:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-04-25T01:24:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-04-25T01:24:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-04-25T01:24:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-04-25T01:24:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-04-25T01:24:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T01:24:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: "squid": [ 2024-04-25T01:24:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-04-25T01:24:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-25T01:24:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-04-25T01:24:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-04-25T01:24:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-04-25T01:24:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-04-25T01:24:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-04-25T01:24:23.431 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-04-25T01:24:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-04-25T01:24:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-04-25T01:24:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T01:24:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: "last_failure_osd_epoch": 3, 2024-04-25T01:24:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: "active_clients": [ 2024-04-25T01:24:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-04-25T01:24:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-04-25T01:24:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-04-25T01:24:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-04-25T01:24:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 638237247 2024-04-25T01:24:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.432 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T01:24:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "libcephsqlite", 2024-04-25T01:24:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-04-25T01:24:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-04-25T01:24:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-04-25T01:24:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 187704274 2024-04-25T01:24:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T01:24:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-04-25T01:24:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-04-25T01:24:23.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-04-25T01:24:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-04-25T01:24:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3643837105 2024-04-25T01:24:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T01:24:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-04-25T01:24:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-04-25T01:24:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T01:24:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-04-25T01:24:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-04-25T01:24:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2811376504 2024-04-25T01:24:23.434 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.435 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T01:24:23.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:23.435 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T01:24:23.435 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T01:24:23.436 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:22 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3117342106' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:24:23.436 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:22 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3676795655' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T01:24:23.436 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:22 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3117342106' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:24:23.436 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:22 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3676795655' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T01:24:23.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:22.167+0000 7f21eec00640 1 -- 172.21.15.31:0/3676795655 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f21cc0a2c10 msgr2=0x7f21cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:23.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:22.167+0000 7f21eec00640 1 --2- 172.21.15.31:0/3676795655 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f21cc0a2c10 0x7f21cc0a50d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f21e8104810 tx=0x7f21d8009260 comp rx=0 tx=0).stop 2024-04-25T01:24:23.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:22.167+0000 7f21eec00640 1 -- 172.21.15.31:0/3676795655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e8105370 msgr2=0x7f21e81a1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:23.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:22.167+0000 7f21eec00640 1 --2- 172.21.15.31:0/3676795655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e8105370 0x7f21e81a1010 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009b30 tx=0x7f21e00305d0 comp rx=0 tx=0).stop 2024-04-25T01:24:23.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:22.168+0000 7f21eec00640 1 -- 172.21.15.31:0/3676795655 shutdown_connections 2024-04-25T01:24:23.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:22.168+0000 7f21eec00640 1 --2- 172.21.15.31:0/3676795655 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f21cc0a2c10 0x7f21cc0a50d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:23.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:22.168+0000 7f21eec00640 1 --2- 172.21.15.31:0/3676795655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e8105370 0x7f21e81a1010 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:23.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:22.168+0000 7f21eec00640 1 --2- 172.21.15.31:0/3676795655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21e81049b0 0x7f21e8199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:23.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:22.168+0000 7f21eec00640 1 --2- 172.21.15.31:0/3676795655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21e81037c0 0x7f21e8199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:23.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:22.168+0000 7f21eec00640 1 -- 172.21.15.31:0/3676795655 >> 172.21.15.31:0/3676795655 conn(0x7f21e80fed50 msgr2=0x7f21e8107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:24:23.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:22.168+0000 7f21eec00640 1 -- 172.21.15.31:0/3676795655 shutdown_connections 2024-04-25T01:24:23.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:22.168+0000 7f21eec00640 1 -- 172.21.15.31:0/3676795655 wait complete. 2024-04-25T01:24:23.444 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T01:24:23.444 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T01:24:23.445 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:22 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3117342106' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:24:23.445 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:22 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3676795655' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T01:24:23.909 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:24:23.938 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:23 smithi169 ceph-mon[32759]: pgmap v644: 97 pgs: 97 active+clean; 2.5 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 166 KiB/s rd, 12 MiB/s wr, 2.68k op/s 2024-04-25T01:24:23.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:23 smithi040 ceph-mon[32518]: pgmap v644: 97 pgs: 97 active+clean; 2.5 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 166 KiB/s rd, 12 MiB/s wr, 2.68k op/s 2024-04-25T01:24:23.999 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:23 smithi031 ceph-mon[28053]: pgmap v644: 97 pgs: 97 active+clean; 2.5 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 166 KiB/s rd, 12 MiB/s wr, 2.68k op/s 2024-04-25T01:24:24.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:24 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:24:24.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:24 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:24:24.990 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:24 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:24:25.749 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:25 smithi031 ceph-mon[28053]: pgmap v645: 97 pgs: 97 active+clean; 320 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 161 KiB/s rd, 5.1 MiB/s wr, 1.34k op/s 2024-04-25T01:24:25.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:24:25.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:24:25.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:25 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:24:25.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:25 smithi169 ceph-mon[32759]: pgmap v645: 97 pgs: 97 active+clean; 320 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 161 KiB/s rd, 5.1 MiB/s wr, 1.34k op/s 2024-04-25T01:24:25.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:24:25.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:24:25.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:25 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:24:25.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:25 smithi040 ceph-mon[32518]: pgmap v645: 97 pgs: 97 active+clean; 320 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 161 KiB/s rd, 5.1 MiB/s wr, 1.34k op/s 2024-04-25T01:24:25.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:24:25.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:24:25.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:25 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' 2024-04-25T01:24:26.447 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T01:24:26.447 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T01:24:26.447 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 186, 2024-04-25T01:24:26.447 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 181, 2024-04-25T01:24:26.447 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 470416, 2024-04-25T01:24:26.448 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 110576, 2024-04-25T01:24:26.448 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-04-25T01:24:26.448 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2024-04-25T01:24:26.448 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.071281955, 2024-04-25T01:24:26.448 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.034591485, 2024-04-25T01:24:26.448 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.025995271, 2024-04-25T01:24:26.448 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.007352451, 2024-04-25T01:24:26.448 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:26.448 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 181, 2024-04-25T01:24:26.448 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.041034816, 2024-04-25T01:24:26.448 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000226711 2024-04-25T01:24:26.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:26.448 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:26.448 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T01:24:26.449 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T01:24:26.449 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T01:24:26.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:26.449 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 470416, 2024-04-25T01:24:26.449 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 110576 2024-04-25T01:24:26.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:26.449 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T01:24:26.449 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 1, 2024-04-25T01:24:26.449 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 136, 2024-04-25T01:24:26.449 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 160, 2024-04-25T01:24:26.449 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 101488, 2024-04-25T01:24:26.449 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 3, 2024-04-25T01:24:26.449 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 2, 2024-04-25T01:24:26.449 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.025218639, 2024-04-25T01:24:26.449 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.019578936, 2024-04-25T01:24:26.449 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.002533679, 2024-04-25T01:24:26.450 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T01:24:26.450 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:26.450 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 134, 2024-04-25T01:24:26.450 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.028006804, 2024-04-25T01:24:26.450 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000209006 2024-04-25T01:24:26.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:26.450 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:26.450 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T01:24:26.450 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T01:24:26.450 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T01:24:26.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:26.450 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T01:24:26.450 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 101488 2024-04-25T01:24:26.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:26.451 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T01:24:26.451 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 0, 2024-04-25T01:24:26.451 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 2, 2024-04-25T01:24:26.451 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 0, 2024-04-25T01:24:26.451 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 0, 2024-04-25T01:24:26.451 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2024-04-25T01:24:26.451 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2024-04-25T01:24:26.451 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.001567477, 2024-04-25T01:24:26.451 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T01:24:26.451 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.001344599, 2024-04-25T01:24:26.451 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T01:24:26.451 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:26.451 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T01:24:26.452 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T01:24:26.452 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T01:24:26.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:26.452 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:26.452 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T01:24:26.452 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T01:24:26.452 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T01:24:26.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:26.452 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T01:24:26.452 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T01:24:26.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:26.452 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-04-25T01:24:26.452 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2024-04-25T01:24:26.452 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-04-25T01:24:26.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:26.453 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-04-25T01:24:26.453 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-04-25T01:24:26.453 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-04-25T01:24:26.453 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T01:24:26.453 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-04-25T01:24:26.453 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T01:24:26.453 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-04-25T01:24:26.453 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T01:24:26.453 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T01:24:26.453 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T01:24:26.453 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T01:24:26.453 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T01:24:26.453 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-04-25T01:24:26.453 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T01:24:26.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T01:24:26.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-04-25T01:24:26.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-04-25T01:24:26.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-04-25T01:24:26.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-04-25T01:24:26.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T01:24:26.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T01:24:26.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T01:24:26.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T01:24:26.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T01:24:26.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-04-25T01:24:26.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-04-25T01:24:26.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-04-25T01:24:26.454 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T01:24:26.455 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T01:24:26.455 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-04-25T01:24:26.455 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-04-25T01:24:26.455 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-04-25T01:24:26.455 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-04-25T01:24:26.455 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T01:24:26.455 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-04-25T01:24:26.455 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T01:24:26.455 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-04-25T01:24:26.455 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 148967, 2024-04-25T01:24:26.455 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 29, 2024-04-25T01:24:26.455 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-04-25T01:24:26.455 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-04-25T01:24:26.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-04-25T01:24:26.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-04-25T01:24:26.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-04-25T01:24:26.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-04-25T01:24:26.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-04-25T01:24:26.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-04-25T01:24:26.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2024-04-25T01:24:26.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2024-04-25T01:24:26.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T01:24:26.456 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-04-25T01:24:26.456 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-04-25T01:24:26.456 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-04-25T01:24:26.456 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 928, 2024-04-25T01:24:26.456 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2024-04-25T01:24:26.456 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-04-25T01:24:26.456 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-04-25T01:24:26.456 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-04-25T01:24:26.456 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-04-25T01:24:26.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:26.457 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T01:24:26.457 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-04-25T01:24:26.457 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-04-25T01:24:26.457 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-04-25T01:24:26.457 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 183, 2024-04-25T01:24:26.457 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 441713, 2024-04-25T01:24:26.457 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:26.457 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 183, 2024-04-25T01:24:26.457 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-04-25T01:24:26.457 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-04-25T01:24:26.457 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 183, 2024-04-25T01:24:26.457 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 441713, 2024-04-25T01:24:26.457 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-04-25T01:24:26.458 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T01:24:26.458 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T01:24:26.458 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T01:24:26.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:26.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:26.458 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T01:24:26.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:26 smithi169 ceph-mon[32759]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:24:26.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:26 smithi040 ceph-mon[32518]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:24:26.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:26 smithi031 ceph-mon[28053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:24:27.389 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-25T01:24:27.389 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-25T01:24:27.389 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-04-25T01:24:27.389 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T01:24:27.390 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T01:24:27.820 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:24:28.026 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:27 smithi031 ceph-mon[28053]: pgmap v646: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 164 KiB/s rd, 4.8 MiB/s wr, 1.27k op/s 2024-04-25T01:24:28.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:27 smithi169 ceph-mon[32759]: pgmap v646: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 164 KiB/s rd, 4.8 MiB/s wr, 1.27k op/s 2024-04-25T01:24:28.191 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:27 smithi040 ceph-mon[32518]: pgmap v646: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 164 KiB/s rd, 4.8 MiB/s wr, 1.27k op/s 2024-04-25T01:24:29.981 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:29 smithi031 ceph-mon[28053]: pgmap v647: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 160 KiB/s rd, 145 KiB/s wr, 310 op/s 2024-04-25T01:24:30.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:29 smithi169 ceph-mon[32759]: pgmap v647: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 160 KiB/s rd, 145 KiB/s wr, 310 op/s 2024-04-25T01:24:30.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:29 smithi040 ceph-mon[32518]: pgmap v647: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 160 KiB/s rd, 145 KiB/s wr, 310 op/s 2024-04-25T01:24:30.215 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T01:24:30.216 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T01:24:30.216 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 0, 2024-04-25T01:24:30.216 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 2, 2024-04-25T01:24:30.216 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 0, 2024-04-25T01:24:30.216 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 0, 2024-04-25T01:24:30.216 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-04-25T01:24:30.216 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2024-04-25T01:24:30.216 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.004005948, 2024-04-25T01:24:30.216 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T01:24:30.216 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.000900937, 2024-04-25T01:24:30.217 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T01:24:30.217 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:30.217 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T01:24:30.217 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T01:24:30.217 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T01:24:30.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:30.217 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:30.217 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T01:24:30.217 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T01:24:30.217 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T01:24:30.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:30.218 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T01:24:30.218 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T01:24:30.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:30.218 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T01:24:30.218 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 188, 2024-04-25T01:24:30.218 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 185, 2024-04-25T01:24:30.218 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 469056, 2024-04-25T01:24:30.218 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 113968, 2024-04-25T01:24:30.218 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2024-04-25T01:24:30.218 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2024-04-25T01:24:30.218 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.048996618, 2024-04-25T01:24:30.219 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.014611114, 2024-04-25T01:24:30.219 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.027176271, 2024-04-25T01:24:30.219 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.008182098, 2024-04-25T01:24:30.219 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:30.219 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 184, 2024-04-25T01:24:30.219 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.034369430, 2024-04-25T01:24:30.219 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000186790 2024-04-25T01:24:30.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:30.219 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:30.219 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T01:24:30.219 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T01:24:30.219 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T01:24:30.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:30.220 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 469056, 2024-04-25T01:24:30.220 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 113968 2024-04-25T01:24:30.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:30.220 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T01:24:30.220 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 5, 2024-04-25T01:24:30.220 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 144, 2024-04-25T01:24:30.220 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 3152, 2024-04-25T01:24:30.220 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 102992, 2024-04-25T01:24:30.220 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 3, 2024-04-25T01:24:30.220 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 2, 2024-04-25T01:24:30.220 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.022459861, 2024-04-25T01:24:30.221 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.017474969, 2024-04-25T01:24:30.221 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.002701039, 2024-04-25T01:24:30.221 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T01:24:30.221 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:30.221 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 143, 2024-04-25T01:24:30.221 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.038855029, 2024-04-25T01:24:30.221 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000271713 2024-04-25T01:24:30.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:30.221 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:30.221 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T01:24:30.222 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T01:24:30.222 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T01:24:30.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:30.222 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-25T01:24:30.222 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 102992 2024-04-25T01:24:30.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:30.222 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-04-25T01:24:30.222 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2024-04-25T01:24:30.222 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-04-25T01:24:30.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:30.222 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-04-25T01:24:30.223 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-04-25T01:24:30.223 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-04-25T01:24:30.223 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T01:24:30.223 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-04-25T01:24:30.223 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T01:24:30.223 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-04-25T01:24:30.223 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T01:24:30.223 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T01:24:30.223 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T01:24:30.223 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T01:24:30.224 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T01:24:30.224 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-04-25T01:24:30.224 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T01:24:30.224 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T01:24:30.224 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-04-25T01:24:30.224 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-04-25T01:24:30.224 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-04-25T01:24:30.224 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-04-25T01:24:30.224 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T01:24:30.224 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T01:24:30.224 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T01:24:30.225 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T01:24:30.225 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T01:24:30.225 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-04-25T01:24:30.225 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-04-25T01:24:30.225 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-04-25T01:24:30.225 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T01:24:30.225 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T01:24:30.225 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-04-25T01:24:30.225 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-04-25T01:24:30.225 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-04-25T01:24:30.225 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-04-25T01:24:30.225 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T01:24:30.226 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-04-25T01:24:30.226 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T01:24:30.226 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-04-25T01:24:30.226 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 151029, 2024-04-25T01:24:30.226 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 31, 2024-04-25T01:24:30.226 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-04-25T01:24:30.226 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-04-25T01:24:30.226 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-04-25T01:24:30.226 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-04-25T01:24:30.226 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-04-25T01:24:30.226 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-04-25T01:24:30.227 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-04-25T01:24:30.227 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-04-25T01:24:30.227 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2024-04-25T01:24:30.227 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2024-04-25T01:24:30.227 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T01:24:30.227 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-04-25T01:24:30.227 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-04-25T01:24:30.227 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-04-25T01:24:30.227 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 928, 2024-04-25T01:24:30.227 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2024-04-25T01:24:30.227 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-04-25T01:24:30.228 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-04-25T01:24:30.228 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-04-25T01:24:30.228 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-04-25T01:24:30.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:30.228 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T01:24:30.228 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-04-25T01:24:30.228 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-04-25T01:24:30.228 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-04-25T01:24:30.228 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 189, 2024-04-25T01:24:30.228 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 442493, 2024-04-25T01:24:30.228 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:30.228 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 189, 2024-04-25T01:24:30.229 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-04-25T01:24:30.229 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-04-25T01:24:30.229 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 189, 2024-04-25T01:24:30.229 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 442493, 2024-04-25T01:24:30.229 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-04-25T01:24:30.229 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T01:24:30.229 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T01:24:30.229 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T01:24:30.229 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:30.229 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:30.229 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T01:24:31.129 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-25T01:24:31.129 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-25T01:24:31.129 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-25T01:24:31.129 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T01:24:31.129 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T01:24:31.580 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:24:31.999 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:31 smithi031 ceph-mon[28053]: pgmap v648: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 102 KiB/s rd, 80 KiB/s wr, 216 op/s 2024-04-25T01:24:32.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:31 smithi169 ceph-mon[32759]: pgmap v648: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 102 KiB/s rd, 80 KiB/s wr, 216 op/s 2024-04-25T01:24:32.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:31 smithi040 ceph-mon[32518]: pgmap v648: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 102 KiB/s rd, 80 KiB/s wr, 216 op/s 2024-04-25T01:24:33.785 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T01:24:33.786 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T01:24:33.786 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 185, 2024-04-25T01:24:33.786 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 182, 2024-04-25T01:24:33.786 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 468240, 2024-04-25T01:24:33.786 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 112048, 2024-04-25T01:24:33.786 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-04-25T01:24:33.786 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2024-04-25T01:24:33.786 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.068845313, 2024-04-25T01:24:33.786 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.030880674, 2024-04-25T01:24:33.786 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.023750778, 2024-04-25T01:24:33.786 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.008975403, 2024-04-25T01:24:33.786 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:33.787 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 181, 2024-04-25T01:24:33.787 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.063975290, 2024-04-25T01:24:33.787 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000353454 2024-04-25T01:24:33.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:33.787 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:33.787 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T01:24:33.787 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T01:24:33.787 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T01:24:33.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:33.787 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 468240, 2024-04-25T01:24:33.787 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 112048 2024-04-25T01:24:33.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:33.787 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T01:24:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 1, 2024-04-25T01:24:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 139, 2024-04-25T01:24:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 160, 2024-04-25T01:24:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 102016, 2024-04-25T01:24:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 3, 2024-04-25T01:24:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 2, 2024-04-25T01:24:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.030105758, 2024-04-25T01:24:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.025262950, 2024-04-25T01:24:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.002652652, 2024-04-25T01:24:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T01:24:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 137, 2024-04-25T01:24:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.020436941, 2024-04-25T01:24:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000149174 2024-04-25T01:24:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T01:24:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T01:24:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T01:24:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T01:24:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 102016 2024-04-25T01:24:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T01:24:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 4, 2024-04-25T01:24:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 4, 2024-04-25T01:24:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 2992, 2024-04-25T01:24:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 448, 2024-04-25T01:24:33.790 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2024-04-25T01:24:33.790 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2024-04-25T01:24:33.790 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.001497517, 2024-04-25T01:24:33.790 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000088850, 2024-04-25T01:24:33.790 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.001351613, 2024-04-25T01:24:33.790 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T01:24:33.790 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:33.790 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 3, 2024-04-25T01:24:33.790 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.001548437, 2024-04-25T01:24:33.790 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000516145 2024-04-25T01:24:33.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:33.790 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:33.790 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T01:24:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T01:24:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T01:24:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T01:24:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T01:24:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-04-25T01:24:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2024-04-25T01:24:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-04-25T01:24:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-04-25T01:24:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-04-25T01:24:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-04-25T01:24:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T01:24:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-04-25T01:24:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T01:24:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-04-25T01:24:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T01:24:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T01:24:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T01:24:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T01:24:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T01:24:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-04-25T01:24:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T01:24:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T01:24:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-04-25T01:24:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-04-25T01:24:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-04-25T01:24:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-04-25T01:24:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T01:24:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T01:24:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T01:24:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T01:24:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T01:24:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-04-25T01:24:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-04-25T01:24:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-04-25T01:24:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T01:24:33.794 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T01:24:33.794 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-04-25T01:24:33.794 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-04-25T01:24:33.794 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-04-25T01:24:33.794 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-04-25T01:24:33.794 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T01:24:33.794 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-04-25T01:24:33.794 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T01:24:33.794 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-04-25T01:24:33.794 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 151029, 2024-04-25T01:24:33.794 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 31, 2024-04-25T01:24:33.794 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-04-25T01:24:33.794 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-04-25T01:24:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-04-25T01:24:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-04-25T01:24:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-04-25T01:24:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-04-25T01:24:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-04-25T01:24:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-04-25T01:24:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2024-04-25T01:24:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2024-04-25T01:24:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T01:24:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-04-25T01:24:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-04-25T01:24:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-04-25T01:24:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 928, 2024-04-25T01:24:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2024-04-25T01:24:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-04-25T01:24:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-04-25T01:24:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-04-25T01:24:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-04-25T01:24:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T01:24:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-04-25T01:24:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-04-25T01:24:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-04-25T01:24:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 186, 2024-04-25T01:24:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 442103, 2024-04-25T01:24:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 186, 2024-04-25T01:24:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-04-25T01:24:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-04-25T01:24:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 186, 2024-04-25T01:24:33.797 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 442103, 2024-04-25T01:24:33.797 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-04-25T01:24:33.797 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T01:24:33.797 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T01:24:33.797 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T01:24:33.797 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:33.797 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:33.797 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T01:24:34.000 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:33 smithi031 ceph-mon[28053]: pgmap v649: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 87 KiB/s rd, 79 KiB/s wr, 145 op/s 2024-04-25T01:24:34.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:33 smithi169 ceph-mon[32759]: pgmap v649: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 87 KiB/s rd, 79 KiB/s wr, 145 op/s 2024-04-25T01:24:34.191 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:33 smithi040 ceph-mon[32518]: pgmap v649: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 87 KiB/s rd, 79 KiB/s wr, 145 op/s 2024-04-25T01:24:34.589 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-25T01:24:34.589 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-25T01:24:34.589 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-04-25T01:24:34.589 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T01:24:34.590 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T01:24:35.042 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:24:35.999 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:35 smithi031 ceph-mon[28053]: pgmap v650: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 77 op/s 2024-04-25T01:24:36.188 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:35 smithi169 ceph-mon[32759]: pgmap v650: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 77 op/s 2024-04-25T01:24:36.191 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:35 smithi040 ceph-mon[32518]: pgmap v650: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 77 op/s 2024-04-25T01:24:37.297 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T01:24:37.297 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T01:24:37.297 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 0, 2024-04-25T01:24:37.297 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 2, 2024-04-25T01:24:37.297 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 0, 2024-04-25T01:24:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 0, 2024-04-25T01:24:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-04-25T01:24:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2024-04-25T01:24:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.004520148, 2024-04-25T01:24:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T01:24:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.001165171, 2024-04-25T01:24:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T01:24:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T01:24:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T01:24:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T01:24:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T01:24:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T01:24:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T01:24:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T01:24:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T01:24:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T01:24:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 191, 2024-04-25T01:24:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 188, 2024-04-25T01:24:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 469872, 2024-04-25T01:24:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 115888, 2024-04-25T01:24:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2024-04-25T01:24:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2024-04-25T01:24:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.085530154, 2024-04-25T01:24:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.030009994, 2024-04-25T01:24:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.038899656, 2024-04-25T01:24:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.024024532, 2024-04-25T01:24:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 187, 2024-04-25T01:24:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.042303931, 2024-04-25T01:24:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000226224 2024-04-25T01:24:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T01:24:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T01:24:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T01:24:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 469872, 2024-04-25T01:24:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 115888 2024-04-25T01:24:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:37.300 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T01:24:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 5, 2024-04-25T01:24:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 146, 2024-04-25T01:24:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 3152, 2024-04-25T01:24:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 103344, 2024-04-25T01:24:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 3, 2024-04-25T01:24:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 2, 2024-04-25T01:24:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.030326499, 2024-04-25T01:24:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.024699399, 2024-04-25T01:24:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.003356623, 2024-04-25T01:24:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T01:24:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 145, 2024-04-25T01:24:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.028461284, 2024-04-25T01:24:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000196284 2024-04-25T01:24:37.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T01:24:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T01:24:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T01:24:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-25T01:24:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 103344 2024-04-25T01:24:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-04-25T01:24:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2024-04-25T01:24:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-04-25T01:24:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-04-25T01:24:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-04-25T01:24:37.302 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-04-25T01:24:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T01:24:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-04-25T01:24:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T01:24:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-04-25T01:24:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T01:24:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T01:24:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T01:24:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T01:24:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T01:24:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-04-25T01:24:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T01:24:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T01:24:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-04-25T01:24:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-04-25T01:24:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-04-25T01:24:37.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-04-25T01:24:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T01:24:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T01:24:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T01:24:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T01:24:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T01:24:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-04-25T01:24:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-04-25T01:24:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-04-25T01:24:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T01:24:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T01:24:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-04-25T01:24:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-04-25T01:24:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-04-25T01:24:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-04-25T01:24:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T01:24:37.304 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-04-25T01:24:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T01:24:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-04-25T01:24:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 149998, 2024-04-25T01:24:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 30, 2024-04-25T01:24:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-04-25T01:24:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-04-25T01:24:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-04-25T01:24:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-04-25T01:24:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-04-25T01:24:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-04-25T01:24:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-04-25T01:24:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-04-25T01:24:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2024-04-25T01:24:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2024-04-25T01:24:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T01:24:37.305 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-04-25T01:24:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-04-25T01:24:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-04-25T01:24:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 928, 2024-04-25T01:24:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2024-04-25T01:24:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-04-25T01:24:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-04-25T01:24:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-04-25T01:24:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-04-25T01:24:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T01:24:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T01:24:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-04-25T01:24:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-04-25T01:24:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-04-25T01:24:37.306 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 192, 2024-04-25T01:24:37.307 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 442883, 2024-04-25T01:24:37.307 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:37.307 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 192, 2024-04-25T01:24:37.307 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-04-25T01:24:37.307 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-04-25T01:24:37.307 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 192, 2024-04-25T01:24:37.307 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 442883, 2024-04-25T01:24:37.307 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-04-25T01:24:37.307 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T01:24:37.307 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T01:24:37.307 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T01:24:37.307 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:37.307 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T01:24:37.307 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T01:24:38.000 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:37 smithi031 ceph-mon[28053]: pgmap v651: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T01:24:38.117 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-25T01:24:38.117 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-25T01:24:38.117 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-04-25T01:24:38.117 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T01:24:38.117 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T01:24:38.140 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:37 smithi040 ceph-mon[32518]: pgmap v651: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T01:24:38.188 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:37 smithi169 ceph-mon[32759]: pgmap v651: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T01:24:38.363 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T01:24:40.077 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:39 smithi040 ceph-mon[32518]: pgmap v652: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-04-25T01:24:40.143 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-25T01:24:40.143 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T01:24:40.143 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 3438, 2024-04-25T01:24:40.143 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 3300, 2024-04-25T01:24:40.143 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 1329203, 2024-04-25T01:24:40.143 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 11563506, 2024-04-25T01:24:40.143 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 79, 2024-04-25T01:24:40.143 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 7, 2024-04-25T01:24:40.143 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.323750537, 2024-04-25T01:24:40.143 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.100998597, 2024-04-25T01:24:40.143 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.147431066, 2024-04-25T01:24:40.144 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.069697851, 2024-04-25T01:24:40.144 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:40.144 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 3301, 2024-04-25T01:24:40.144 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.437929192, 2024-04-25T01:24:40.144 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000132665 2024-04-25T01:24:40.144 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.144 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:40.144 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 3182, 2024-04-25T01:24:40.144 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000794978, 2024-04-25T01:24:40.144 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000249 2024-04-25T01:24:40.144 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.144 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T01:24:40.144 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T01:24:40.144 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.144 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T01:24:40.145 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 1785, 2024-04-25T01:24:40.145 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 1932, 2024-04-25T01:24:40.145 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 630755, 2024-04-25T01:24:40.145 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 1349493, 2024-04-25T01:24:40.145 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 8, 2024-04-25T01:24:40.145 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 5, 2024-04-25T01:24:40.145 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.163864692, 2024-04-25T01:24:40.145 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.055138985, 2024-04-25T01:24:40.145 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.049225223, 2024-04-25T01:24:40.145 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.046633583, 2024-04-25T01:24:40.145 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:40.145 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1934, 2024-04-25T01:24:40.145 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.668364438, 2024-04-25T01:24:40.145 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000345586 2024-04-25T01:24:40.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.146 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:40.146 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1114, 2024-04-25T01:24:40.146 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000256634, 2024-04-25T01:24:40.146 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000230 2024-04-25T01:24:40.146 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.146 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-25T01:24:40.146 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 330448 2024-04-25T01:24:40.146 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.146 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T01:24:40.146 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 10166, 2024-04-25T01:24:40.146 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 12773, 2024-04-25T01:24:40.146 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 3882616, 2024-04-25T01:24:40.146 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 11590418, 2024-04-25T01:24:40.146 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 14, 2024-04-25T01:24:40.147 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 8, 2024-04-25T01:24:40.147 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.815677786, 2024-04-25T01:24:40.147 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.299685439, 2024-04-25T01:24:40.147 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.473813298, 2024-04-25T01:24:40.147 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.087231421, 2024-04-25T01:24:40.147 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:40.147 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 12766, 2024-04-25T01:24:40.147 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 1.707814373, 2024-04-25T01:24:40.147 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000133778 2024-04-25T01:24:40.147 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.147 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:40.147 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 7759, 2024-04-25T01:24:40.147 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.003640678, 2024-04-25T01:24:40.148 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000469 2024-04-25T01:24:40.148 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.148 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 609104, 2024-04-25T01:24:40.148 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 148480 2024-04-25T01:24:40.148 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.148 INFO:teuthology.orchestra.run.smithi040.stdout: "cct": { 2024-04-25T01:24:40.148 INFO:teuthology.orchestra.run.smithi040.stdout: "total_workers": 1, 2024-04-25T01:24:40.148 INFO:teuthology.orchestra.run.smithi040.stdout: "unhealthy_workers": 0 2024-04-25T01:24:40.148 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.148 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-MDSRank": { 2024-04-25T01:24:40.148 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2024-04-25T01:24:40.148 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2024-04-25T01:24:40.148 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 2087, 2024-04-25T01:24:40.148 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 2.913051219, 2024-04-25T01:24:40.149 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.001395807 2024-04-25T01:24:40.149 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:40.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.149 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-PurgeQueue": { 2024-04-25T01:24:40.149 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2024-04-25T01:24:40.149 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2024-04-25T01:24:40.149 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1973, 2024-04-25T01:24:40.149 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.093167566, 2024-04-25T01:24:40.149 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000047221 2024-04-25T01:24:40.149 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:40.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.149 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-04-25T01:24:40.149 INFO:teuthology.orchestra.run.smithi040.stdout: "request": 1897, 2024-04-25T01:24:40.149 INFO:teuthology.orchestra.run.smithi040.stdout: "reply": 1894, 2024-04-25T01:24:40.149 INFO:teuthology.orchestra.run.smithi040.stdout: "reply_latency": { 2024-04-25T01:24:40.149 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1894, 2024-04-25T01:24:40.150 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 15.850730908, 2024-04-25T01:24:40.150 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.008368918 2024-04-25T01:24:40.150 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.150 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_reply": 0, 2024-04-25T01:24:40.150 INFO:teuthology.orchestra.run.smithi040.stdout: "forward": 3, 2024-04-25T01:24:40.150 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_complete": 0, 2024-04-25T01:24:40.150 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_keys": 0, 2024-04-25T01:24:40.150 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_commit": 418, 2024-04-25T01:24:40.150 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_split": 0, 2024-04-25T01:24:40.150 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_merge": 0, 2024-04-25T01:24:40.150 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes": 15, 2024-04-25T01:24:40.150 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_top": 0, 2024-04-25T01:24:40.150 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_bottom": 5, 2024-04-25T01:24:40.150 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pin_tail": 10, 2024-04-25T01:24:40.151 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pinned": 14, 2024-04-25T01:24:40.151 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_expired": 709, 2024-04-25T01:24:40.151 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_with_caps": 0, 2024-04-25T01:24:40.151 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": 0, 2024-04-25T01:24:40.151 INFO:teuthology.orchestra.run.smithi040.stdout: "subtrees": 4, 2024-04-25T01:24:40.151 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse": 2570, 2024-04-25T01:24:40.151 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_hit": 2398, 2024-04-25T01:24:40.151 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_forward": 1, 2024-04-25T01:24:40.151 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_discover": 2, 2024-04-25T01:24:40.151 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_dir_fetch": 0, 2024-04-25T01:24:40.151 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_remote_ino": 0, 2024-04-25T01:24:40.151 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_lock": 48, 2024-04-25T01:24:40.151 INFO:teuthology.orchestra.run.smithi040.stdout: "load_cent": 12, 2024-04-25T01:24:40.151 INFO:teuthology.orchestra.run.smithi040.stdout: "q": 0, 2024-04-25T01:24:40.152 INFO:teuthology.orchestra.run.smithi040.stdout: "exported": 9, 2024-04-25T01:24:40.152 INFO:teuthology.orchestra.run.smithi040.stdout: "exported_inodes": 736, 2024-04-25T01:24:40.152 INFO:teuthology.orchestra.run.smithi040.stdout: "imported": 7, 2024-04-25T01:24:40.152 INFO:teuthology.orchestra.run.smithi040.stdout: "imported_inodes": 328, 2024-04-25T01:24:40.152 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_dir_fetch": 0, 2024-04-25T01:24:40.152 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_backtrace_fetch": 0, 2024-04-25T01:24:40.152 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_peer_discover": 0, 2024-04-25T01:24:40.152 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rfiles": 4, 2024-04-25T01:24:40.152 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rbytes": 234, 2024-04-25T01:24:40.152 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rsnaps": 0, 2024-04-25T01:24:40.152 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T01:24:40.152 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_set_tag": 0, 2024-04-25T01:24:40.152 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T01:24:40.152 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_inotable_repaired": 0, 2024-04-25T01:24:40.152 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_inodes": 0, 2024-04-25T01:24:40.152 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T01:24:40.153 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T01:24:40.153 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_file_inodes": 0, 2024-04-25T01:24:40.153 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_inode_file_caps": 3, 2024-04-25T01:24:40.153 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_revoke": 219, 2024-04-25T01:24:40.153 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_grant": 2058, 2024-04-25T01:24:40.153 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_trunc": 6, 2024-04-25T01:24:40.153 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T01:24:40.153 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T01:24:40.153 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps": 5635, 2024-04-25T01:24:40.153 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps_dirty": 4658, 2024-04-25T01:24:40.153 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_cap_release": 132, 2024-04-25T01:24:40.153 INFO:teuthology.orchestra.run.smithi040.stdout: "process_request_cap_release": 964 2024-04-25T01:24:40.153 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.154 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_cache": { 2024-04-25T01:24:40.154 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update": 2, 2024-04-25T01:24:40.154 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update_receipt": 0, 2024-04-25T01:24:40.154 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_try_discover": 0, 2024-04-25T01:24:40.154 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_send_discover": 2, 2024-04-25T01:24:40.154 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_handle_discover": 15, 2024-04-25T01:24:40.154 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays": 0, 2024-04-25T01:24:40.154 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_delayed": 0, 2024-04-25T01:24:40.154 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_enqueuing": 0, 2024-04-25T01:24:40.154 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_created": 553, 2024-04-25T01:24:40.154 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_enqueued": 492, 2024-04-25T01:24:40.154 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_reintegrated": 61, 2024-04-25T01:24:40.154 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_migrated": 0, 2024-04-25T01:24:40.154 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_processing": 0, 2024-04-25T01:24:40.155 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_enqueued": 0, 2024-04-25T01:24:40.155 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_prioritized": 0, 2024-04-25T01:24:40.155 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_started": 0, 2024-04-25T01:24:40.155 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_completed": 0, 2024-04-25T01:24:40.155 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_quiesce_path": 0, 2024-04-25T01:24:40.155 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_quiesce_inode": 0, 2024-04-25T01:24:40.155 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T01:24:40.155 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_exportdir": 9, 2024-04-25T01:24:40.155 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_flush": 0, 2024-04-25T01:24:40.155 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragmentdir": 0, 2024-04-25T01:24:40.155 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragstats": 0, 2024-04-25T01:24:40.155 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_inodestats": 0 2024-04-25T01:24:40.155 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.155 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_log": { 2024-04-25T01:24:40.155 INFO:teuthology.orchestra.run.smithi040.stdout: "evlrg": 0, 2024-04-25T01:24:40.156 INFO:teuthology.orchestra.run.smithi040.stdout: "evadd": 7504, 2024-04-25T01:24:40.156 INFO:teuthology.orchestra.run.smithi040.stdout: "evex": 7359, 2024-04-25T01:24:40.156 INFO:teuthology.orchestra.run.smithi040.stdout: "evtrm": 7359, 2024-04-25T01:24:40.156 INFO:teuthology.orchestra.run.smithi040.stdout: "ev": 145, 2024-04-25T01:24:40.156 INFO:teuthology.orchestra.run.smithi040.stdout: "evexg": 0, 2024-04-25T01:24:40.156 INFO:teuthology.orchestra.run.smithi040.stdout: "evexd": 23, 2024-04-25T01:24:40.156 INFO:teuthology.orchestra.run.smithi040.stdout: "segadd": 41, 2024-04-25T01:24:40.156 INFO:teuthology.orchestra.run.smithi040.stdout: "segex": 39, 2024-04-25T01:24:40.156 INFO:teuthology.orchestra.run.smithi040.stdout: "segtrm": 39, 2024-04-25T01:24:40.156 INFO:teuthology.orchestra.run.smithi040.stdout: "seg": 2, 2024-04-25T01:24:40.156 INFO:teuthology.orchestra.run.smithi040.stdout: "segmjr": 1, 2024-04-25T01:24:40.156 INFO:teuthology.orchestra.run.smithi040.stdout: "segexg": 0, 2024-04-25T01:24:40.156 INFO:teuthology.orchestra.run.smithi040.stdout: "segexd": 1, 2024-04-25T01:24:40.156 INFO:teuthology.orchestra.run.smithi040.stdout: "expos": 16670410, 2024-04-25T01:24:40.157 INFO:teuthology.orchestra.run.smithi040.stdout: "wrpos": 16886675, 2024-04-25T01:24:40.157 INFO:teuthology.orchestra.run.smithi040.stdout: "rdpos": 0, 2024-04-25T01:24:40.157 INFO:teuthology.orchestra.run.smithi040.stdout: "jlat": { 2024-04-25T01:24:40.157 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 729, 2024-04-25T01:24:40.157 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 18.606575287, 2024-04-25T01:24:40.157 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.025523422 2024-04-25T01:24:40.157 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.157 INFO:teuthology.orchestra.run.smithi040.stdout: "replayed": 0 2024-04-25T01:24:40.157 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.157 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_mem": { 2024-04-25T01:24:40.157 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 20, 2024-04-25T01:24:40.157 INFO:teuthology.orchestra.run.smithi040.stdout: "ino+": 1002, 2024-04-25T01:24:40.157 INFO:teuthology.orchestra.run.smithi040.stdout: "ino-": 982, 2024-04-25T01:24:40.157 INFO:teuthology.orchestra.run.smithi040.stdout: "dir": 21, 2024-04-25T01:24:40.158 INFO:teuthology.orchestra.run.smithi040.stdout: "dir+": 195, 2024-04-25T01:24:40.158 INFO:teuthology.orchestra.run.smithi040.stdout: "dir-": 174, 2024-04-25T01:24:40.158 INFO:teuthology.orchestra.run.smithi040.stdout: "dn": 57, 2024-04-25T01:24:40.158 INFO:teuthology.orchestra.run.smithi040.stdout: "dn+": 1405, 2024-04-25T01:24:40.158 INFO:teuthology.orchestra.run.smithi040.stdout: "dn-": 1348, 2024-04-25T01:24:40.158 INFO:teuthology.orchestra.run.smithi040.stdout: "cap": 0, 2024-04-25T01:24:40.158 INFO:teuthology.orchestra.run.smithi040.stdout: "cap+": 970, 2024-04-25T01:24:40.158 INFO:teuthology.orchestra.run.smithi040.stdout: "cap-": 970, 2024-04-25T01:24:40.158 INFO:teuthology.orchestra.run.smithi040.stdout: "rss": 64596, 2024-04-25T01:24:40.158 INFO:teuthology.orchestra.run.smithi040.stdout: "heap": 231804 2024-04-25T01:24:40.158 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.158 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_server": { 2024-04-25T01:24:40.158 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_client_request": 2320, 2024-04-25T01:24:40.158 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_server_request": 42, 2024-04-25T01:24:40.159 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_request": 1897, 2024-04-25T01:24:40.159 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_session": 20, 2024-04-25T01:24:40.159 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_peer_request": 130, 2024-04-25T01:24:40.159 INFO:teuthology.orchestra.run.smithi040.stdout: "req_create_latency": { 2024-04-25T01:24:40.159 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 632, 2024-04-25T01:24:40.159 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.746521212, 2024-04-25T01:24:40.159 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.001181204 2024-04-25T01:24:40.159 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.159 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getattr_latency": { 2024-04-25T01:24:40.159 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 77, 2024-04-25T01:24:40.159 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.014822929, 2024-04-25T01:24:40.159 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000192505 2024-04-25T01:24:40.159 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.159 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getfilelock_latency": { 2024-04-25T01:24:40.159 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:40.159 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:40.160 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:40.160 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.160 INFO:teuthology.orchestra.run.smithi040.stdout: "req_link_latency": { 2024-04-25T01:24:40.160 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 61, 2024-04-25T01:24:40.160 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.051969663, 2024-04-25T01:24:40.160 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000851961 2024-04-25T01:24:40.160 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.160 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookup_latency": { 2024-04-25T01:24:40.160 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 142, 2024-04-25T01:24:40.160 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.093317461, 2024-04-25T01:24:40.160 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000657165 2024-04-25T01:24:40.160 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.160 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookuphash_latency": { 2024-04-25T01:24:40.160 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:40.160 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:40.160 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:40.160 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.161 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupino_latency": { 2024-04-25T01:24:40.161 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:40.161 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:40.161 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:40.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.161 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupname_latency": { 2024-04-25T01:24:40.161 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:40.161 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:40.161 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:40.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.161 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupparent_latency": { 2024-04-25T01:24:40.161 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:40.161 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:40.161 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:40.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.161 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupsnap_latency": { 2024-04-25T01:24:40.162 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:40.162 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:40.162 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:40.162 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.162 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lssnap_latency": { 2024-04-25T01:24:40.162 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:40.162 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:40.163 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:40.163 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.163 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mkdir_latency": { 2024-04-25T01:24:40.163 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 105, 2024-04-25T01:24:40.163 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.111148653, 2024-04-25T01:24:40.163 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.001058558 2024-04-25T01:24:40.163 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.163 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mknod_latency": { 2024-04-25T01:24:40.163 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:40.163 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:40.163 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:40.163 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.163 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mksnap_latency": { 2024-04-25T01:24:40.163 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:40.164 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:40.164 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:40.164 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.164 INFO:teuthology.orchestra.run.smithi040.stdout: "req_open_latency": { 2024-04-25T01:24:40.164 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 49, 2024-04-25T01:24:40.164 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.012363919, 2024-04-25T01:24:40.164 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000252324 2024-04-25T01:24:40.164 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.164 INFO:teuthology.orchestra.run.smithi040.stdout: "req_readdir_latency": { 2024-04-25T01:24:40.164 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 164, 2024-04-25T01:24:40.164 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 1.049336456, 2024-04-25T01:24:40.164 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.006398393 2024-04-25T01:24:40.164 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.164 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rename_latency": { 2024-04-25T01:24:40.165 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 95, 2024-04-25T01:24:40.165 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 12.696977020, 2024-04-25T01:24:40.165 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.133652389 2024-04-25T01:24:40.165 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.165 INFO:teuthology.orchestra.run.smithi040.stdout: "req_renamesnap_latency": { 2024-04-25T01:24:40.165 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:40.165 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:40.165 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:40.165 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.165 INFO:teuthology.orchestra.run.smithi040.stdout: "req_snapdiff_latency": { 2024-04-25T01:24:40.165 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:40.165 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:40.165 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:40.165 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.166 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmdir_latency": { 2024-04-25T01:24:40.166 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 16, 2024-04-25T01:24:40.166 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.168766209, 2024-04-25T01:24:40.166 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.010547888 2024-04-25T01:24:40.166 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.166 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmsnap_latency": { 2024-04-25T01:24:40.166 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:40.166 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:40.166 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:40.166 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.166 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmxattr_latency": { 2024-04-25T01:24:40.166 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:40.166 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:40.166 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:40.167 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.167 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setattr_latency": { 2024-04-25T01:24:40.167 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 7, 2024-04-25T01:24:40.167 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.033879472, 2024-04-25T01:24:40.167 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.004839924 2024-04-25T01:24:40.167 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.167 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setdirlayout_latency": { 2024-04-25T01:24:40.167 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:40.167 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:40.167 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:40.167 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.167 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setfilelock_latency": { 2024-04-25T01:24:40.167 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 4, 2024-04-25T01:24:40.167 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.001184885, 2024-04-25T01:24:40.167 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000296221 2024-04-25T01:24:40.167 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.168 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setlayout_latency": { 2024-04-25T01:24:40.168 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:40.168 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:40.168 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:40.168 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.168 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setxattr_latency": { 2024-04-25T01:24:40.168 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 7, 2024-04-25T01:24:40.168 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.022296939, 2024-04-25T01:24:40.168 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.003185277 2024-04-25T01:24:40.168 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.168 INFO:teuthology.orchestra.run.smithi040.stdout: "req_symlink_latency": { 2024-04-25T01:24:40.168 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 13, 2024-04-25T01:24:40.168 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.011150571, 2024-04-25T01:24:40.168 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000857736 2024-04-25T01:24:40.168 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.168 INFO:teuthology.orchestra.run.smithi040.stdout: "req_unlink_latency": { 2024-04-25T01:24:40.169 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 522, 2024-04-25T01:24:40.169 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.836995519, 2024-04-25T01:24:40.169 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.001603439 2024-04-25T01:24:40.169 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.169 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_revoke_eviction": 0, 2024-04-25T01:24:40.169 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_acquisition_throttle": 0, 2024-04-25T01:24:40.169 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getvxattr_latency": { 2024-04-25T01:24:40.169 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:40.169 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:40.169 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:40.169 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:40.169 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.169 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_sessions": { 2024-04-25T01:24:40.169 INFO:teuthology.orchestra.run.smithi040.stdout: "session_count": 1, 2024-04-25T01:24:40.169 INFO:teuthology.orchestra.run.smithi040.stdout: "session_add": 2, 2024-04-25T01:24:40.170 INFO:teuthology.orchestra.run.smithi040.stdout: "session_remove": 1, 2024-04-25T01:24:40.170 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_open": 1, 2024-04-25T01:24:40.170 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_stale": 0, 2024-04-25T01:24:40.170 INFO:teuthology.orchestra.run.smithi040.stdout: "total_load": 105, 2024-04-25T01:24:40.170 INFO:teuthology.orchestra.run.smithi040.stdout: "average_load": 105, 2024-04-25T01:24:40.170 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_session_uptime": 521, 2024-04-25T01:24:40.170 INFO:teuthology.orchestra.run.smithi040.stdout: "mdthresh_evicted": 0 2024-04-25T01:24:40.170 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.170 INFO:teuthology.orchestra.run.smithi040.stdout: "mempool": { 2024-04-25T01:24:40.170 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_bytes": 0, 2024-04-25T01:24:40.170 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_items": 0, 2024-04-25T01:24:40.170 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T01:24:40.171 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_items": 0, 2024-04-25T01:24:40.171 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T01:24:40.171 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_items": 0, 2024-04-25T01:24:40.171 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T01:24:40.171 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T01:24:40.171 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T01:24:40.171 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T01:24:40.171 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T01:24:40.171 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_items": 0, 2024-04-25T01:24:40.171 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T01:24:40.171 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T01:24:40.171 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_bytes": 0, 2024-04-25T01:24:40.173 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_items": 0, 2024-04-25T01:24:40.173 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_bytes": 0, 2024-04-25T01:24:40.173 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_items": 0, 2024-04-25T01:24:40.173 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T01:24:40.173 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T01:24:40.173 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T01:24:40.173 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T01:24:40.173 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T01:24:40.173 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_items": 0, 2024-04-25T01:24:40.173 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_bytes": 0, 2024-04-25T01:24:40.174 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_items": 0, 2024-04-25T01:24:40.174 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T01:24:40.174 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T01:24:40.174 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_bytes": 0, 2024-04-25T01:24:40.174 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_items": 0, 2024-04-25T01:24:40.174 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_bytes": 0, 2024-04-25T01:24:40.174 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_items": 0, 2024-04-25T01:24:40.174 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T01:24:40.174 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_items": 0, 2024-04-25T01:24:40.174 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T01:24:40.174 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_items": 0, 2024-04-25T01:24:40.174 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_bytes": 278922, 2024-04-25T01:24:40.174 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_items": 98, 2024-04-25T01:24:40.175 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_bytes": 0, 2024-04-25T01:24:40.175 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_items": 0, 2024-04-25T01:24:40.175 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_bytes": 0, 2024-04-25T01:24:40.175 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_items": 0, 2024-04-25T01:24:40.175 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_bytes": 0, 2024-04-25T01:24:40.175 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_items": 0, 2024-04-25T01:24:40.175 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_bytes": 0, 2024-04-25T01:24:40.175 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_items": 0, 2024-04-25T01:24:40.175 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_bytes": 7344, 2024-04-25T01:24:40.175 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_items": 155, 2024-04-25T01:24:40.175 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T01:24:40.175 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_items": 0, 2024-04-25T01:24:40.175 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_bytes": 0, 2024-04-25T01:24:40.176 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_items": 0, 2024-04-25T01:24:40.176 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_bytes": 118814, 2024-04-25T01:24:40.176 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_items": 1046, 2024-04-25T01:24:40.176 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_bytes": 0, 2024-04-25T01:24:40.176 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_items": 0, 2024-04-25T01:24:40.176 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_bytes": 0, 2024-04-25T01:24:40.176 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_items": 0 2024-04-25T01:24:40.176 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.176 INFO:teuthology.orchestra.run.smithi040.stdout: "objecter": { 2024-04-25T01:24:40.176 INFO:teuthology.orchestra.run.smithi040.stdout: "op_active": 0, 2024-04-25T01:24:40.176 INFO:teuthology.orchestra.run.smithi040.stdout: "op_laggy": 0, 2024-04-25T01:24:40.176 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send": 4170, 2024-04-25T01:24:40.176 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send_bytes": 14298840, 2024-04-25T01:24:40.176 INFO:teuthology.orchestra.run.smithi040.stdout: "op_resend": 32, 2024-04-25T01:24:40.177 INFO:teuthology.orchestra.run.smithi040.stdout: "op_reply": 4170, 2024-04-25T01:24:40.177 INFO:teuthology.orchestra.run.smithi040.stdout: "op_latency": { 2024-04-25T01:24:40.177 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 4170, 2024-04-25T01:24:40.177 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 140.862411467, 2024-04-25T01:24:40.177 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.033779954 2024-04-25T01:24:40.177 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.177 INFO:teuthology.orchestra.run.smithi040.stdout: "op_inflight": 0, 2024-04-25T01:24:40.177 INFO:teuthology.orchestra.run.smithi040.stdout: "oplen_avg": { 2024-04-25T01:24:40.177 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 4170, 2024-04-25T01:24:40.177 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 6228 2024-04-25T01:24:40.177 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.177 INFO:teuthology.orchestra.run.smithi040.stdout: "op": 4170, 2024-04-25T01:24:40.177 INFO:teuthology.orchestra.run.smithi040.stdout: "op_r": 25, 2024-04-25T01:24:40.177 INFO:teuthology.orchestra.run.smithi040.stdout: "op_w": 4145, 2024-04-25T01:24:40.177 INFO:teuthology.orchestra.run.smithi040.stdout: "op_rmw": 0, 2024-04-25T01:24:40.177 INFO:teuthology.orchestra.run.smithi040.stdout: "op_pg": 0, 2024-04-25T01:24:40.178 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_stat": 320, 2024-04-25T01:24:40.178 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_create": 530, 2024-04-25T01:24:40.178 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_read": 25, 2024-04-25T01:24:40.178 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_write": 754, 2024-04-25T01:24:40.178 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writefull": 132, 2024-04-25T01:24:40.178 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writesame": 0, 2024-04-25T01:24:40.178 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_append": 0, 2024-04-25T01:24:40.178 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_zero": 0, 2024-04-25T01:24:40.178 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_truncate": 0, 2024-04-25T01:24:40.178 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_delete": 2071, 2024-04-25T01:24:40.178 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_mapext": 0, 2024-04-25T01:24:40.178 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_sparse_read": 0, 2024-04-25T01:24:40.178 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_clonerange": 0, 2024-04-25T01:24:40.178 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_getxattr": 0, 2024-04-25T01:24:40.179 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_setxattr": 1060, 2024-04-25T01:24:40.179 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_cmpxattr": 0, 2024-04-25T01:24:40.179 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_rmxattr": 0, 2024-04-25T01:24:40.179 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_resetxattrs": 0, 2024-04-25T01:24:40.179 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_call": 0, 2024-04-25T01:24:40.179 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_watch": 0, 2024-04-25T01:24:40.179 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_notify": 0, 2024-04-25T01:24:40.179 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T01:24:40.179 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls": 0, 2024-04-25T01:24:40.179 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls_filter": 0, 2024-04-25T01:24:40.179 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_other": 6, 2024-04-25T01:24:40.179 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_active": 0, 2024-04-25T01:24:40.179 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_send": 0, 2024-04-25T01:24:40.179 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_resend": 0, 2024-04-25T01:24:40.180 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_ping": 0, 2024-04-25T01:24:40.180 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_active": 0, 2024-04-25T01:24:40.180 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_send": 0, 2024-04-25T01:24:40.180 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_resend": 0, 2024-04-25T01:24:40.180 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_active": 0, 2024-04-25T01:24:40.180 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_send": 0, 2024-04-25T01:24:40.180 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_resend": 0, 2024-04-25T01:24:40.180 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_active": 0, 2024-04-25T01:24:40.180 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_send": 0, 2024-04-25T01:24:40.180 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_resend": 0, 2024-04-25T01:24:40.180 INFO:teuthology.orchestra.run.smithi040.stdout: "command_active": 0, 2024-04-25T01:24:40.180 INFO:teuthology.orchestra.run.smithi040.stdout: "command_send": 0, 2024-04-25T01:24:40.180 INFO:teuthology.orchestra.run.smithi040.stdout: "command_resend": 0, 2024-04-25T01:24:40.180 INFO:teuthology.orchestra.run.smithi040.stdout: "map_epoch": 89, 2024-04-25T01:24:40.180 INFO:teuthology.orchestra.run.smithi040.stdout: "map_full": 0, 2024-04-25T01:24:40.181 INFO:teuthology.orchestra.run.smithi040.stdout: "map_inc": 3, 2024-04-25T01:24:40.181 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_sessions": 12, 2024-04-25T01:24:40.181 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_open": 12, 2024-04-25T01:24:40.181 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_close": 0, 2024-04-25T01:24:40.181 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_laggy": 0, 2024-04-25T01:24:40.181 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_wr": 1155, 2024-04-25T01:24:40.181 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_rd": 0, 2024-04-25T01:24:40.181 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_del": 175 2024-04-25T01:24:40.181 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.181 INFO:teuthology.orchestra.run.smithi040.stdout: "oft": { 2024-04-25T01:24:40.181 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_objs": 1, 2024-04-25T01:24:40.181 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_kv_pairs": 0, 2024-04-25T01:24:40.181 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_updates": 0, 2024-04-25T01:24:40.181 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_removes": 0 2024-04-25T01:24:40.181 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.181 INFO:teuthology.orchestra.run.smithi040.stdout: "osd-slow-ops": { 2024-04-25T01:24:40.182 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_ops_count": 0 2024-04-25T01:24:40.182 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.182 INFO:teuthology.orchestra.run.smithi040.stdout: "purge_queue": { 2024-04-25T01:24:40.182 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops": 0, 2024-04-25T01:24:40.182 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops_high_water": 51, 2024-04-25T01:24:40.182 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing": 0, 2024-04-25T01:24:40.182 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_high_water": 10, 2024-04-25T01:24:40.182 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executed_ops": 1554, 2024-04-25T01:24:40.182 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executed": 492, 2024-04-25T01:24:40.182 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_item_in_journal": 0 2024-04-25T01:24:40.182 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.182 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T01:24:40.182 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T01:24:40.182 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2024-04-25T01:24:40.183 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T01:24:40.183 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 15385, 2024-04-25T01:24:40.183 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 4572645, 2024-04-25T01:24:40.183 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:40.183 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 15385, 2024-04-25T01:24:40.183 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-04-25T01:24:40.183 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-04-25T01:24:40.183 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 15385, 2024-04-25T01:24:40.183 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 4572645, 2024-04-25T01:24:40.183 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T01:24:40.183 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:40.183 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:40.183 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:40.183 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:40.183 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.184 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_bytes": { 2024-04-25T01:24:40.184 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T01:24:40.184 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2024-04-25T01:24:40.184 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T01:24:40.184 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-04-25T01:24:40.184 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-04-25T01:24:40.184 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:40.184 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-04-25T01:24:40.184 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 4170, 2024-04-25T01:24:40.184 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 14348532, 2024-04-25T01:24:40.184 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 4170, 2024-04-25T01:24:40.184 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 14348532, 2024-04-25T01:24:40.184 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T01:24:40.184 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:40.184 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:40.185 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:40.185 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:40.185 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.185 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_ops": { 2024-04-25T01:24:40.185 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T01:24:40.185 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 1024, 2024-04-25T01:24:40.185 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T01:24:40.185 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-04-25T01:24:40.185 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-04-25T01:24:40.185 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:40.185 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-04-25T01:24:40.185 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 4170, 2024-04-25T01:24:40.185 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 4170, 2024-04-25T01:24:40.185 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 4170, 2024-04-25T01:24:40.185 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 4170, 2024-04-25T01:24:40.185 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T01:24:40.185 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:40.186 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:40.186 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:40.186 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:40.186 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.186 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle": { 2024-04-25T01:24:40.186 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T01:24:40.186 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2024-04-25T01:24:40.186 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T01:24:40.186 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 492, 2024-04-25T01:24:40.186 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 49692, 2024-04-25T01:24:40.186 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:40.186 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 492, 2024-04-25T01:24:40.186 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-04-25T01:24:40.186 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-04-25T01:24:40.187 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 25, 2024-04-25T01:24:40.187 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 49692, 2024-04-25T01:24:40.187 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T01:24:40.187 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:40.187 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:40.187 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:40.187 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:40.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:40.187 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle-0x556e8d158500": { 2024-04-25T01:24:40.187 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T01:24:40.187 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2024-04-25T01:24:40.187 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T01:24:40.187 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 7504, 2024-04-25T01:24:40.187 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 12692371, 2024-04-25T01:24:40.187 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:40.188 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 7504, 2024-04-25T01:24:40.188 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-04-25T01:24:40.188 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-04-25T01:24:40.188 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 729, 2024-04-25T01:24:40.188 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 12692371, 2024-04-25T01:24:40.188 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T01:24:40.188 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:40.188 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:40.188 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:40.188 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:40.188 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:40.188 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-25T01:24:40.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:39 smithi169 ceph-mon[32759]: pgmap v652: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-04-25T01:24:40.249 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:39 smithi031 ceph-mon[28053]: pgmap v652: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-04-25T01:24:40.970 INFO:tasks.check_counter:Daemon mds.b mds.exported=9 2024-04-25T01:24:40.970 INFO:tasks.check_counter:Daemon mds.b mds.imported=7 2024-04-25T01:24:40.970 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=0 2024-04-25T01:24:40.971 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T01:24:40.971 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T01:24:41.261 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T01:24:42.073 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:41 smithi040 ceph-mon[32518]: pgmap v653: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-25T01:24:42.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:41 smithi169 ceph-mon[32759]: pgmap v653: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-25T01:24:42.249 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:41 smithi031 ceph-mon[28053]: pgmap v653: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-25T01:24:43.044 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-25T01:24:43.045 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T01:24:43.045 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 214, 2024-04-25T01:24:43.045 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 198, 2024-04-25T01:24:43.045 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 535586, 2024-04-25T01:24:43.045 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 117844, 2024-04-25T01:24:43.045 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 5, 2024-04-25T01:24:43.045 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 1, 2024-04-25T01:24:43.045 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.030914612, 2024-04-25T01:24:43.045 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.010628423, 2024-04-25T01:24:43.045 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.015254368, 2024-04-25T01:24:43.045 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.004921343, 2024-04-25T01:24:43.045 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:43.046 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 198, 2024-04-25T01:24:43.046 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.012699052, 2024-04-25T01:24:43.046 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000064136 2024-04-25T01:24:43.046 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.046 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:43.046 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.046 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.046 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.046 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.046 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 534752, 2024-04-25T01:24:43.046 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 117520 2024-04-25T01:24:43.046 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.046 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T01:24:43.046 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 558, 2024-04-25T01:24:43.047 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 701, 2024-04-25T01:24:43.047 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 2562649, 2024-04-25T01:24:43.047 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 582139, 2024-04-25T01:24:43.047 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 6, 2024-04-25T01:24:43.047 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 4, 2024-04-25T01:24:43.047 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.067587904, 2024-04-25T01:24:43.047 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.019316313, 2024-04-25T01:24:43.047 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.019112677, 2024-04-25T01:24:43.047 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.023069120, 2024-04-25T01:24:43.047 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:43.047 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 699, 2024-04-25T01:24:43.047 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.041125114, 2024-04-25T01:24:43.047 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000058834 2024-04-25T01:24:43.047 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T01:24:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 405008 2024-04-25T01:24:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T01:24:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 1252, 2024-04-25T01:24:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 1254, 2024-04-25T01:24:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 4562275, 2024-04-25T01:24:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 398194, 2024-04-25T01:24:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 5, 2024-04-25T01:24:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 4, 2024-04-25T01:24:43.049 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.128938850, 2024-04-25T01:24:43.049 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.039525243, 2024-04-25T01:24:43.049 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.036215519, 2024-04-25T01:24:43.049 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.042899281, 2024-04-25T01:24:43.049 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:43.049 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1252, 2024-04-25T01:24:43.049 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.057900521, 2024-04-25T01:24:43.049 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000046246 2024-04-25T01:24:43.049 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.049 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:43.049 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.049 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.049 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.049 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.049 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T01:24:43.050 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T01:24:43.050 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.050 INFO:teuthology.orchestra.run.smithi040.stdout: "cct": { 2024-04-25T01:24:43.050 INFO:teuthology.orchestra.run.smithi040.stdout: "total_workers": 1, 2024-04-25T01:24:43.050 INFO:teuthology.orchestra.run.smithi040.stdout: "unhealthy_workers": 0 2024-04-25T01:24:43.050 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.050 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-MDSRank": { 2024-04-25T01:24:43.050 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2024-04-25T01:24:43.050 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2024-04-25T01:24:43.050 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 3113, 2024-04-25T01:24:43.050 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.083394554, 2024-04-25T01:24:43.050 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000026789 2024-04-25T01:24:43.050 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:43.051 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.051 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-PurgeQueue": { 2024-04-25T01:24:43.051 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2024-04-25T01:24:43.051 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2024-04-25T01:24:43.051 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.051 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.051 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.051 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:43.051 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.051 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-04-25T01:24:43.051 INFO:teuthology.orchestra.run.smithi040.stdout: "request": 0, 2024-04-25T01:24:43.051 INFO:teuthology.orchestra.run.smithi040.stdout: "reply": 0, 2024-04-25T01:24:43.051 INFO:teuthology.orchestra.run.smithi040.stdout: "reply_latency": { 2024-04-25T01:24:43.051 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.051 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.051 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.052 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.052 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_reply": 0, 2024-04-25T01:24:43.052 INFO:teuthology.orchestra.run.smithi040.stdout: "forward": 0, 2024-04-25T01:24:43.052 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_complete": 0, 2024-04-25T01:24:43.052 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_keys": 0, 2024-04-25T01:24:43.052 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_commit": 0, 2024-04-25T01:24:43.052 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_split": 0, 2024-04-25T01:24:43.052 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_merge": 0, 2024-04-25T01:24:43.052 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes": 51, 2024-04-25T01:24:43.052 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_top": 0, 2024-04-25T01:24:43.052 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_bottom": 51, 2024-04-25T01:24:43.052 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pin_tail": 0, 2024-04-25T01:24:43.052 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pinned": 51, 2024-04-25T01:24:43.052 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_expired": 0, 2024-04-25T01:24:43.053 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_with_caps": 0, 2024-04-25T01:24:43.053 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": 0, 2024-04-25T01:24:43.053 INFO:teuthology.orchestra.run.smithi040.stdout: "subtrees": 1, 2024-04-25T01:24:43.053 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse": 0, 2024-04-25T01:24:43.053 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_hit": 0, 2024-04-25T01:24:43.053 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_forward": 0, 2024-04-25T01:24:43.053 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_discover": 0, 2024-04-25T01:24:43.053 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_dir_fetch": 0, 2024-04-25T01:24:43.053 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_remote_ino": 0, 2024-04-25T01:24:43.053 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_lock": 0, 2024-04-25T01:24:43.053 INFO:teuthology.orchestra.run.smithi040.stdout: "load_cent": 0, 2024-04-25T01:24:43.053 INFO:teuthology.orchestra.run.smithi040.stdout: "q": 0, 2024-04-25T01:24:43.053 INFO:teuthology.orchestra.run.smithi040.stdout: "exported": 0, 2024-04-25T01:24:43.053 INFO:teuthology.orchestra.run.smithi040.stdout: "exported_inodes": 0, 2024-04-25T01:24:43.054 INFO:teuthology.orchestra.run.smithi040.stdout: "imported": 0, 2024-04-25T01:24:43.054 INFO:teuthology.orchestra.run.smithi040.stdout: "imported_inodes": 0, 2024-04-25T01:24:43.054 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_dir_fetch": 0, 2024-04-25T01:24:43.054 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_backtrace_fetch": 0, 2024-04-25T01:24:43.054 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_peer_discover": 0, 2024-04-25T01:24:43.054 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rfiles": 0, 2024-04-25T01:24:43.054 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rbytes": 0, 2024-04-25T01:24:43.054 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rsnaps": 0, 2024-04-25T01:24:43.054 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T01:24:43.054 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_set_tag": 0, 2024-04-25T01:24:43.054 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T01:24:43.054 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_inotable_repaired": 0, 2024-04-25T01:24:43.054 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_inodes": 0, 2024-04-25T01:24:43.055 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T01:24:43.055 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T01:24:43.055 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_file_inodes": 0, 2024-04-25T01:24:43.055 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_inode_file_caps": 0, 2024-04-25T01:24:43.055 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T01:24:43.055 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_grant": 0, 2024-04-25T01:24:43.055 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T01:24:43.055 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T01:24:43.055 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T01:24:43.055 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps": 0, 2024-04-25T01:24:43.055 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps_dirty": 0, 2024-04-25T01:24:43.055 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_cap_release": 0, 2024-04-25T01:24:43.055 INFO:teuthology.orchestra.run.smithi040.stdout: "process_request_cap_release": 0 2024-04-25T01:24:43.055 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.055 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_cache": { 2024-04-25T01:24:43.056 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update": 0, 2024-04-25T01:24:43.056 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update_receipt": 0, 2024-04-25T01:24:43.056 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_try_discover": 0, 2024-04-25T01:24:43.056 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_send_discover": 0, 2024-04-25T01:24:43.056 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_handle_discover": 0, 2024-04-25T01:24:43.056 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays": 0, 2024-04-25T01:24:43.056 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_delayed": 0, 2024-04-25T01:24:43.056 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_enqueuing": 0, 2024-04-25T01:24:43.056 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_created": 0, 2024-04-25T01:24:43.056 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_enqueued": 0, 2024-04-25T01:24:43.056 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_reintegrated": 0, 2024-04-25T01:24:43.056 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_migrated": 0, 2024-04-25T01:24:43.056 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_processing": 0, 2024-04-25T01:24:43.056 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_enqueued": 0, 2024-04-25T01:24:43.056 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_prioritized": 0, 2024-04-25T01:24:43.057 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_started": 0, 2024-04-25T01:24:43.057 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_completed": 0, 2024-04-25T01:24:43.057 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_quiesce_path": 0, 2024-04-25T01:24:43.057 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_quiesce_inode": 0, 2024-04-25T01:24:43.057 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T01:24:43.057 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_exportdir": 0, 2024-04-25T01:24:43.057 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_flush": 0, 2024-04-25T01:24:43.057 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragmentdir": 0, 2024-04-25T01:24:43.057 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragstats": 0, 2024-04-25T01:24:43.057 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_inodestats": 0 2024-04-25T01:24:43.057 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.057 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_log": { 2024-04-25T01:24:43.057 INFO:teuthology.orchestra.run.smithi040.stdout: "evlrg": 0, 2024-04-25T01:24:43.057 INFO:teuthology.orchestra.run.smithi040.stdout: "evadd": 0, 2024-04-25T01:24:43.058 INFO:teuthology.orchestra.run.smithi040.stdout: "evex": 0, 2024-04-25T01:24:43.058 INFO:teuthology.orchestra.run.smithi040.stdout: "evtrm": 0, 2024-04-25T01:24:43.058 INFO:teuthology.orchestra.run.smithi040.stdout: "ev": 5390, 2024-04-25T01:24:43.058 INFO:teuthology.orchestra.run.smithi040.stdout: "evexg": 0, 2024-04-25T01:24:43.058 INFO:teuthology.orchestra.run.smithi040.stdout: "evexd": 0, 2024-04-25T01:24:43.058 INFO:teuthology.orchestra.run.smithi040.stdout: "segadd": 0, 2024-04-25T01:24:43.058 INFO:teuthology.orchestra.run.smithi040.stdout: "segex": 0, 2024-04-25T01:24:43.058 INFO:teuthology.orchestra.run.smithi040.stdout: "segtrm": 0, 2024-04-25T01:24:43.058 INFO:teuthology.orchestra.run.smithi040.stdout: "seg": 7, 2024-04-25T01:24:43.058 INFO:teuthology.orchestra.run.smithi040.stdout: "segmjr": 2, 2024-04-25T01:24:43.058 INFO:teuthology.orchestra.run.smithi040.stdout: "segexg": 0, 2024-04-25T01:24:43.058 INFO:teuthology.orchestra.run.smithi040.stdout: "segexd": 0, 2024-04-25T01:24:43.058 INFO:teuthology.orchestra.run.smithi040.stdout: "expos": 4194304, 2024-04-25T01:24:43.058 INFO:teuthology.orchestra.run.smithi040.stdout: "wrpos": 10811909, 2024-04-25T01:24:43.058 INFO:teuthology.orchestra.run.smithi040.stdout: "rdpos": 10809617, 2024-04-25T01:24:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: "jlat": { 2024-04-25T01:24:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: "replayed": 5390 2024-04-25T01:24:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_mem": { 2024-04-25T01:24:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 4, 2024-04-25T01:24:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: "ino+": 64, 2024-04-25T01:24:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: "ino-": 60, 2024-04-25T01:24:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: "dir": 2, 2024-04-25T01:24:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: "dir+": 15, 2024-04-25T01:24:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: "dir-": 13, 2024-04-25T01:24:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: "dn": 51, 2024-04-25T01:24:43.060 INFO:teuthology.orchestra.run.smithi040.stdout: "dn+": 111, 2024-04-25T01:24:43.060 INFO:teuthology.orchestra.run.smithi040.stdout: "dn-": 60, 2024-04-25T01:24:43.060 INFO:teuthology.orchestra.run.smithi040.stdout: "cap": 0, 2024-04-25T01:24:43.060 INFO:teuthology.orchestra.run.smithi040.stdout: "cap+": 0, 2024-04-25T01:24:43.060 INFO:teuthology.orchestra.run.smithi040.stdout: "cap-": 0, 2024-04-25T01:24:43.060 INFO:teuthology.orchestra.run.smithi040.stdout: "rss": 52896, 2024-04-25T01:24:43.060 INFO:teuthology.orchestra.run.smithi040.stdout: "heap": 215420 2024-04-25T01:24:43.060 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.060 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_server": { 2024-04-25T01:24:43.060 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_client_request": 0, 2024-04-25T01:24:43.060 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_server_request": 0, 2024-04-25T01:24:43.060 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_request": 0, 2024-04-25T01:24:43.060 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_session": 0, 2024-04-25T01:24:43.060 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_peer_request": 0, 2024-04-25T01:24:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: "req_create_latency": { 2024-04-25T01:24:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getattr_latency": { 2024-04-25T01:24:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getfilelock_latency": { 2024-04-25T01:24:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.062 INFO:teuthology.orchestra.run.smithi040.stdout: "req_link_latency": { 2024-04-25T01:24:43.062 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.062 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.062 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.062 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.062 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookup_latency": { 2024-04-25T01:24:43.062 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.062 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.062 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.062 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.062 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookuphash_latency": { 2024-04-25T01:24:43.062 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.062 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.063 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.063 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.063 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupino_latency": { 2024-04-25T01:24:43.063 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.063 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.063 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.063 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.063 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupname_latency": { 2024-04-25T01:24:43.063 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.063 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.063 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.063 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.063 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupparent_latency": { 2024-04-25T01:24:43.063 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.064 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.064 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.064 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.064 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupsnap_latency": { 2024-04-25T01:24:43.064 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.064 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.064 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.064 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.064 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lssnap_latency": { 2024-04-25T01:24:43.064 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.064 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.064 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.064 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.064 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mkdir_latency": { 2024-04-25T01:24:43.064 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.065 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.065 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.065 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.065 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mknod_latency": { 2024-04-25T01:24:43.065 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.065 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.066 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.066 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.066 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mksnap_latency": { 2024-04-25T01:24:43.066 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.066 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.066 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.066 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.066 INFO:teuthology.orchestra.run.smithi040.stdout: "req_open_latency": { 2024-04-25T01:24:43.066 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.066 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.066 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.066 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.066 INFO:teuthology.orchestra.run.smithi040.stdout: "req_readdir_latency": { 2024-04-25T01:24:43.067 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.067 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.067 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.067 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.067 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rename_latency": { 2024-04-25T01:24:43.067 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.067 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.067 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.067 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.067 INFO:teuthology.orchestra.run.smithi040.stdout: "req_renamesnap_latency": { 2024-04-25T01:24:43.067 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.067 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.067 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.067 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: "req_snapdiff_latency": { 2024-04-25T01:24:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmdir_latency": { 2024-04-25T01:24:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmsnap_latency": { 2024-04-25T01:24:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.069 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.069 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmxattr_latency": { 2024-04-25T01:24:43.069 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.069 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.069 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.069 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.069 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setattr_latency": { 2024-04-25T01:24:43.069 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.069 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.069 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.069 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.069 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setdirlayout_latency": { 2024-04-25T01:24:43.069 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.069 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.070 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.070 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.070 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setfilelock_latency": { 2024-04-25T01:24:43.070 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.070 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.070 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.070 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.070 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setlayout_latency": { 2024-04-25T01:24:43.070 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.070 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.070 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.070 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.070 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setxattr_latency": { 2024-04-25T01:24:43.070 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: "req_symlink_latency": { 2024-04-25T01:24:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: "req_unlink_latency": { 2024-04-25T01:24:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_revoke_eviction": 0, 2024-04-25T01:24:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_acquisition_throttle": 0, 2024-04-25T01:24:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getvxattr_latency": { 2024-04-25T01:24:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_sessions": { 2024-04-25T01:24:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: "session_count": 1, 2024-04-25T01:24:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: "session_add": 2, 2024-04-25T01:24:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: "session_remove": 1, 2024-04-25T01:24:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_open": 1, 2024-04-25T01:24:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_stale": 0, 2024-04-25T01:24:43.073 INFO:teuthology.orchestra.run.smithi040.stdout: "total_load": 0, 2024-04-25T01:24:43.073 INFO:teuthology.orchestra.run.smithi040.stdout: "average_load": 0, 2024-04-25T01:24:43.073 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_session_uptime": 2026, 2024-04-25T01:24:43.073 INFO:teuthology.orchestra.run.smithi040.stdout: "mdthresh_evicted": 0 2024-04-25T01:24:43.073 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.073 INFO:teuthology.orchestra.run.smithi040.stdout: "mempool": { 2024-04-25T01:24:43.073 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_bytes": 0, 2024-04-25T01:24:43.073 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_items": 0, 2024-04-25T01:24:43.073 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T01:24:43.073 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_items": 0, 2024-04-25T01:24:43.073 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T01:24:43.073 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_items": 0, 2024-04-25T01:24:43.073 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T01:24:43.073 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T01:24:43.074 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T01:24:43.074 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T01:24:43.074 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T01:24:43.074 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_items": 0, 2024-04-25T01:24:43.074 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T01:24:43.074 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T01:24:43.074 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_bytes": 0, 2024-04-25T01:24:43.074 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_items": 0, 2024-04-25T01:24:43.074 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_bytes": 0, 2024-04-25T01:24:43.074 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_items": 0, 2024-04-25T01:24:43.074 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T01:24:43.074 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T01:24:43.074 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T01:24:43.075 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T01:24:43.075 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T01:24:43.075 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_items": 0, 2024-04-25T01:24:43.075 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_bytes": 0, 2024-04-25T01:24:43.075 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_items": 0, 2024-04-25T01:24:43.075 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T01:24:43.075 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T01:24:43.075 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_bytes": 0, 2024-04-25T01:24:43.075 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_items": 0, 2024-04-25T01:24:43.075 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_bytes": 0, 2024-04-25T01:24:43.075 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_items": 0, 2024-04-25T01:24:43.075 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T01:24:43.075 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_items": 0, 2024-04-25T01:24:43.075 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T01:24:43.075 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_items": 0, 2024-04-25T01:24:43.076 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_bytes": 198203, 2024-04-25T01:24:43.076 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_items": 56, 2024-04-25T01:24:43.076 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_bytes": 0, 2024-04-25T01:24:43.076 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_items": 0, 2024-04-25T01:24:43.076 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_bytes": 0, 2024-04-25T01:24:43.076 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_items": 0, 2024-04-25T01:24:43.076 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_bytes": 0, 2024-04-25T01:24:43.076 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_items": 0, 2024-04-25T01:24:43.076 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_bytes": 0, 2024-04-25T01:24:43.076 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_items": 0, 2024-04-25T01:24:43.076 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_bytes": 7344, 2024-04-25T01:24:43.076 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_items": 155, 2024-04-25T01:24:43.076 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T01:24:43.076 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_items": 0, 2024-04-25T01:24:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_bytes": 0, 2024-04-25T01:24:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_items": 0, 2024-04-25T01:24:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_bytes": 47888, 2024-04-25T01:24:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_items": 232, 2024-04-25T01:24:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_bytes": 0, 2024-04-25T01:24:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_items": 0, 2024-04-25T01:24:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_bytes": 0, 2024-04-25T01:24:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_items": 0 2024-04-25T01:24:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: "objecter": { 2024-04-25T01:24:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: "op_active": 0, 2024-04-25T01:24:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: "op_laggy": 0, 2024-04-25T01:24:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send": 1810, 2024-04-25T01:24:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send_bytes": 21, 2024-04-25T01:24:43.078 INFO:teuthology.orchestra.run.smithi040.stdout: "op_resend": 3, 2024-04-25T01:24:43.078 INFO:teuthology.orchestra.run.smithi040.stdout: "op_reply": 1810, 2024-04-25T01:24:43.078 INFO:teuthology.orchestra.run.smithi040.stdout: "op_latency": { 2024-04-25T01:24:43.078 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1810, 2024-04-25T01:24:43.078 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 86.802024912, 2024-04-25T01:24:43.078 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.047956919 2024-04-25T01:24:43.078 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.078 INFO:teuthology.orchestra.run.smithi040.stdout: "op_inflight": 0, 2024-04-25T01:24:43.078 INFO:teuthology.orchestra.run.smithi040.stdout: "oplen_avg": { 2024-04-25T01:24:43.078 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1810, 2024-04-25T01:24:43.078 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 1811 2024-04-25T01:24:43.078 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.078 INFO:teuthology.orchestra.run.smithi040.stdout: "op": 1810, 2024-04-25T01:24:43.078 INFO:teuthology.orchestra.run.smithi040.stdout: "op_r": 1810, 2024-04-25T01:24:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: "op_w": 0, 2024-04-25T01:24:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: "op_rmw": 0, 2024-04-25T01:24:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: "op_pg": 0, 2024-04-25T01:24:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_stat": 1080, 2024-04-25T01:24:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_create": 0, 2024-04-25T01:24:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_read": 728, 2024-04-25T01:24:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_write": 0, 2024-04-25T01:24:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writefull": 0, 2024-04-25T01:24:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writesame": 0, 2024-04-25T01:24:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_append": 0, 2024-04-25T01:24:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_zero": 0, 2024-04-25T01:24:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_truncate": 0, 2024-04-25T01:24:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_delete": 0, 2024-04-25T01:24:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_mapext": 0, 2024-04-25T01:24:43.080 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_sparse_read": 0, 2024-04-25T01:24:43.080 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_clonerange": 0, 2024-04-25T01:24:43.080 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_getxattr": 1, 2024-04-25T01:24:43.080 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_setxattr": 0, 2024-04-25T01:24:43.080 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_cmpxattr": 0, 2024-04-25T01:24:43.080 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_rmxattr": 0, 2024-04-25T01:24:43.080 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_resetxattrs": 0, 2024-04-25T01:24:43.080 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_call": 0, 2024-04-25T01:24:43.080 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_watch": 0, 2024-04-25T01:24:43.080 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_notify": 0, 2024-04-25T01:24:43.080 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T01:24:43.080 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls": 0, 2024-04-25T01:24:43.080 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls_filter": 0, 2024-04-25T01:24:43.080 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_other": 0, 2024-04-25T01:24:43.080 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_active": 0, 2024-04-25T01:24:43.081 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_send": 0, 2024-04-25T01:24:43.081 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_resend": 0, 2024-04-25T01:24:43.081 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_ping": 0, 2024-04-25T01:24:43.081 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_active": 0, 2024-04-25T01:24:43.081 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_send": 0, 2024-04-25T01:24:43.081 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_resend": 0, 2024-04-25T01:24:43.081 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_active": 0, 2024-04-25T01:24:43.081 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_send": 0, 2024-04-25T01:24:43.081 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_resend": 0, 2024-04-25T01:24:43.081 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_active": 0, 2024-04-25T01:24:43.081 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_send": 0, 2024-04-25T01:24:43.081 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_resend": 0, 2024-04-25T01:24:43.081 INFO:teuthology.orchestra.run.smithi040.stdout: "command_active": 0, 2024-04-25T01:24:43.081 INFO:teuthology.orchestra.run.smithi040.stdout: "command_send": 0, 2024-04-25T01:24:43.081 INFO:teuthology.orchestra.run.smithi040.stdout: "command_resend": 0, 2024-04-25T01:24:43.082 INFO:teuthology.orchestra.run.smithi040.stdout: "map_epoch": 89, 2024-04-25T01:24:43.082 INFO:teuthology.orchestra.run.smithi040.stdout: "map_full": 0, 2024-04-25T01:24:43.082 INFO:teuthology.orchestra.run.smithi040.stdout: "map_inc": 2, 2024-04-25T01:24:43.082 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_sessions": 6, 2024-04-25T01:24:43.082 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_open": 6, 2024-04-25T01:24:43.082 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_close": 0, 2024-04-25T01:24:43.082 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_laggy": 0, 2024-04-25T01:24:43.082 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_wr": 0, 2024-04-25T01:24:43.082 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_rd": 2, 2024-04-25T01:24:43.082 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_del": 0 2024-04-25T01:24:43.082 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.082 INFO:teuthology.orchestra.run.smithi040.stdout: "oft": { 2024-04-25T01:24:43.082 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_objs": 0, 2024-04-25T01:24:43.082 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_kv_pairs": 0, 2024-04-25T01:24:43.083 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_updates": 0, 2024-04-25T01:24:43.083 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_removes": 0 2024-04-25T01:24:43.083 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.083 INFO:teuthology.orchestra.run.smithi040.stdout: "osd-slow-ops": { 2024-04-25T01:24:43.083 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_ops_count": 0 2024-04-25T01:24:43.083 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.083 INFO:teuthology.orchestra.run.smithi040.stdout: "purge_queue": { 2024-04-25T01:24:43.083 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops": 0, 2024-04-25T01:24:43.083 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T01:24:43.083 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing": 0, 2024-04-25T01:24:43.083 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_high_water": 0, 2024-04-25T01:24:43.083 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executed_ops": 0, 2024-04-25T01:24:43.083 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executed": 0, 2024-04-25T01:24:43.083 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_item_in_journal": 0 2024-04-25T01:24:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T01:24:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T01:24:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2024-04-25T01:24:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T01:24:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 2020, 2024-04-25T01:24:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 7465006, 2024-04-25T01:24:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 2020, 2024-04-25T01:24:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-04-25T01:24:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-04-25T01:24:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2020, 2024-04-25T01:24:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 7465006, 2024-04-25T01:24:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T01:24:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.085 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.085 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.085 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:43.085 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.085 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_bytes": { 2024-04-25T01:24:43.085 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T01:24:43.085 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2024-04-25T01:24:43.085 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T01:24:43.085 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-04-25T01:24:43.085 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-04-25T01:24:43.085 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:43.085 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-04-25T01:24:43.085 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 1810, 2024-04-25T01:24:43.085 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 6617692, 2024-04-25T01:24:43.086 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 1810, 2024-04-25T01:24:43.086 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 6617692, 2024-04-25T01:24:43.086 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T01:24:43.086 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.086 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.086 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.086 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:43.086 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.086 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_ops": { 2024-04-25T01:24:43.086 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T01:24:43.086 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 1024, 2024-04-25T01:24:43.086 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T01:24:43.086 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-04-25T01:24:43.086 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-04-25T01:24:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-04-25T01:24:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 1810, 2024-04-25T01:24:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 1810, 2024-04-25T01:24:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 1810, 2024-04-25T01:24:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 1810, 2024-04-25T01:24:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T01:24:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle": { 2024-04-25T01:24:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T01:24:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2024-04-25T01:24:43.088 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T01:24:43.088 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-04-25T01:24:43.088 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-04-25T01:24:43.088 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:43.088 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-04-25T01:24:43.088 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-04-25T01:24:43.088 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-04-25T01:24:43.088 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 0, 2024-04-25T01:24:43.088 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 0, 2024-04-25T01:24:43.088 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T01:24:43.088 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.088 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.088 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.088 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle-0x56453d6dc6e0": { 2024-04-25T01:24:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T01:24:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2024-04-25T01:24:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T01:24:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-04-25T01:24:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-04-25T01:24:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-04-25T01:24:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-04-25T01:24:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-04-25T01:24:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 0, 2024-04-25T01:24:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 0, 2024-04-25T01:24:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T01:24:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:43.090 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:43.090 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:43.090 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:43.090 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:43.090 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-25T01:24:43.683 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-25T01:24:43.683 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-25T01:24:43.683 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-04-25T01:24:43.683 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T01:24:43.683 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T01:24:43.938 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T01:24:43.941 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:43 smithi040 ceph-mon[32518]: pgmap v654: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-25T01:24:44.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:43 smithi169 ceph-mon[32759]: pgmap v654: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-25T01:24:44.249 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:43 smithi031 ceph-mon[28053]: pgmap v654: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-25T01:24:45.557 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-25T01:24:45.557 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T01:24:45.557 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 943, 2024-04-25T01:24:45.557 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 943, 2024-04-25T01:24:45.557 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 4492164, 2024-04-25T01:24:45.557 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 299060, 2024-04-25T01:24:45.557 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 6, 2024-04-25T01:24:45.557 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 2, 2024-04-25T01:24:45.557 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.094936531, 2024-04-25T01:24:45.557 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.033166316, 2024-04-25T01:24:45.558 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.024930889, 2024-04-25T01:24:45.558 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.028152542, 2024-04-25T01:24:45.558 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:45.558 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 942, 2024-04-25T01:24:45.558 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.036986663, 2024-04-25T01:24:45.558 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000039263 2024-04-25T01:24:45.558 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.558 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:45.558 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.558 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.558 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.558 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.558 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T01:24:45.559 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T01:24:45.559 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.559 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T01:24:45.559 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 467, 2024-04-25T01:24:45.559 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 450, 2024-04-25T01:24:45.559 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 717319, 2024-04-25T01:24:45.559 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 197426, 2024-04-25T01:24:45.559 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 4, 2024-04-25T01:24:45.559 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 3, 2024-04-25T01:24:45.559 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.054000778, 2024-04-25T01:24:45.559 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.015492315, 2024-04-25T01:24:45.559 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.021451325, 2024-04-25T01:24:45.559 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.016001681, 2024-04-25T01:24:45.559 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:45.559 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 449, 2024-04-25T01:24:45.560 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.022214087, 2024-04-25T01:24:45.560 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000049474 2024-04-25T01:24:45.560 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.560 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:45.560 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.560 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.560 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.560 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.560 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 542048, 2024-04-25T01:24:45.560 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 115952 2024-04-25T01:24:45.560 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.560 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T01:24:45.560 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 985, 2024-04-25T01:24:45.560 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 1126, 2024-04-25T01:24:45.561 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 8637867, 2024-04-25T01:24:45.561 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 641281, 2024-04-25T01:24:45.561 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 5, 2024-04-25T01:24:45.561 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 4, 2024-04-25T01:24:45.561 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.092860509, 2024-04-25T01:24:45.561 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.024764092, 2024-04-25T01:24:45.561 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.031133494, 2024-04-25T01:24:45.561 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.029600713, 2024-04-25T01:24:45.561 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:45.561 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1124, 2024-04-25T01:24:45.561 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.055014104, 2024-04-25T01:24:45.561 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000048944 2024-04-25T01:24:45.561 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.561 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:45.561 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.562 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.562 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.562 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.562 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T01:24:45.562 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 328368 2024-04-25T01:24:45.562 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.562 INFO:teuthology.orchestra.run.smithi040.stdout: "cct": { 2024-04-25T01:24:45.562 INFO:teuthology.orchestra.run.smithi040.stdout: "total_workers": 1, 2024-04-25T01:24:45.562 INFO:teuthology.orchestra.run.smithi040.stdout: "unhealthy_workers": 0 2024-04-25T01:24:45.562 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.562 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-MDSRank": { 2024-04-25T01:24:45.562 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2024-04-25T01:24:45.562 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2024-04-25T01:24:45.563 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 3686, 2024-04-25T01:24:45.563 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.109520637, 2024-04-25T01:24:45.563 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000029712 2024-04-25T01:24:45.563 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:45.563 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.563 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-PurgeQueue": { 2024-04-25T01:24:45.563 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2024-04-25T01:24:45.563 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2024-04-25T01:24:45.563 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.563 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.564 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.564 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:45.564 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.564 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2024-04-25T01:24:45.564 INFO:teuthology.orchestra.run.smithi040.stdout: "request": 0, 2024-04-25T01:24:45.564 INFO:teuthology.orchestra.run.smithi040.stdout: "reply": 0, 2024-04-25T01:24:45.564 INFO:teuthology.orchestra.run.smithi040.stdout: "reply_latency": { 2024-04-25T01:24:45.564 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.564 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.564 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.564 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.564 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_reply": 0, 2024-04-25T01:24:45.565 INFO:teuthology.orchestra.run.smithi040.stdout: "forward": 0, 2024-04-25T01:24:45.565 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_complete": 0, 2024-04-25T01:24:45.565 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_keys": 0, 2024-04-25T01:24:45.565 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_commit": 0, 2024-04-25T01:24:45.565 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_split": 0, 2024-04-25T01:24:45.565 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_merge": 0, 2024-04-25T01:24:45.565 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes": 558, 2024-04-25T01:24:45.565 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_top": 358, 2024-04-25T01:24:45.565 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_bottom": 200, 2024-04-25T01:24:45.565 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pin_tail": 0, 2024-04-25T01:24:45.565 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pinned": 46, 2024-04-25T01:24:45.565 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_expired": 0, 2024-04-25T01:24:45.565 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_with_caps": 0, 2024-04-25T01:24:45.566 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": 0, 2024-04-25T01:24:45.566 INFO:teuthology.orchestra.run.smithi040.stdout: "subtrees": 3, 2024-04-25T01:24:45.566 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse": 0, 2024-04-25T01:24:45.566 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_hit": 0, 2024-04-25T01:24:45.566 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_forward": 0, 2024-04-25T01:24:45.566 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_discover": 0, 2024-04-25T01:24:45.566 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_dir_fetch": 0, 2024-04-25T01:24:45.566 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_remote_ino": 0, 2024-04-25T01:24:45.566 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_lock": 0, 2024-04-25T01:24:45.566 INFO:teuthology.orchestra.run.smithi040.stdout: "load_cent": 0, 2024-04-25T01:24:45.566 INFO:teuthology.orchestra.run.smithi040.stdout: "q": 0, 2024-04-25T01:24:45.566 INFO:teuthology.orchestra.run.smithi040.stdout: "exported": 0, 2024-04-25T01:24:45.566 INFO:teuthology.orchestra.run.smithi040.stdout: "exported_inodes": 0, 2024-04-25T01:24:45.566 INFO:teuthology.orchestra.run.smithi040.stdout: "imported": 0, 2024-04-25T01:24:45.567 INFO:teuthology.orchestra.run.smithi040.stdout: "imported_inodes": 0, 2024-04-25T01:24:45.567 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_dir_fetch": 0, 2024-04-25T01:24:45.567 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_backtrace_fetch": 0, 2024-04-25T01:24:45.567 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_peer_discover": 0, 2024-04-25T01:24:45.567 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rfiles": 4, 2024-04-25T01:24:45.567 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rbytes": 234, 2024-04-25T01:24:45.567 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rsnaps": 0, 2024-04-25T01:24:45.567 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T01:24:45.567 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_set_tag": 0, 2024-04-25T01:24:45.567 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T01:24:45.567 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_inotable_repaired": 0, 2024-04-25T01:24:45.567 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_inodes": 0, 2024-04-25T01:24:45.567 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T01:24:45.567 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T01:24:45.567 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_file_inodes": 0, 2024-04-25T01:24:45.568 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_inode_file_caps": 0, 2024-04-25T01:24:45.568 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T01:24:45.568 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_grant": 0, 2024-04-25T01:24:45.568 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T01:24:45.568 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T01:24:45.568 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T01:24:45.568 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps": 0, 2024-04-25T01:24:45.568 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps_dirty": 0, 2024-04-25T01:24:45.568 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_cap_release": 0, 2024-04-25T01:24:45.568 INFO:teuthology.orchestra.run.smithi040.stdout: "process_request_cap_release": 0 2024-04-25T01:24:45.568 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.568 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_cache": { 2024-04-25T01:24:45.568 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update": 0, 2024-04-25T01:24:45.569 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update_receipt": 0, 2024-04-25T01:24:45.569 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_try_discover": 0, 2024-04-25T01:24:45.569 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_send_discover": 0, 2024-04-25T01:24:45.569 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_handle_discover": 0, 2024-04-25T01:24:45.569 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays": 0, 2024-04-25T01:24:45.569 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_delayed": 0, 2024-04-25T01:24:45.569 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_enqueuing": 0, 2024-04-25T01:24:45.569 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_created": 0, 2024-04-25T01:24:45.569 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_enqueued": 0, 2024-04-25T01:24:45.569 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_reintegrated": 0, 2024-04-25T01:24:45.569 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_migrated": 0, 2024-04-25T01:24:45.569 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_processing": 0, 2024-04-25T01:24:45.569 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_enqueued": 0, 2024-04-25T01:24:45.569 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_prioritized": 0, 2024-04-25T01:24:45.570 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_started": 0, 2024-04-25T01:24:45.570 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_completed": 0, 2024-04-25T01:24:45.570 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_quiesce_path": 0, 2024-04-25T01:24:45.570 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_quiesce_inode": 0, 2024-04-25T01:24:45.570 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T01:24:45.570 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_exportdir": 0, 2024-04-25T01:24:45.570 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_flush": 0, 2024-04-25T01:24:45.570 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragmentdir": 0, 2024-04-25T01:24:45.570 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragstats": 0, 2024-04-25T01:24:45.570 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_inodestats": 0 2024-04-25T01:24:45.570 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.570 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_log": { 2024-04-25T01:24:45.570 INFO:teuthology.orchestra.run.smithi040.stdout: "evlrg": 0, 2024-04-25T01:24:45.571 INFO:teuthology.orchestra.run.smithi040.stdout: "evadd": 0, 2024-04-25T01:24:45.571 INFO:teuthology.orchestra.run.smithi040.stdout: "evex": 0, 2024-04-25T01:24:45.571 INFO:teuthology.orchestra.run.smithi040.stdout: "evtrm": 0, 2024-04-25T01:24:45.571 INFO:teuthology.orchestra.run.smithi040.stdout: "ev": 7505, 2024-04-25T01:24:45.571 INFO:teuthology.orchestra.run.smithi040.stdout: "evexg": 0, 2024-04-25T01:24:45.571 INFO:teuthology.orchestra.run.smithi040.stdout: "evexd": 0, 2024-04-25T01:24:45.571 INFO:teuthology.orchestra.run.smithi040.stdout: "segadd": 0, 2024-04-25T01:24:45.571 INFO:teuthology.orchestra.run.smithi040.stdout: "segex": 0, 2024-04-25T01:24:45.571 INFO:teuthology.orchestra.run.smithi040.stdout: "segtrm": 0, 2024-04-25T01:24:45.571 INFO:teuthology.orchestra.run.smithi040.stdout: "seg": 2, 2024-04-25T01:24:45.571 INFO:teuthology.orchestra.run.smithi040.stdout: "segmjr": 37, 2024-04-25T01:24:45.571 INFO:teuthology.orchestra.run.smithi040.stdout: "segexg": 0, 2024-04-25T01:24:45.571 INFO:teuthology.orchestra.run.smithi040.stdout: "segexd": 0, 2024-04-25T01:24:45.571 INFO:teuthology.orchestra.run.smithi040.stdout: "expos": 16670410, 2024-04-25T01:24:45.572 INFO:teuthology.orchestra.run.smithi040.stdout: "wrpos": 16888327, 2024-04-25T01:24:45.572 INFO:teuthology.orchestra.run.smithi040.stdout: "rdpos": 16886675, 2024-04-25T01:24:45.572 INFO:teuthology.orchestra.run.smithi040.stdout: "jlat": { 2024-04-25T01:24:45.572 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.572 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.572 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.572 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.572 INFO:teuthology.orchestra.run.smithi040.stdout: "replayed": 7505 2024-04-25T01:24:45.572 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.572 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_mem": { 2024-04-25T01:24:45.572 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 8, 2024-04-25T01:24:45.572 INFO:teuthology.orchestra.run.smithi040.stdout: "ino+": 993, 2024-04-25T01:24:45.572 INFO:teuthology.orchestra.run.smithi040.stdout: "ino-": 985, 2024-04-25T01:24:45.572 INFO:teuthology.orchestra.run.smithi040.stdout: "dir": 7, 2024-04-25T01:24:45.572 INFO:teuthology.orchestra.run.smithi040.stdout: "dir+": 187, 2024-04-25T01:24:45.572 INFO:teuthology.orchestra.run.smithi040.stdout: "dir-": 180, 2024-04-25T01:24:45.573 INFO:teuthology.orchestra.run.smithi040.stdout: "dn": 558, 2024-04-25T01:24:45.573 INFO:teuthology.orchestra.run.smithi040.stdout: "dn+": 1352, 2024-04-25T01:24:45.573 INFO:teuthology.orchestra.run.smithi040.stdout: "dn-": 794, 2024-04-25T01:24:45.573 INFO:teuthology.orchestra.run.smithi040.stdout: "cap": 0, 2024-04-25T01:24:45.573 INFO:teuthology.orchestra.run.smithi040.stdout: "cap+": 0, 2024-04-25T01:24:45.573 INFO:teuthology.orchestra.run.smithi040.stdout: "cap-": 0, 2024-04-25T01:24:45.573 INFO:teuthology.orchestra.run.smithi040.stdout: "rss": 52604, 2024-04-25T01:24:45.573 INFO:teuthology.orchestra.run.smithi040.stdout: "heap": 215420 2024-04-25T01:24:45.573 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.573 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_server": { 2024-04-25T01:24:45.573 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_client_request": 0, 2024-04-25T01:24:45.573 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_server_request": 0, 2024-04-25T01:24:45.573 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_request": 0, 2024-04-25T01:24:45.573 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_session": 0, 2024-04-25T01:24:45.573 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_peer_request": 0, 2024-04-25T01:24:45.574 INFO:teuthology.orchestra.run.smithi040.stdout: "req_create_latency": { 2024-04-25T01:24:45.574 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.574 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.574 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.574 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.574 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getattr_latency": { 2024-04-25T01:24:45.574 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.574 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.574 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.574 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.574 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getfilelock_latency": { 2024-04-25T01:24:45.574 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.574 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.574 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.574 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.575 INFO:teuthology.orchestra.run.smithi040.stdout: "req_link_latency": { 2024-04-25T01:24:45.575 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.575 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.575 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.575 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.575 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookup_latency": { 2024-04-25T01:24:45.575 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.575 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.575 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.575 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.575 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookuphash_latency": { 2024-04-25T01:24:45.575 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.575 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.575 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.575 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.576 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupino_latency": { 2024-04-25T01:24:45.576 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.576 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.576 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.576 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.576 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupname_latency": { 2024-04-25T01:24:45.576 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.576 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.576 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.576 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.576 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupparent_latency": { 2024-04-25T01:24:45.576 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.576 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.576 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.577 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.577 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupsnap_latency": { 2024-04-25T01:24:45.577 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.577 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.577 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.577 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.577 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lssnap_latency": { 2024-04-25T01:24:45.577 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.577 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.577 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.577 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.577 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mkdir_latency": { 2024-04-25T01:24:45.578 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.578 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.578 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.578 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.578 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mknod_latency": { 2024-04-25T01:24:45.578 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.578 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.579 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.579 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.579 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mksnap_latency": { 2024-04-25T01:24:45.579 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.579 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.579 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.579 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.579 INFO:teuthology.orchestra.run.smithi040.stdout: "req_open_latency": { 2024-04-25T01:24:45.579 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.579 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.579 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.579 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.579 INFO:teuthology.orchestra.run.smithi040.stdout: "req_readdir_latency": { 2024-04-25T01:24:45.579 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.579 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.579 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.580 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.580 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rename_latency": { 2024-04-25T01:24:45.580 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.580 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.580 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.580 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.580 INFO:teuthology.orchestra.run.smithi040.stdout: "req_renamesnap_latency": { 2024-04-25T01:24:45.580 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.580 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.580 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.580 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.580 INFO:teuthology.orchestra.run.smithi040.stdout: "req_snapdiff_latency": { 2024-04-25T01:24:45.580 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.580 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.580 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.581 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.581 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmdir_latency": { 2024-04-25T01:24:45.581 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.581 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.581 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.581 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.581 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmsnap_latency": { 2024-04-25T01:24:45.581 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.581 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.581 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.581 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.581 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmxattr_latency": { 2024-04-25T01:24:45.581 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.581 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.582 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.582 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.582 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setattr_latency": { 2024-04-25T01:24:45.582 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.582 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.582 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.582 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.582 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setdirlayout_latency": { 2024-04-25T01:24:45.582 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.582 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.582 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.582 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.582 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setfilelock_latency": { 2024-04-25T01:24:45.582 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.583 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.583 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.583 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.583 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setlayout_latency": { 2024-04-25T01:24:45.583 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.583 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.583 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.583 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.583 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setxattr_latency": { 2024-04-25T01:24:45.583 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.583 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.583 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.583 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.583 INFO:teuthology.orchestra.run.smithi040.stdout: "req_symlink_latency": { 2024-04-25T01:24:45.584 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.584 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.584 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.584 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.584 INFO:teuthology.orchestra.run.smithi040.stdout: "req_unlink_latency": { 2024-04-25T01:24:45.584 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.584 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.584 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.584 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.584 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_revoke_eviction": 0, 2024-04-25T01:24:45.584 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_acquisition_throttle": 0, 2024-04-25T01:24:45.584 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getvxattr_latency": { 2024-04-25T01:24:45.584 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.584 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.584 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.585 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:45.585 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.585 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_sessions": { 2024-04-25T01:24:45.585 INFO:teuthology.orchestra.run.smithi040.stdout: "session_count": 1, 2024-04-25T01:24:45.585 INFO:teuthology.orchestra.run.smithi040.stdout: "session_add": 2, 2024-04-25T01:24:45.585 INFO:teuthology.orchestra.run.smithi040.stdout: "session_remove": 1, 2024-04-25T01:24:45.585 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_open": 1, 2024-04-25T01:24:45.585 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_stale": 0, 2024-04-25T01:24:45.585 INFO:teuthology.orchestra.run.smithi040.stdout: "total_load": 0, 2024-04-25T01:24:45.585 INFO:teuthology.orchestra.run.smithi040.stdout: "average_load": 0, 2024-04-25T01:24:45.585 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_session_uptime": 2029, 2024-04-25T01:24:45.585 INFO:teuthology.orchestra.run.smithi040.stdout: "mdthresh_evicted": 0 2024-04-25T01:24:45.585 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.586 INFO:teuthology.orchestra.run.smithi040.stdout: "mempool": { 2024-04-25T01:24:45.586 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_bytes": 0, 2024-04-25T01:24:45.586 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_items": 0, 2024-04-25T01:24:45.586 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T01:24:45.586 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_items": 0, 2024-04-25T01:24:45.586 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T01:24:45.586 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_items": 0, 2024-04-25T01:24:45.586 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T01:24:45.586 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T01:24:45.586 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T01:24:45.586 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T01:24:45.587 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T01:24:45.587 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_items": 0, 2024-04-25T01:24:45.587 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T01:24:45.587 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T01:24:45.587 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_bytes": 0, 2024-04-25T01:24:45.587 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_items": 0, 2024-04-25T01:24:45.587 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_bytes": 0, 2024-04-25T01:24:45.587 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_items": 0, 2024-04-25T01:24:45.587 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T01:24:45.587 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T01:24:45.587 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T01:24:45.587 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T01:24:45.587 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T01:24:45.588 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_items": 0, 2024-04-25T01:24:45.588 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_bytes": 0, 2024-04-25T01:24:45.588 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_items": 0, 2024-04-25T01:24:45.588 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T01:24:45.588 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T01:24:45.588 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_bytes": 0, 2024-04-25T01:24:45.588 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_items": 0, 2024-04-25T01:24:45.588 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_bytes": 0, 2024-04-25T01:24:45.588 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_items": 0, 2024-04-25T01:24:45.588 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T01:24:45.588 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_items": 0, 2024-04-25T01:24:45.588 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T01:24:45.589 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_items": 0, 2024-04-25T01:24:45.589 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_bytes": 198242, 2024-04-25T01:24:45.589 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_items": 57, 2024-04-25T01:24:45.589 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_bytes": 0, 2024-04-25T01:24:45.589 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_items": 0, 2024-04-25T01:24:45.589 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_bytes": 0, 2024-04-25T01:24:45.589 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_items": 0, 2024-04-25T01:24:45.589 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_bytes": 0, 2024-04-25T01:24:45.589 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_items": 0, 2024-04-25T01:24:45.589 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_bytes": 0, 2024-04-25T01:24:45.589 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_items": 0, 2024-04-25T01:24:45.589 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_bytes": 7344, 2024-04-25T01:24:45.589 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_items": 155, 2024-04-25T01:24:45.589 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T01:24:45.589 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_items": 0, 2024-04-25T01:24:45.590 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_bytes": 0, 2024-04-25T01:24:45.590 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_items": 0, 2024-04-25T01:24:45.590 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_bytes": 420224, 2024-04-25T01:24:45.590 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_items": 2314, 2024-04-25T01:24:45.590 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_bytes": 0, 2024-04-25T01:24:45.590 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_items": 0, 2024-04-25T01:24:45.590 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_bytes": 0, 2024-04-25T01:24:45.590 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_items": 0 2024-04-25T01:24:45.590 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.590 INFO:teuthology.orchestra.run.smithi040.stdout: "objecter": { 2024-04-25T01:24:45.590 INFO:teuthology.orchestra.run.smithi040.stdout: "op_active": 0, 2024-04-25T01:24:45.590 INFO:teuthology.orchestra.run.smithi040.stdout: "op_laggy": 0, 2024-04-25T01:24:45.590 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send": 2179, 2024-04-25T01:24:45.591 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send_bytes": 26, 2024-04-25T01:24:45.591 INFO:teuthology.orchestra.run.smithi040.stdout: "op_resend": 4, 2024-04-25T01:24:45.591 INFO:teuthology.orchestra.run.smithi040.stdout: "op_reply": 2179, 2024-04-25T01:24:45.591 INFO:teuthology.orchestra.run.smithi040.stdout: "op_latency": { 2024-04-25T01:24:45.591 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 2179, 2024-04-25T01:24:45.591 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 20.089776980, 2024-04-25T01:24:45.591 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.009219723 2024-04-25T01:24:45.591 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.591 INFO:teuthology.orchestra.run.smithi040.stdout: "op_inflight": 0, 2024-04-25T01:24:45.591 INFO:teuthology.orchestra.run.smithi040.stdout: "oplen_avg": { 2024-04-25T01:24:45.591 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 2179, 2024-04-25T01:24:45.591 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 2180 2024-04-25T01:24:45.591 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.591 INFO:teuthology.orchestra.run.smithi040.stdout: "op": 2179, 2024-04-25T01:24:45.591 INFO:teuthology.orchestra.run.smithi040.stdout: "op_r": 2179, 2024-04-25T01:24:45.591 INFO:teuthology.orchestra.run.smithi040.stdout: "op_w": 0, 2024-04-25T01:24:45.592 INFO:teuthology.orchestra.run.smithi040.stdout: "op_rmw": 0, 2024-04-25T01:24:45.592 INFO:teuthology.orchestra.run.smithi040.stdout: "op_pg": 0, 2024-04-25T01:24:45.592 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_stat": 1286, 2024-04-25T01:24:45.592 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_create": 0, 2024-04-25T01:24:45.592 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_read": 890, 2024-04-25T01:24:45.592 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_write": 0, 2024-04-25T01:24:45.592 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writefull": 0, 2024-04-25T01:24:45.592 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writesame": 0, 2024-04-25T01:24:45.592 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_append": 0, 2024-04-25T01:24:45.592 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_zero": 0, 2024-04-25T01:24:45.592 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_truncate": 0, 2024-04-25T01:24:45.592 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_delete": 0, 2024-04-25T01:24:45.592 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_mapext": 0, 2024-04-25T01:24:45.592 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_sparse_read": 0, 2024-04-25T01:24:45.593 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_clonerange": 0, 2024-04-25T01:24:45.593 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_getxattr": 2, 2024-04-25T01:24:45.593 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_setxattr": 0, 2024-04-25T01:24:45.593 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_cmpxattr": 0, 2024-04-25T01:24:45.593 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_rmxattr": 0, 2024-04-25T01:24:45.593 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_resetxattrs": 0, 2024-04-25T01:24:45.593 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_call": 0, 2024-04-25T01:24:45.593 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_watch": 0, 2024-04-25T01:24:45.593 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_notify": 0, 2024-04-25T01:24:45.593 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T01:24:45.593 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls": 0, 2024-04-25T01:24:45.593 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls_filter": 0, 2024-04-25T01:24:45.593 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_other": 0, 2024-04-25T01:24:45.593 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_active": 0, 2024-04-25T01:24:45.593 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_send": 0, 2024-04-25T01:24:45.594 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_resend": 0, 2024-04-25T01:24:45.594 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_ping": 0, 2024-04-25T01:24:45.594 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_active": 0, 2024-04-25T01:24:45.594 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_send": 0, 2024-04-25T01:24:45.594 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_resend": 0, 2024-04-25T01:24:45.594 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_active": 0, 2024-04-25T01:24:45.594 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_send": 0, 2024-04-25T01:24:45.594 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_resend": 0, 2024-04-25T01:24:45.594 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_active": 0, 2024-04-25T01:24:45.594 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_send": 0, 2024-04-25T01:24:45.594 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_resend": 0, 2024-04-25T01:24:45.594 INFO:teuthology.orchestra.run.smithi040.stdout: "command_active": 0, 2024-04-25T01:24:45.594 INFO:teuthology.orchestra.run.smithi040.stdout: "command_send": 0, 2024-04-25T01:24:45.594 INFO:teuthology.orchestra.run.smithi040.stdout: "command_resend": 0, 2024-04-25T01:24:45.595 INFO:teuthology.orchestra.run.smithi040.stdout: "map_epoch": 89, 2024-04-25T01:24:45.595 INFO:teuthology.orchestra.run.smithi040.stdout: "map_full": 0, 2024-04-25T01:24:45.595 INFO:teuthology.orchestra.run.smithi040.stdout: "map_inc": 2, 2024-04-25T01:24:45.595 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_sessions": 6, 2024-04-25T01:24:45.595 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_open": 6, 2024-04-25T01:24:45.595 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_close": 0, 2024-04-25T01:24:45.595 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_laggy": 0, 2024-04-25T01:24:45.595 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_wr": 0, 2024-04-25T01:24:45.595 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_rd": 2, 2024-04-25T01:24:45.595 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_del": 0 2024-04-25T01:24:45.595 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.595 INFO:teuthology.orchestra.run.smithi040.stdout: "oft": { 2024-04-25T01:24:45.595 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_objs": 0, 2024-04-25T01:24:45.595 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_kv_pairs": 0, 2024-04-25T01:24:45.596 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_updates": 0, 2024-04-25T01:24:45.596 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_removes": 0 2024-04-25T01:24:45.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.596 INFO:teuthology.orchestra.run.smithi040.stdout: "osd-slow-ops": { 2024-04-25T01:24:45.596 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_ops_count": 0 2024-04-25T01:24:45.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.596 INFO:teuthology.orchestra.run.smithi040.stdout: "purge_queue": { 2024-04-25T01:24:45.596 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops": 0, 2024-04-25T01:24:45.596 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T01:24:45.596 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing": 0, 2024-04-25T01:24:45.596 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_high_water": 0, 2024-04-25T01:24:45.596 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executed_ops": 0, 2024-04-25T01:24:45.596 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executed": 0, 2024-04-25T01:24:45.596 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_item_in_journal": 0 2024-04-25T01:24:45.597 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.597 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T01:24:45.597 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T01:24:45.597 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2024-04-25T01:24:45.597 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T01:24:45.597 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 2391, 2024-04-25T01:24:45.597 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 13618375, 2024-04-25T01:24:45.597 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:45.597 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 2391, 2024-04-25T01:24:45.597 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-04-25T01:24:45.597 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-04-25T01:24:45.597 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2391, 2024-04-25T01:24:45.597 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 13618375, 2024-04-25T01:24:45.598 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T01:24:45.598 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.598 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.598 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.598 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:45.598 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.598 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_bytes": { 2024-04-25T01:24:45.598 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T01:24:45.598 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2024-04-25T01:24:45.598 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T01:24:45.598 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-04-25T01:24:45.598 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-04-25T01:24:45.598 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:45.599 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-04-25T01:24:45.599 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 2179, 2024-04-25T01:24:45.599 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 12695142, 2024-04-25T01:24:45.599 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2179, 2024-04-25T01:24:45.599 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 12695142, 2024-04-25T01:24:45.599 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T01:24:45.599 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.599 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.599 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.599 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:45.599 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.599 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_ops": { 2024-04-25T01:24:45.599 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T01:24:45.600 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 1024, 2024-04-25T01:24:45.600 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T01:24:45.600 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-04-25T01:24:45.600 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-04-25T01:24:45.600 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:45.600 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-04-25T01:24:45.600 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 2179, 2024-04-25T01:24:45.600 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 2179, 2024-04-25T01:24:45.600 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2179, 2024-04-25T01:24:45.600 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 2179, 2024-04-25T01:24:45.600 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T01:24:45.600 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.600 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.600 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.600 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:45.601 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.601 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle": { 2024-04-25T01:24:45.601 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T01:24:45.601 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2024-04-25T01:24:45.601 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T01:24:45.601 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-04-25T01:24:45.601 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-04-25T01:24:45.601 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:45.601 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-04-25T01:24:45.601 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-04-25T01:24:45.601 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-04-25T01:24:45.601 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 0, 2024-04-25T01:24:45.601 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 0, 2024-04-25T01:24:45.601 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T01:24:45.602 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.602 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.602 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.602 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:45.602 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:45.602 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle-0x55f326a286e0": { 2024-04-25T01:24:45.602 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T01:24:45.602 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2024-04-25T01:24:45.602 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T01:24:45.602 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2024-04-25T01:24:45.602 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2024-04-25T01:24:45.602 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:45.602 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2024-04-25T01:24:45.602 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-04-25T01:24:45.603 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-04-25T01:24:45.603 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 0, 2024-04-25T01:24:45.603 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 0, 2024-04-25T01:24:45.603 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T01:24:45.603 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:45.603 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:45.603 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:45.603 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:45.603 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:45.603 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-25T01:24:46.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:45 smithi169 ceph-mon[32759]: pgmap v655: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T01:24:46.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:45 smithi040 ceph-mon[32518]: pgmap v655: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T01:24:46.249 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:45 smithi031 ceph-mon[28053]: pgmap v655: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T01:24:46.306 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-25T01:24:46.306 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-25T01:24:46.306 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-04-25T01:24:46.307 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T01:24:46.307 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T01:24:46.554 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T01:24:47.942 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:47 smithi040 ceph-mon[32518]: pgmap v656: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T01:24:48.188 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:47 smithi169 ceph-mon[32759]: pgmap v656: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T01:24:48.249 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:47 smithi031 ceph-mon[28053]: pgmap v656: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T01:24:48.269 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-25T01:24:48.269 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T01:24:48.270 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 0, 2024-04-25T01:24:48.270 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 2, 2024-04-25T01:24:48.270 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 0, 2024-04-25T01:24:48.270 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 0, 2024-04-25T01:24:48.270 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 4, 2024-04-25T01:24:48.270 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 0, 2024-04-25T01:24:48.270 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.001126061, 2024-04-25T01:24:48.270 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T01:24:48.270 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.000174799, 2024-04-25T01:24:48.270 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T01:24:48.270 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:48.270 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:48.271 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:48.271 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:48.271 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:48.271 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:48.271 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:48.271 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:48.271 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:48.271 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:48.271 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T01:24:48.271 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T01:24:48.271 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:48.271 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T01:24:48.271 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 5, 2024-04-25T01:24:48.272 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 149, 2024-04-25T01:24:48.272 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 3152, 2024-04-25T01:24:48.272 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 103872, 2024-04-25T01:24:48.272 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 3, 2024-04-25T01:24:48.272 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 2, 2024-04-25T01:24:48.272 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.008777765, 2024-04-25T01:24:48.272 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.005912471, 2024-04-25T01:24:48.272 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.001231250, 2024-04-25T01:24:48.272 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T01:24:48.272 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:48.272 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 148, 2024-04-25T01:24:48.272 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.006980691, 2024-04-25T01:24:48.272 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000047166 2024-04-25T01:24:48.273 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:48.273 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:48.273 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:48.273 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:48.273 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:48.273 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:48.273 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-25T01:24:48.273 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 103872 2024-04-25T01:24:48.273 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:48.273 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T01:24:48.273 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 195, 2024-04-25T01:24:48.273 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 192, 2024-04-25T01:24:48.273 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 470960, 2024-04-25T01:24:48.274 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 118448, 2024-04-25T01:24:48.274 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 2, 2024-04-25T01:24:48.274 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 1, 2024-04-25T01:24:48.274 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.029341916, 2024-04-25T01:24:48.274 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.010480499, 2024-04-25T01:24:48.274 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.013463593, 2024-04-25T01:24:48.274 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.005189433, 2024-04-25T01:24:48.274 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:48.274 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 191, 2024-04-25T01:24:48.274 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.010162194, 2024-04-25T01:24:48.274 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000053205 2024-04-25T01:24:48.274 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:48.274 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:48.275 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:48.275 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:48.275 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:48.275 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:48.275 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 470960, 2024-04-25T01:24:48.275 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 118448 2024-04-25T01:24:48.275 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:48.275 INFO:teuthology.orchestra.run.smithi040.stdout: "cct": { 2024-04-25T01:24:48.275 INFO:teuthology.orchestra.run.smithi040.stdout: "total_workers": 0, 2024-04-25T01:24:48.275 INFO:teuthology.orchestra.run.smithi040.stdout: "unhealthy_workers": 0 2024-04-25T01:24:48.275 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:48.275 INFO:teuthology.orchestra.run.smithi040.stdout: "mempool": { 2024-04-25T01:24:48.275 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_bytes": 0, 2024-04-25T01:24:48.276 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_items": 0, 2024-04-25T01:24:48.276 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T01:24:48.276 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_items": 0, 2024-04-25T01:24:48.276 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T01:24:48.276 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_items": 0, 2024-04-25T01:24:48.276 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T01:24:48.276 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T01:24:48.276 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T01:24:48.276 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T01:24:48.276 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T01:24:48.276 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_items": 0, 2024-04-25T01:24:48.276 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T01:24:48.276 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T01:24:48.276 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_bytes": 0, 2024-04-25T01:24:48.277 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_items": 0, 2024-04-25T01:24:48.277 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_bytes": 0, 2024-04-25T01:24:48.277 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_items": 0, 2024-04-25T01:24:48.277 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T01:24:48.277 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T01:24:48.277 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T01:24:48.277 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T01:24:48.277 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T01:24:48.277 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_items": 0, 2024-04-25T01:24:48.277 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_bytes": 0, 2024-04-25T01:24:48.277 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_items": 0, 2024-04-25T01:24:48.277 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T01:24:48.277 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T01:24:48.277 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_bytes": 0, 2024-04-25T01:24:48.278 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_items": 0, 2024-04-25T01:24:48.278 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_bytes": 0, 2024-04-25T01:24:48.278 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_items": 0, 2024-04-25T01:24:48.278 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T01:24:48.278 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_items": 0, 2024-04-25T01:24:48.278 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T01:24:48.278 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_items": 0, 2024-04-25T01:24:48.278 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_bytes": 148984, 2024-04-25T01:24:48.278 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_items": 30, 2024-04-25T01:24:48.278 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_bytes": 0, 2024-04-25T01:24:48.278 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_items": 0, 2024-04-25T01:24:48.278 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_bytes": 0, 2024-04-25T01:24:48.278 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_items": 0, 2024-04-25T01:24:48.279 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_bytes": 0, 2024-04-25T01:24:48.279 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_items": 0, 2024-04-25T01:24:48.279 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_bytes": 0, 2024-04-25T01:24:48.279 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_items": 0, 2024-04-25T01:24:48.279 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_bytes": 0, 2024-04-25T01:24:48.279 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_items": 0, 2024-04-25T01:24:48.279 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T01:24:48.279 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_items": 0, 2024-04-25T01:24:48.279 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_bytes": 0, 2024-04-25T01:24:48.279 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_items": 0, 2024-04-25T01:24:48.279 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_bytes": 928, 2024-04-25T01:24:48.279 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_items": 2, 2024-04-25T01:24:48.279 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_bytes": 0, 2024-04-25T01:24:48.279 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_items": 0, 2024-04-25T01:24:48.280 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_bytes": 0, 2024-04-25T01:24:48.280 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_items": 0 2024-04-25T01:24:48.280 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-25T01:24:48.280 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T01:24:48.280 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2024-04-25T01:24:48.280 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2024-04-25T01:24:48.280 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2024-04-25T01:24:48.280 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 196, 2024-04-25T01:24:48.280 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 443403, 2024-04-25T01:24:48.280 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:48.280 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 196, 2024-04-25T01:24:48.280 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2024-04-25T01:24:48.280 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2024-04-25T01:24:48.280 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 196, 2024-04-25T01:24:48.281 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 443403, 2024-04-25T01:24:48.281 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2024-04-25T01:24:48.281 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2024-04-25T01:24:48.281 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2024-04-25T01:24:48.281 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2024-04-25T01:24:48.281 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:48.281 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-25T01:24:48.281 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-25T01:24:49.046 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-25T01:24:49.046 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-25T01:24:49.046 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-04-25T01:24:49.046 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T01:24:49.046 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T01:24:49.297 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:24:50.067 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:49 smithi169 ceph-mon[32759]: pgmap v657: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-25T01:24:50.191 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:49 smithi040 ceph-mon[32518]: pgmap v657: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-25T01:24:50.249 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:49 smithi031 ceph-mon[28053]: pgmap v657: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-25T01:24:50.843 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T01:24:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T01:24:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 849, 2024-04-25T01:24:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 830, 2024-04-25T01:24:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 758112, 2024-04-25T01:24:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 318814, 2024-04-25T01:24:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 5, 2024-04-25T01:24:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 1, 2024-04-25T01:24:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.097477867, 2024-04-25T01:24:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.031147055, 2024-04-25T01:24:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.032211553, 2024-04-25T01:24:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.027849165, 2024-04-25T01:24:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 830, 2024-04-25T01:24:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.046004882, 2024-04-25T01:24:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000055427 2024-04-25T01:24:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 545424, 2024-04-25T01:24:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 117520 2024-04-25T01:24:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T01:24:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 194, 2024-04-25T01:24:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 338, 2024-04-25T01:24:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 1477821, 2024-04-25T01:24:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 469534, 2024-04-25T01:24:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 6, 2024-04-25T01:24:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 4, 2024-04-25T01:24:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.028021761, 2024-04-25T01:24:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.009375061, 2024-04-25T01:24:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.007446740, 2024-04-25T01:24:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.008756263, 2024-04-25T01:24:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 335, 2024-04-25T01:24:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.123460405, 2024-04-25T01:24:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000368538 2024-04-25T01:24:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T01:24:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 408416 2024-04-25T01:24:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T01:24:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 1154, 2024-04-25T01:24:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 1156, 2024-04-25T01:24:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 12879319, 2024-04-25T01:24:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 366979, 2024-04-25T01:24:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 4, 2024-04-25T01:24:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 3, 2024-04-25T01:24:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.095228491, 2024-04-25T01:24:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.029339078, 2024-04-25T01:24:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.029331678, 2024-04-25T01:24:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.030466059, 2024-04-25T01:24:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1154, 2024-04-25T01:24:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.047740640, 2024-04-25T01:24:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000041369 2024-04-25T01:24:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T01:24:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T01:24:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-04-25T01:24:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 1, 2024-04-25T01:24:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-04-25T01:24:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-MDSRank": { 2024-04-25T01:24:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-25T01:24:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-25T01:24:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 3427, 2024-04-25T01:24:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.086026183, 2024-04-25T01:24:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000025102 2024-04-25T01:24:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-PurgeQueue": { 2024-04-25T01:24:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-25T01:24:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-25T01:24:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": { 2024-04-25T01:24:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "request": 0, 2024-04-25T01:24:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "reply": 0, 2024-04-25T01:24:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "reply_latency": { 2024-04-25T01:24:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_reply": 0, 2024-04-25T01:24:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "forward": 0, 2024-04-25T01:24:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_complete": 0, 2024-04-25T01:24:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_keys": 0, 2024-04-25T01:24:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_commit": 0, 2024-04-25T01:24:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_split": 0, 2024-04-25T01:24:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_merge": 0, 2024-04-25T01:24:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes": 419, 2024-04-25T01:24:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_top": 0, 2024-04-25T01:24:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_bottom": 419, 2024-04-25T01:24:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pin_tail": 0, 2024-04-25T01:24:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pinned": 419, 2024-04-25T01:24:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_expired": 0, 2024-04-25T01:24:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_with_caps": 0, 2024-04-25T01:24:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": 0, 2024-04-25T01:24:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "subtrees": 3, 2024-04-25T01:24:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse": 0, 2024-04-25T01:24:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_hit": 0, 2024-04-25T01:24:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_forward": 0, 2024-04-25T01:24:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_discover": 0, 2024-04-25T01:24:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_dir_fetch": 0, 2024-04-25T01:24:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_remote_ino": 0, 2024-04-25T01:24:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_lock": 0, 2024-04-25T01:24:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "load_cent": 0, 2024-04-25T01:24:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "q": 0, 2024-04-25T01:24:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "exported": 0, 2024-04-25T01:24:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "exported_inodes": 0, 2024-04-25T01:24:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "imported": 0, 2024-04-25T01:24:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "imported_inodes": 0, 2024-04-25T01:24:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_dir_fetch": 0, 2024-04-25T01:24:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_backtrace_fetch": 0, 2024-04-25T01:24:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_peer_discover": 0, 2024-04-25T01:24:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rfiles": 0, 2024-04-25T01:24:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rbytes": 0, 2024-04-25T01:24:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rsnaps": 0, 2024-04-25T01:24:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T01:24:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_set_tag": 0, 2024-04-25T01:24:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T01:24:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_inotable_repaired": 0, 2024-04-25T01:24:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_inodes": 0, 2024-04-25T01:24:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T01:24:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T01:24:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_file_inodes": 0, 2024-04-25T01:24:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_inode_file_caps": 0, 2024-04-25T01:24:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T01:24:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_grant": 0, 2024-04-25T01:24:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T01:24:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T01:24:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T01:24:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps": 0, 2024-04-25T01:24:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps_dirty": 0, 2024-04-25T01:24:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_cap_release": 0, 2024-04-25T01:24:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "process_request_cap_release": 0 2024-04-25T01:24:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache": { 2024-04-25T01:24:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update": 0, 2024-04-25T01:24:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update_receipt": 0, 2024-04-25T01:24:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_try_discover": 0, 2024-04-25T01:24:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_send_discover": 0, 2024-04-25T01:24:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_handle_discover": 0, 2024-04-25T01:24:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays": 0, 2024-04-25T01:24:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_delayed": 0, 2024-04-25T01:24:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_enqueuing": 0, 2024-04-25T01:24:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_created": 0, 2024-04-25T01:24:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_enqueued": 0, 2024-04-25T01:24:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_reintegrated": 0, 2024-04-25T01:24:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_migrated": 0, 2024-04-25T01:24:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_processing": 0, 2024-04-25T01:24:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_enqueued": 0, 2024-04-25T01:24:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_prioritized": 0, 2024-04-25T01:24:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_started": 0, 2024-04-25T01:24:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_completed": 0, 2024-04-25T01:24:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_path": 0, 2024-04-25T01:24:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_inode": 0, 2024-04-25T01:24:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T01:24:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_exportdir": 0, 2024-04-25T01:24:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_flush": 0, 2024-04-25T01:24:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragmentdir": 0, 2024-04-25T01:24:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragstats": 0, 2024-04-25T01:24:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_inodestats": 0 2024-04-25T01:24:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log": { 2024-04-25T01:24:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "evlrg": 0, 2024-04-25T01:24:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "evadd": 0, 2024-04-25T01:24:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "evex": 0, 2024-04-25T01:24:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "evtrm": 0, 2024-04-25T01:24:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "ev": 6713, 2024-04-25T01:24:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "evexg": 0, 2024-04-25T01:24:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "evexd": 0, 2024-04-25T01:24:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "segadd": 0, 2024-04-25T01:24:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "segex": 0, 2024-04-25T01:24:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "segtrm": 0, 2024-04-25T01:24:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "seg": 8, 2024-04-25T01:24:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "segmjr": 2, 2024-04-25T01:24:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "segexg": 0, 2024-04-25T01:24:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "segexd": 0, 2024-04-25T01:24:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "expos": 4194304, 2024-04-25T01:24:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "wrpos": 18206829, 2024-04-25T01:24:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "rdpos": 18206255, 2024-04-25T01:24:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "jlat": { 2024-04-25T01:24:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "replayed": 6713 2024-04-25T01:24:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_mem": { 2024-04-25T01:24:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "ino": 12, 2024-04-25T01:24:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "ino+": 631, 2024-04-25T01:24:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "ino-": 619, 2024-04-25T01:24:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "dir": 18, 2024-04-25T01:24:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "dir+": 178, 2024-04-25T01:24:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "dir-": 160, 2024-04-25T01:24:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "dn": 419, 2024-04-25T01:24:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "dn+": 1209, 2024-04-25T01:24:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "dn-": 790, 2024-04-25T01:24:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "cap": 0, 2024-04-25T01:24:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "cap+": 0, 2024-04-25T01:24:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "cap-": 0, 2024-04-25T01:24:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "rss": 53032, 2024-04-25T01:24:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "heap": 215420 2024-04-25T01:24:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_server": { 2024-04-25T01:24:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_client_request": 0, 2024-04-25T01:24:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_server_request": 0, 2024-04-25T01:24:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_request": 0, 2024-04-25T01:24:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_session": 0, 2024-04-25T01:24:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_peer_request": 0, 2024-04-25T01:24:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "req_create_latency": { 2024-04-25T01:24:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getattr_latency": { 2024-04-25T01:24:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getfilelock_latency": { 2024-04-25T01:24:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "req_link_latency": { 2024-04-25T01:24:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookup_latency": { 2024-04-25T01:24:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookuphash_latency": { 2024-04-25T01:24:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupino_latency": { 2024-04-25T01:24:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupname_latency": { 2024-04-25T01:24:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupparent_latency": { 2024-04-25T01:24:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupsnap_latency": { 2024-04-25T01:24:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lssnap_latency": { 2024-04-25T01:24:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mkdir_latency": { 2024-04-25T01:24:50.864 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.864 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.864 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.864 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.864 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mknod_latency": { 2024-04-25T01:24:50.864 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.864 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.864 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.865 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.865 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mksnap_latency": { 2024-04-25T01:24:50.865 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.865 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.865 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.865 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.865 INFO:teuthology.orchestra.run.smithi169.stdout: "req_open_latency": { 2024-04-25T01:24:50.865 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.865 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.865 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.865 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.865 INFO:teuthology.orchestra.run.smithi169.stdout: "req_readdir_latency": { 2024-04-25T01:24:50.865 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.865 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.865 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.865 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.866 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rename_latency": { 2024-04-25T01:24:50.866 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.866 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.866 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.866 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.866 INFO:teuthology.orchestra.run.smithi169.stdout: "req_renamesnap_latency": { 2024-04-25T01:24:50.866 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.866 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.866 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.866 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.866 INFO:teuthology.orchestra.run.smithi169.stdout: "req_snapdiff_latency": { 2024-04-25T01:24:50.866 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.866 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.866 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.867 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.867 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmdir_latency": { 2024-04-25T01:24:50.867 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.867 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.867 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.867 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.867 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmsnap_latency": { 2024-04-25T01:24:50.867 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.867 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.867 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.867 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.867 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmxattr_latency": { 2024-04-25T01:24:50.867 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.867 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.868 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.868 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.868 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setattr_latency": { 2024-04-25T01:24:50.868 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.868 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.868 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.868 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.868 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setdirlayout_latency": { 2024-04-25T01:24:50.868 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.868 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.868 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.868 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.868 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setfilelock_latency": { 2024-04-25T01:24:50.868 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.868 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.869 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.869 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.869 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setlayout_latency": { 2024-04-25T01:24:50.869 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.869 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.869 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.869 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.869 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setxattr_latency": { 2024-04-25T01:24:50.869 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.869 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.869 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.869 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.869 INFO:teuthology.orchestra.run.smithi169.stdout: "req_symlink_latency": { 2024-04-25T01:24:50.869 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.869 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.869 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.870 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.870 INFO:teuthology.orchestra.run.smithi169.stdout: "req_unlink_latency": { 2024-04-25T01:24:50.870 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.870 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.870 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.870 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.870 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_revoke_eviction": 0, 2024-04-25T01:24:50.870 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_acquisition_throttle": 0, 2024-04-25T01:24:50.870 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getvxattr_latency": { 2024-04-25T01:24:50.870 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.870 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.870 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.870 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:50.870 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.871 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_sessions": { 2024-04-25T01:24:50.871 INFO:teuthology.orchestra.run.smithi169.stdout: "session_count": 1, 2024-04-25T01:24:50.871 INFO:teuthology.orchestra.run.smithi169.stdout: "session_add": 1, 2024-04-25T01:24:50.871 INFO:teuthology.orchestra.run.smithi169.stdout: "session_remove": 0, 2024-04-25T01:24:50.871 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_open": 1, 2024-04-25T01:24:50.871 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_stale": 0, 2024-04-25T01:24:50.871 INFO:teuthology.orchestra.run.smithi169.stdout: "total_load": 0, 2024-04-25T01:24:50.871 INFO:teuthology.orchestra.run.smithi169.stdout: "average_load": 0, 2024-04-25T01:24:50.871 INFO:teuthology.orchestra.run.smithi169.stdout: "avg_session_uptime": 2017, 2024-04-25T01:24:50.871 INFO:teuthology.orchestra.run.smithi169.stdout: "mdthresh_evicted": 0 2024-04-25T01:24:50.871 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.871 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-04-25T01:24:50.871 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-04-25T01:24:50.871 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-04-25T01:24:50.871 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T01:24:50.872 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-04-25T01:24:50.872 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T01:24:50.872 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-04-25T01:24:50.872 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T01:24:50.872 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T01:24:50.872 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T01:24:50.872 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T01:24:50.872 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T01:24:50.872 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-04-25T01:24:50.872 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T01:24:50.872 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T01:24:50.872 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-04-25T01:24:50.872 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-04-25T01:24:50.872 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-04-25T01:24:50.872 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-04-25T01:24:50.873 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T01:24:50.873 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T01:24:50.873 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T01:24:50.873 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T01:24:50.873 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T01:24:50.873 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-04-25T01:24:50.873 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-04-25T01:24:50.873 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-04-25T01:24:50.873 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T01:24:50.873 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T01:24:50.873 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-04-25T01:24:50.873 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-04-25T01:24:50.873 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-04-25T01:24:50.873 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-04-25T01:24:50.873 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T01:24:50.873 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-04-25T01:24:50.874 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T01:24:50.874 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-04-25T01:24:50.874 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 190219, 2024-04-25T01:24:50.874 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 54, 2024-04-25T01:24:50.874 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 0, 2024-04-25T01:24:50.874 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 0, 2024-04-25T01:24:50.874 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-04-25T01:24:50.874 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-04-25T01:24:50.874 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-04-25T01:24:50.874 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-04-25T01:24:50.874 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-04-25T01:24:50.874 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-04-25T01:24:50.874 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 7344, 2024-04-25T01:24:50.874 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 155, 2024-04-25T01:24:50.875 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T01:24:50.875 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-04-25T01:24:50.875 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-04-25T01:24:50.875 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-04-25T01:24:50.875 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 346466, 2024-04-25T01:24:50.875 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 2013, 2024-04-25T01:24:50.875 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-04-25T01:24:50.875 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-04-25T01:24:50.875 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-04-25T01:24:50.875 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-04-25T01:24:50.875 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.875 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter": { 2024-04-25T01:24:50.875 INFO:teuthology.orchestra.run.smithi169.stdout: "op_active": 0, 2024-04-25T01:24:50.875 INFO:teuthology.orchestra.run.smithi169.stdout: "op_laggy": 0, 2024-04-25T01:24:50.875 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send": 1979, 2024-04-25T01:24:50.875 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send_bytes": 21, 2024-04-25T01:24:50.876 INFO:teuthology.orchestra.run.smithi169.stdout: "op_resend": 3, 2024-04-25T01:24:50.876 INFO:teuthology.orchestra.run.smithi169.stdout: "op_reply": 1979, 2024-04-25T01:24:50.876 INFO:teuthology.orchestra.run.smithi169.stdout: "op_latency": { 2024-04-25T01:24:50.876 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1979, 2024-04-25T01:24:50.876 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 42.852096063, 2024-04-25T01:24:50.876 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.021653408 2024-04-25T01:24:50.876 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.876 INFO:teuthology.orchestra.run.smithi169.stdout: "op_inflight": 0, 2024-04-25T01:24:50.876 INFO:teuthology.orchestra.run.smithi169.stdout: "oplen_avg": { 2024-04-25T01:24:50.876 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1979, 2024-04-25T01:24:50.876 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 1980 2024-04-25T01:24:50.876 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.876 INFO:teuthology.orchestra.run.smithi169.stdout: "op": 1979, 2024-04-25T01:24:50.876 INFO:teuthology.orchestra.run.smithi169.stdout: "op_r": 1979, 2024-04-25T01:24:50.876 INFO:teuthology.orchestra.run.smithi169.stdout: "op_w": 0, 2024-04-25T01:24:50.876 INFO:teuthology.orchestra.run.smithi169.stdout: "op_rmw": 0, 2024-04-25T01:24:50.877 INFO:teuthology.orchestra.run.smithi169.stdout: "op_pg": 0, 2024-04-25T01:24:50.877 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_stat": 1122, 2024-04-25T01:24:50.877 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_create": 0, 2024-04-25T01:24:50.877 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_read": 855, 2024-04-25T01:24:50.877 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_write": 0, 2024-04-25T01:24:50.877 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writefull": 0, 2024-04-25T01:24:50.877 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writesame": 0, 2024-04-25T01:24:50.877 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_append": 0, 2024-04-25T01:24:50.877 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_zero": 0, 2024-04-25T01:24:50.877 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_truncate": 0, 2024-04-25T01:24:50.877 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_delete": 0, 2024-04-25T01:24:50.877 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_mapext": 0, 2024-04-25T01:24:50.877 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_sparse_read": 0, 2024-04-25T01:24:50.877 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_clonerange": 0, 2024-04-25T01:24:50.878 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_getxattr": 1, 2024-04-25T01:24:50.878 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_setxattr": 0, 2024-04-25T01:24:50.878 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_cmpxattr": 0, 2024-04-25T01:24:50.878 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_rmxattr": 0, 2024-04-25T01:24:50.878 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_resetxattrs": 0, 2024-04-25T01:24:50.878 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_call": 0, 2024-04-25T01:24:50.878 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_watch": 0, 2024-04-25T01:24:50.878 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_notify": 0, 2024-04-25T01:24:50.878 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T01:24:50.878 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls": 0, 2024-04-25T01:24:50.878 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls_filter": 0, 2024-04-25T01:24:50.878 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_other": 0, 2024-04-25T01:24:50.878 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_active": 0, 2024-04-25T01:24:50.878 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_send": 0, 2024-04-25T01:24:50.878 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_resend": 0, 2024-04-25T01:24:50.879 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_ping": 0, 2024-04-25T01:24:50.879 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_active": 0, 2024-04-25T01:24:50.879 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_send": 0, 2024-04-25T01:24:50.879 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_resend": 0, 2024-04-25T01:24:50.879 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_active": 0, 2024-04-25T01:24:50.879 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_send": 0, 2024-04-25T01:24:50.879 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_resend": 0, 2024-04-25T01:24:50.879 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_active": 0, 2024-04-25T01:24:50.879 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_send": 0, 2024-04-25T01:24:50.879 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_resend": 0, 2024-04-25T01:24:50.879 INFO:teuthology.orchestra.run.smithi169.stdout: "command_active": 0, 2024-04-25T01:24:50.879 INFO:teuthology.orchestra.run.smithi169.stdout: "command_send": 0, 2024-04-25T01:24:50.879 INFO:teuthology.orchestra.run.smithi169.stdout: "command_resend": 0, 2024-04-25T01:24:50.879 INFO:teuthology.orchestra.run.smithi169.stdout: "map_epoch": 89, 2024-04-25T01:24:50.880 INFO:teuthology.orchestra.run.smithi169.stdout: "map_full": 0, 2024-04-25T01:24:50.880 INFO:teuthology.orchestra.run.smithi169.stdout: "map_inc": 2, 2024-04-25T01:24:50.880 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_sessions": 5, 2024-04-25T01:24:50.880 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_open": 5, 2024-04-25T01:24:50.880 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_close": 0, 2024-04-25T01:24:50.880 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_laggy": 0, 2024-04-25T01:24:50.880 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_wr": 0, 2024-04-25T01:24:50.880 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_rd": 2, 2024-04-25T01:24:50.880 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_del": 0 2024-04-25T01:24:50.880 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.880 INFO:teuthology.orchestra.run.smithi169.stdout: "oft": { 2024-04-25T01:24:50.880 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_objs": 0, 2024-04-25T01:24:50.880 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_kv_pairs": 0, 2024-04-25T01:24:50.880 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_updates": 0, 2024-04-25T01:24:50.880 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_removes": 0 2024-04-25T01:24:50.880 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.881 INFO:teuthology.orchestra.run.smithi169.stdout: "osd-slow-ops": { 2024-04-25T01:24:50.881 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_ops_count": 0 2024-04-25T01:24:50.881 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.881 INFO:teuthology.orchestra.run.smithi169.stdout: "purge_queue": { 2024-04-25T01:24:50.881 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops": 0, 2024-04-25T01:24:50.881 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T01:24:50.881 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing": 0, 2024-04-25T01:24:50.881 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_high_water": 0, 2024-04-25T01:24:50.881 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed_ops": 0, 2024-04-25T01:24:50.881 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed": 0, 2024-04-25T01:24:50.881 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_item_in_journal": 0 2024-04-25T01:24:50.881 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.881 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T01:24:50.881 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T01:24:50.881 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T01:24:50.881 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T01:24:50.882 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 2193, 2024-04-25T01:24:50.882 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 14903996, 2024-04-25T01:24:50.882 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:50.882 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 2193, 2024-04-25T01:24:50.882 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T01:24:50.882 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T01:24:50.882 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 2193, 2024-04-25T01:24:50.882 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 14903996, 2024-04-25T01:24:50.882 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T01:24:50.882 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.882 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.882 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.882 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:50.882 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.883 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_bytes": { 2024-04-25T01:24:50.883 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T01:24:50.883 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T01:24:50.883 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T01:24:50.883 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T01:24:50.883 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T01:24:50.883 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:50.883 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T01:24:50.883 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 1979, 2024-04-25T01:24:50.883 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 14014440, 2024-04-25T01:24:50.883 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 1979, 2024-04-25T01:24:50.883 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 14014440, 2024-04-25T01:24:50.883 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T01:24:50.883 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.884 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.884 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.884 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:50.884 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.884 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_ops": { 2024-04-25T01:24:50.884 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T01:24:50.884 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 1024, 2024-04-25T01:24:50.884 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T01:24:50.884 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T01:24:50.884 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T01:24:50.884 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:50.884 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T01:24:50.884 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 1979, 2024-04-25T01:24:50.884 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 1979, 2024-04-25T01:24:50.884 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 1979, 2024-04-25T01:24:50.885 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 1979, 2024-04-25T01:24:50.885 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T01:24:50.885 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.885 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.885 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.885 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:50.885 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.885 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle": { 2024-04-25T01:24:50.885 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T01:24:50.885 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-25T01:24:50.885 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T01:24:50.885 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T01:24:50.885 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T01:24:50.885 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:50.885 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T01:24:50.886 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T01:24:50.886 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T01:24:50.886 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 0, 2024-04-25T01:24:50.886 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 0, 2024-04-25T01:24:50.886 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T01:24:50.886 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.886 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.886 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.886 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:50.886 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:50.886 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle-0x561eb08466e0": { 2024-04-25T01:24:50.886 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T01:24:50.886 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-25T01:24:50.887 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T01:24:50.887 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T01:24:50.887 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T01:24:50.887 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:50.887 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T01:24:50.887 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T01:24:50.887 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T01:24:50.887 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 0, 2024-04-25T01:24:50.887 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 0, 2024-04-25T01:24:50.887 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T01:24:50.887 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:50.887 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:50.887 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:50.887 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:50.887 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:50.887 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T01:24:51.481 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-25T01:24:51.481 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-25T01:24:51.481 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-04-25T01:24:51.481 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T01:24:51.481 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T01:24:51.737 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:24:52.059 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:51 smithi169 ceph-mon[32759]: pgmap v658: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:24:52.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:51 smithi040 ceph-mon[32518]: pgmap v658: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:24:52.249 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:51 smithi031 ceph-mon[28053]: pgmap v658: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T01:24:53.349 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T01:24:53.350 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T01:24:53.350 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 1101, 2024-04-25T01:24:53.350 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 1091, 2024-04-25T01:24:53.350 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 314036, 2024-04-25T01:24:53.350 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 4506369, 2024-04-25T01:24:53.350 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 9, 2024-04-25T01:24:53.350 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 4, 2024-04-25T01:24:53.350 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.112724677, 2024-04-25T01:24:53.350 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.039467022, 2024-04-25T01:24:53.350 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.045152481, 2024-04-25T01:24:53.350 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.025355293, 2024-04-25T01:24:53.350 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:53.351 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1089, 2024-04-25T01:24:53.351 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.084181986, 2024-04-25T01:24:53.351 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000077302 2024-04-25T01:24:53.351 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.351 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:53.351 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1142, 2024-04-25T01:24:53.351 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000311790, 2024-04-25T01:24:53.351 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000273 2024-04-25T01:24:53.351 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.351 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T01:24:53.351 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T01:24:53.351 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.351 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T01:24:53.351 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 840, 2024-04-25T01:24:53.352 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 985, 2024-04-25T01:24:53.352 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 220684, 2024-04-25T01:24:53.352 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 3132479, 2024-04-25T01:24:53.352 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 10, 2024-04-25T01:24:53.352 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 7, 2024-04-25T01:24:53.352 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.084831961, 2024-04-25T01:24:53.352 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.031274938, 2024-04-25T01:24:53.352 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.021158145, 2024-04-25T01:24:53.352 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.026803129, 2024-04-25T01:24:53.352 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:53.352 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 982, 2024-04-25T01:24:53.352 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.078027607, 2024-04-25T01:24:53.352 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000079457 2024-04-25T01:24:53.353 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.353 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:53.353 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.353 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.353 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.353 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.353 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-25T01:24:53.353 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 410656 2024-04-25T01:24:53.353 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.353 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T01:24:53.353 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 7442, 2024-04-25T01:24:53.353 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 7621, 2024-04-25T01:24:53.353 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 3076581, 2024-04-25T01:24:53.353 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 2663420, 2024-04-25T01:24:53.354 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 13, 2024-04-25T01:24:53.354 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 8, 2024-04-25T01:24:53.354 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.464673775, 2024-04-25T01:24:53.354 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.167752200, 2024-04-25T01:24:53.354 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.366579018, 2024-04-25T01:24:53.354 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.017936711, 2024-04-25T01:24:53.354 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:53.354 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 7622, 2024-04-25T01:24:53.354 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 1.089846065, 2024-04-25T01:24:53.354 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000142986 2024-04-25T01:24:53.354 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.354 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:53.354 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 5499, 2024-04-25T01:24:53.354 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.003036358, 2024-04-25T01:24:53.355 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000552 2024-04-25T01:24:53.355 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.355 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 546560, 2024-04-25T01:24:53.355 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 125088 2024-04-25T01:24:53.355 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.355 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-04-25T01:24:53.355 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 1, 2024-04-25T01:24:53.355 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-04-25T01:24:53.355 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.355 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-MDSRank": { 2024-04-25T01:24:53.355 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-25T01:24:53.355 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-25T01:24:53.355 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 690, 2024-04-25T01:24:53.355 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 1.028581026, 2024-04-25T01:24:53.356 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.001490697 2024-04-25T01:24:53.356 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:53.356 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.356 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-PurgeQueue": { 2024-04-25T01:24:53.356 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-25T01:24:53.356 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-25T01:24:53.356 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1370, 2024-04-25T01:24:53.356 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.042007114, 2024-04-25T01:24:53.356 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000030662 2024-04-25T01:24:53.356 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:53.356 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.356 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": { 2024-04-25T01:24:53.356 INFO:teuthology.orchestra.run.smithi169.stdout: "request": 173, 2024-04-25T01:24:53.356 INFO:teuthology.orchestra.run.smithi169.stdout: "reply": 165, 2024-04-25T01:24:53.357 INFO:teuthology.orchestra.run.smithi169.stdout: "reply_latency": { 2024-04-25T01:24:53.357 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 165, 2024-04-25T01:24:53.357 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.142027201, 2024-04-25T01:24:53.357 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000860770 2024-04-25T01:24:53.357 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.357 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_reply": 0, 2024-04-25T01:24:53.357 INFO:teuthology.orchestra.run.smithi169.stdout: "forward": 8, 2024-04-25T01:24:53.357 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_complete": 0, 2024-04-25T01:24:53.357 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_keys": 0, 2024-04-25T01:24:53.357 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_commit": 11, 2024-04-25T01:24:53.357 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_split": 0, 2024-04-25T01:24:53.357 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_merge": 0, 2024-04-25T01:24:53.357 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes": 15, 2024-04-25T01:24:53.358 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_top": 0, 2024-04-25T01:24:53.358 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_bottom": 15, 2024-04-25T01:24:53.358 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pin_tail": 0, 2024-04-25T01:24:53.358 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pinned": 14, 2024-04-25T01:24:53.358 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_expired": 10, 2024-04-25T01:24:53.358 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_with_caps": 2, 2024-04-25T01:24:53.358 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": 2, 2024-04-25T01:24:53.358 INFO:teuthology.orchestra.run.smithi169.stdout: "subtrees": 4, 2024-04-25T01:24:53.358 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse": 353, 2024-04-25T01:24:53.358 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_hit": 273, 2024-04-25T01:24:53.358 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_forward": 5, 2024-04-25T01:24:53.358 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_discover": 8, 2024-04-25T01:24:53.358 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_dir_fetch": 0, 2024-04-25T01:24:53.358 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_remote_ino": 0, 2024-04-25T01:24:53.359 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_lock": 0, 2024-04-25T01:24:53.359 INFO:teuthology.orchestra.run.smithi169.stdout: "load_cent": 0, 2024-04-25T01:24:53.359 INFO:teuthology.orchestra.run.smithi169.stdout: "q": 0, 2024-04-25T01:24:53.359 INFO:teuthology.orchestra.run.smithi169.stdout: "exported": 3, 2024-04-25T01:24:53.359 INFO:teuthology.orchestra.run.smithi169.stdout: "exported_inodes": 0, 2024-04-25T01:24:53.359 INFO:teuthology.orchestra.run.smithi169.stdout: "imported": 3, 2024-04-25T01:24:53.359 INFO:teuthology.orchestra.run.smithi169.stdout: "imported_inodes": 50, 2024-04-25T01:24:53.359 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_dir_fetch": 0, 2024-04-25T01:24:53.359 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_backtrace_fetch": 0, 2024-04-25T01:24:53.359 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_peer_discover": 0, 2024-04-25T01:24:53.359 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rfiles": 325, 2024-04-25T01:24:53.359 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rbytes": 13399302255, 2024-04-25T01:24:53.359 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rsnaps": 0, 2024-04-25T01:24:53.360 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T01:24:53.360 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_set_tag": 0, 2024-04-25T01:24:53.360 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T01:24:53.360 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_inotable_repaired": 0, 2024-04-25T01:24:53.360 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_inodes": 0, 2024-04-25T01:24:53.360 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T01:24:53.360 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T01:24:53.360 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_file_inodes": 0, 2024-04-25T01:24:53.360 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_inode_file_caps": 2, 2024-04-25T01:24:53.360 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_revoke": 38, 2024-04-25T01:24:53.360 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_grant": 533, 2024-04-25T01:24:53.360 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T01:24:53.360 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T01:24:53.361 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T01:24:53.361 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps": 5298, 2024-04-25T01:24:53.361 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps_dirty": 4822, 2024-04-25T01:24:53.361 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_cap_release": 4, 2024-04-25T01:24:53.361 INFO:teuthology.orchestra.run.smithi169.stdout: "process_request_cap_release": 108 2024-04-25T01:24:53.361 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.361 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache": { 2024-04-25T01:24:53.361 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update": 0, 2024-04-25T01:24:53.361 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update_receipt": 3, 2024-04-25T01:24:53.361 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_try_discover": 2, 2024-04-25T01:24:53.361 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_send_discover": 11, 2024-04-25T01:24:53.361 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_handle_discover": 0, 2024-04-25T01:24:53.361 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays": 0, 2024-04-25T01:24:53.361 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_delayed": 0, 2024-04-25T01:24:53.361 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_enqueuing": 0, 2024-04-25T01:24:53.362 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_created": 50, 2024-04-25T01:24:53.362 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_enqueued": 50, 2024-04-25T01:24:53.362 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_reintegrated": 0, 2024-04-25T01:24:53.362 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_migrated": 0, 2024-04-25T01:24:53.362 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_processing": 0, 2024-04-25T01:24:53.362 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_enqueued": 0, 2024-04-25T01:24:53.362 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_prioritized": 0, 2024-04-25T01:24:53.362 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_started": 0, 2024-04-25T01:24:53.362 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_completed": 0, 2024-04-25T01:24:53.362 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_path": 0, 2024-04-25T01:24:53.362 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_inode": 0, 2024-04-25T01:24:53.362 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T01:24:53.362 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_exportdir": 3, 2024-04-25T01:24:53.362 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_flush": 0, 2024-04-25T01:24:53.363 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragmentdir": 0, 2024-04-25T01:24:53.363 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragstats": 0, 2024-04-25T01:24:53.363 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_inodestats": 0 2024-04-25T01:24:53.363 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.363 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log": { 2024-04-25T01:24:53.363 INFO:teuthology.orchestra.run.smithi169.stdout: "evlrg": 0, 2024-04-25T01:24:53.363 INFO:teuthology.orchestra.run.smithi169.stdout: "evadd": 5390, 2024-04-25T01:24:53.363 INFO:teuthology.orchestra.run.smithi169.stdout: "evex": 0, 2024-04-25T01:24:53.363 INFO:teuthology.orchestra.run.smithi169.stdout: "evtrm": 0, 2024-04-25T01:24:53.363 INFO:teuthology.orchestra.run.smithi169.stdout: "ev": 5390, 2024-04-25T01:24:53.363 INFO:teuthology.orchestra.run.smithi169.stdout: "evexg": 0, 2024-04-25T01:24:53.363 INFO:teuthology.orchestra.run.smithi169.stdout: "evexd": 0, 2024-04-25T01:24:53.363 INFO:teuthology.orchestra.run.smithi169.stdout: "segadd": 7, 2024-04-25T01:24:53.363 INFO:teuthology.orchestra.run.smithi169.stdout: "segex": 0, 2024-04-25T01:24:53.364 INFO:teuthology.orchestra.run.smithi169.stdout: "segtrm": 0, 2024-04-25T01:24:53.364 INFO:teuthology.orchestra.run.smithi169.stdout: "seg": 7, 2024-04-25T01:24:53.364 INFO:teuthology.orchestra.run.smithi169.stdout: "segmjr": 2, 2024-04-25T01:24:53.364 INFO:teuthology.orchestra.run.smithi169.stdout: "segexg": 0, 2024-04-25T01:24:53.364 INFO:teuthology.orchestra.run.smithi169.stdout: "segexd": 0, 2024-04-25T01:24:53.364 INFO:teuthology.orchestra.run.smithi169.stdout: "expos": 4194304, 2024-04-25T01:24:53.364 INFO:teuthology.orchestra.run.smithi169.stdout: "wrpos": 10811909, 2024-04-25T01:24:53.364 INFO:teuthology.orchestra.run.smithi169.stdout: "rdpos": 0, 2024-04-25T01:24:53.364 INFO:teuthology.orchestra.run.smithi169.stdout: "jlat": { 2024-04-25T01:24:53.364 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 244, 2024-04-25T01:24:53.364 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 33.925946872, 2024-04-25T01:24:53.364 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.139040765 2024-04-25T01:24:53.364 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.364 INFO:teuthology.orchestra.run.smithi169.stdout: "replayed": 0 2024-04-25T01:24:53.365 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.365 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_mem": { 2024-04-25T01:24:53.365 INFO:teuthology.orchestra.run.smithi169.stdout: "ino": 19, 2024-04-25T01:24:53.365 INFO:teuthology.orchestra.run.smithi169.stdout: "ino+": 74, 2024-04-25T01:24:53.365 INFO:teuthology.orchestra.run.smithi169.stdout: "ino-": 55, 2024-04-25T01:24:53.365 INFO:teuthology.orchestra.run.smithi169.stdout: "dir": 18, 2024-04-25T01:24:53.365 INFO:teuthology.orchestra.run.smithi169.stdout: "dir+": 26, 2024-04-25T01:24:53.365 INFO:teuthology.orchestra.run.smithi169.stdout: "dir-": 8, 2024-04-25T01:24:53.365 INFO:teuthology.orchestra.run.smithi169.stdout: "dn": 15, 2024-04-25T01:24:53.365 INFO:teuthology.orchestra.run.smithi169.stdout: "dn+": 125, 2024-04-25T01:24:53.365 INFO:teuthology.orchestra.run.smithi169.stdout: "dn-": 110, 2024-04-25T01:24:53.365 INFO:teuthology.orchestra.run.smithi169.stdout: "cap": 2, 2024-04-25T01:24:53.365 INFO:teuthology.orchestra.run.smithi169.stdout: "cap+": 56, 2024-04-25T01:24:53.365 INFO:teuthology.orchestra.run.smithi169.stdout: "cap-": 54, 2024-04-25T01:24:53.366 INFO:teuthology.orchestra.run.smithi169.stdout: "rss": 57696, 2024-04-25T01:24:53.366 INFO:teuthology.orchestra.run.smithi169.stdout: "heap": 223612 2024-04-25T01:24:53.366 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.366 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_server": { 2024-04-25T01:24:53.366 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_client_request": 266, 2024-04-25T01:24:53.366 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_server_request": 1, 2024-04-25T01:24:53.366 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_request": 173, 2024-04-25T01:24:53.366 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_session": 11, 2024-04-25T01:24:53.366 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_peer_request": 56, 2024-04-25T01:24:53.366 INFO:teuthology.orchestra.run.smithi169.stdout: "req_create_latency": { 2024-04-25T01:24:53.366 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.366 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.366 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.367 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.367 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getattr_latency": { 2024-04-25T01:24:53.367 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-04-25T01:24:53.367 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.001460387, 2024-04-25T01:24:53.367 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000730193 2024-04-25T01:24:53.367 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.367 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getfilelock_latency": { 2024-04-25T01:24:53.367 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.367 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.367 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.367 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.367 INFO:teuthology.orchestra.run.smithi169.stdout: "req_link_latency": { 2024-04-25T01:24:53.367 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.367 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.367 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.368 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.368 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookup_latency": { 2024-04-25T01:24:53.368 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 106, 2024-04-25T01:24:53.368 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.052086497, 2024-04-25T01:24:53.368 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000491382 2024-04-25T01:24:53.368 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.368 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookuphash_latency": { 2024-04-25T01:24:53.368 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.368 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.368 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.368 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.368 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupino_latency": { 2024-04-25T01:24:53.368 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1, 2024-04-25T01:24:53.369 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000301793, 2024-04-25T01:24:53.369 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000301793 2024-04-25T01:24:53.369 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.369 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupname_latency": { 2024-04-25T01:24:53.369 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.369 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.369 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.369 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.369 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupparent_latency": { 2024-04-25T01:24:53.369 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.369 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.369 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.369 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.369 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupsnap_latency": { 2024-04-25T01:24:53.370 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.370 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.370 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.370 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.370 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lssnap_latency": { 2024-04-25T01:24:53.370 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.370 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.370 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.371 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.371 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mkdir_latency": { 2024-04-25T01:24:53.371 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.371 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.371 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.371 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.371 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mknod_latency": { 2024-04-25T01:24:53.371 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.371 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.371 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.372 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.372 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mksnap_latency": { 2024-04-25T01:24:53.372 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.372 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.372 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.372 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.372 INFO:teuthology.orchestra.run.smithi169.stdout: "req_open_latency": { 2024-04-25T01:24:53.372 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.372 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.372 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.372 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.372 INFO:teuthology.orchestra.run.smithi169.stdout: "req_readdir_latency": { 2024-04-25T01:24:53.372 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 6, 2024-04-25T01:24:53.372 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.004677118, 2024-04-25T01:24:53.373 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000779519 2024-04-25T01:24:53.373 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.373 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rename_latency": { 2024-04-25T01:24:53.373 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.373 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.373 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.373 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.373 INFO:teuthology.orchestra.run.smithi169.stdout: "req_renamesnap_latency": { 2024-04-25T01:24:53.373 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.373 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.373 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.373 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.373 INFO:teuthology.orchestra.run.smithi169.stdout: "req_snapdiff_latency": { 2024-04-25T01:24:53.373 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.374 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.374 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.374 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.374 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmdir_latency": { 2024-04-25T01:24:53.374 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.374 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.374 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.374 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.374 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmsnap_latency": { 2024-04-25T01:24:53.374 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.374 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.374 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.374 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.374 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmxattr_latency": { 2024-04-25T01:24:53.375 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.375 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.375 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.375 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.375 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setattr_latency": { 2024-04-25T01:24:53.375 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.375 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.375 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.375 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.375 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setdirlayout_latency": { 2024-04-25T01:24:53.375 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.375 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.375 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.375 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.376 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setfilelock_latency": { 2024-04-25T01:24:53.376 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.376 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.376 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.376 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.376 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setlayout_latency": { 2024-04-25T01:24:53.376 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.376 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.376 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.376 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.376 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setxattr_latency": { 2024-04-25T01:24:53.376 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.376 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.376 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.377 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.377 INFO:teuthology.orchestra.run.smithi169.stdout: "req_symlink_latency": { 2024-04-25T01:24:53.377 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.377 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.377 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.377 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.377 INFO:teuthology.orchestra.run.smithi169.stdout: "req_unlink_latency": { 2024-04-25T01:24:53.377 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 50, 2024-04-25T01:24:53.377 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.083501406, 2024-04-25T01:24:53.377 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.001670028 2024-04-25T01:24:53.377 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.377 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_revoke_eviction": 0, 2024-04-25T01:24:53.377 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_acquisition_throttle": 0, 2024-04-25T01:24:53.378 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getvxattr_latency": { 2024-04-25T01:24:53.378 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.378 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.378 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.378 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:53.378 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.378 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_sessions": { 2024-04-25T01:24:53.378 INFO:teuthology.orchestra.run.smithi169.stdout: "session_count": 1, 2024-04-25T01:24:53.378 INFO:teuthology.orchestra.run.smithi169.stdout: "session_add": 2, 2024-04-25T01:24:53.378 INFO:teuthology.orchestra.run.smithi169.stdout: "session_remove": 1, 2024-04-25T01:24:53.378 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_open": 1, 2024-04-25T01:24:53.378 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_stale": 0, 2024-04-25T01:24:53.378 INFO:teuthology.orchestra.run.smithi169.stdout: "total_load": 109, 2024-04-25T01:24:53.378 INFO:teuthology.orchestra.run.smithi169.stdout: "average_load": 109, 2024-04-25T01:24:53.379 INFO:teuthology.orchestra.run.smithi169.stdout: "avg_session_uptime": 532, 2024-04-25T01:24:53.379 INFO:teuthology.orchestra.run.smithi169.stdout: "mdthresh_evicted": 0 2024-04-25T01:24:53.379 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.379 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-04-25T01:24:53.379 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-04-25T01:24:53.379 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-04-25T01:24:53.379 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T01:24:53.379 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-04-25T01:24:53.379 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T01:24:53.379 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-04-25T01:24:53.379 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T01:24:53.379 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T01:24:53.379 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T01:24:53.379 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T01:24:53.380 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T01:24:53.380 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-04-25T01:24:53.380 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T01:24:53.380 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T01:24:53.380 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-04-25T01:24:53.380 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-04-25T01:24:53.380 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-04-25T01:24:53.380 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-04-25T01:24:53.380 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T01:24:53.380 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T01:24:53.380 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T01:24:53.380 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T01:24:53.380 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T01:24:53.380 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-04-25T01:24:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-04-25T01:24:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-04-25T01:24:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T01:24:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T01:24:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-04-25T01:24:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-04-25T01:24:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-04-25T01:24:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-04-25T01:24:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T01:24:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-04-25T01:24:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T01:24:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-04-25T01:24:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 292285, 2024-04-25T01:24:53.381 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 119, 2024-04-25T01:24:53.382 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 0, 2024-04-25T01:24:53.382 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 0, 2024-04-25T01:24:53.382 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-04-25T01:24:53.382 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-04-25T01:24:53.382 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-04-25T01:24:53.382 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-04-25T01:24:53.382 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-04-25T01:24:53.382 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-04-25T01:24:53.382 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 7344, 2024-04-25T01:24:53.382 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 155, 2024-04-25T01:24:53.382 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T01:24:53.382 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-04-25T01:24:53.382 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-04-25T01:24:53.383 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-04-25T01:24:53.383 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 77557, 2024-04-25T01:24:53.383 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 342, 2024-04-25T01:24:53.383 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-04-25T01:24:53.383 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-04-25T01:24:53.383 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-04-25T01:24:53.383 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-04-25T01:24:53.383 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.383 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter": { 2024-04-25T01:24:53.383 INFO:teuthology.orchestra.run.smithi169.stdout: "op_active": 0, 2024-04-25T01:24:53.383 INFO:teuthology.orchestra.run.smithi169.stdout: "op_laggy": 0, 2024-04-25T01:24:53.383 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send": 1730, 2024-04-25T01:24:53.383 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send_bytes": 6653161, 2024-04-25T01:24:53.383 INFO:teuthology.orchestra.run.smithi169.stdout: "op_resend": 33, 2024-04-25T01:24:53.383 INFO:teuthology.orchestra.run.smithi169.stdout: "op_reply": 1730, 2024-04-25T01:24:53.384 INFO:teuthology.orchestra.run.smithi169.stdout: "op_latency": { 2024-04-25T01:24:53.384 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1730, 2024-04-25T01:24:53.384 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 66.385601768, 2024-04-25T01:24:53.384 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.038373180 2024-04-25T01:24:53.384 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.384 INFO:teuthology.orchestra.run.smithi169.stdout: "op_inflight": 0, 2024-04-25T01:24:53.384 INFO:teuthology.orchestra.run.smithi169.stdout: "oplen_avg": { 2024-04-25T01:24:53.384 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1730, 2024-04-25T01:24:53.384 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 1906 2024-04-25T01:24:53.384 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.384 INFO:teuthology.orchestra.run.smithi169.stdout: "op": 1730, 2024-04-25T01:24:53.384 INFO:teuthology.orchestra.run.smithi169.stdout: "op_r": 2, 2024-04-25T01:24:53.384 INFO:teuthology.orchestra.run.smithi169.stdout: "op_w": 1728, 2024-04-25T01:24:53.384 INFO:teuthology.orchestra.run.smithi169.stdout: "op_rmw": 0, 2024-04-25T01:24:53.385 INFO:teuthology.orchestra.run.smithi169.stdout: "op_pg": 0, 2024-04-25T01:24:53.385 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_stat": 0, 2024-04-25T01:24:53.385 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_create": 10, 2024-04-25T01:24:53.385 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_read": 2, 2024-04-25T01:24:53.385 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_write": 246, 2024-04-25T01:24:53.385 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writefull": 31, 2024-04-25T01:24:53.385 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writesame": 0, 2024-04-25T01:24:53.385 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_append": 0, 2024-04-25T01:24:53.385 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_zero": 0, 2024-04-25T01:24:53.385 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_truncate": 0, 2024-04-25T01:24:53.385 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_delete": 1289, 2024-04-25T01:24:53.385 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_mapext": 0, 2024-04-25T01:24:53.385 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_sparse_read": 0, 2024-04-25T01:24:53.385 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_clonerange": 0, 2024-04-25T01:24:53.385 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_getxattr": 0, 2024-04-25T01:24:53.386 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_setxattr": 20, 2024-04-25T01:24:53.386 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_cmpxattr": 0, 2024-04-25T01:24:53.386 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_rmxattr": 0, 2024-04-25T01:24:53.386 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_resetxattrs": 0, 2024-04-25T01:24:53.386 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_call": 0, 2024-04-25T01:24:53.386 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_watch": 0, 2024-04-25T01:24:53.386 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_notify": 0, 2024-04-25T01:24:53.386 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T01:24:53.386 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls": 0, 2024-04-25T01:24:53.386 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls_filter": 0, 2024-04-25T01:24:53.386 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_other": 0, 2024-04-25T01:24:53.386 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_active": 0, 2024-04-25T01:24:53.386 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_send": 0, 2024-04-25T01:24:53.387 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_resend": 0, 2024-04-25T01:24:53.387 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_ping": 0, 2024-04-25T01:24:53.387 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_active": 0, 2024-04-25T01:24:53.387 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_send": 0, 2024-04-25T01:24:53.387 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_resend": 0, 2024-04-25T01:24:53.387 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_active": 0, 2024-04-25T01:24:53.387 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_send": 0, 2024-04-25T01:24:53.387 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_resend": 0, 2024-04-25T01:24:53.387 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_active": 0, 2024-04-25T01:24:53.387 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_send": 0, 2024-04-25T01:24:53.387 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_resend": 0, 2024-04-25T01:24:53.387 INFO:teuthology.orchestra.run.smithi169.stdout: "command_active": 0, 2024-04-25T01:24:53.387 INFO:teuthology.orchestra.run.smithi169.stdout: "command_send": 0, 2024-04-25T01:24:53.387 INFO:teuthology.orchestra.run.smithi169.stdout: "command_resend": 0, 2024-04-25T01:24:53.387 INFO:teuthology.orchestra.run.smithi169.stdout: "map_epoch": 89, 2024-04-25T01:24:53.388 INFO:teuthology.orchestra.run.smithi169.stdout: "map_full": 0, 2024-04-25T01:24:53.388 INFO:teuthology.orchestra.run.smithi169.stdout: "map_inc": 2, 2024-04-25T01:24:53.388 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_sessions": 12, 2024-04-25T01:24:53.388 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_open": 12, 2024-04-25T01:24:53.388 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_close": 0, 2024-04-25T01:24:53.388 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_laggy": 0, 2024-04-25T01:24:53.388 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_wr": 226, 2024-04-25T01:24:53.388 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_rd": 0, 2024-04-25T01:24:53.388 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_del": 82 2024-04-25T01:24:53.388 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.388 INFO:teuthology.orchestra.run.smithi169.stdout: "oft": { 2024-04-25T01:24:53.388 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_objs": 1, 2024-04-25T01:24:53.388 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_kv_pairs": 0, 2024-04-25T01:24:53.389 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_updates": 0, 2024-04-25T01:24:53.389 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_removes": 0 2024-04-25T01:24:53.389 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.389 INFO:teuthology.orchestra.run.smithi169.stdout: "osd-slow-ops": { 2024-04-25T01:24:53.389 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_ops_count": 0 2024-04-25T01:24:53.389 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.389 INFO:teuthology.orchestra.run.smithi169.stdout: "purge_queue": { 2024-04-25T01:24:53.389 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops": 0, 2024-04-25T01:24:53.389 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops_high_water": 51, 2024-04-25T01:24:53.389 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing": 0, 2024-04-25T01:24:53.389 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_high_water": 1, 2024-04-25T01:24:53.389 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed_ops": 1276, 2024-04-25T01:24:53.389 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed": 50, 2024-04-25T01:24:53.389 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_item_in_journal": 0 2024-04-25T01:24:53.389 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.390 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T01:24:53.390 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T01:24:53.390 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T01:24:53.390 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T01:24:53.390 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 9379, 2024-04-25T01:24:53.390 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 2848407, 2024-04-25T01:24:53.390 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:53.390 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 9379, 2024-04-25T01:24:53.390 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T01:24:53.390 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T01:24:53.390 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 9379, 2024-04-25T01:24:53.390 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 2848407, 2024-04-25T01:24:53.390 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T01:24:53.391 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.391 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.391 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.391 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:53.391 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.391 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_bytes": { 2024-04-25T01:24:53.391 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T01:24:53.391 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T01:24:53.391 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T01:24:53.391 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T01:24:53.391 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T01:24:53.391 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:53.391 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T01:24:53.391 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 1730, 2024-04-25T01:24:53.392 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 6658211, 2024-04-25T01:24:53.392 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 1730, 2024-04-25T01:24:53.392 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 6658211, 2024-04-25T01:24:53.392 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T01:24:53.392 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.392 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.392 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.392 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:53.392 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.392 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_ops": { 2024-04-25T01:24:53.392 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T01:24:53.392 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 1024, 2024-04-25T01:24:53.392 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T01:24:53.393 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T01:24:53.393 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T01:24:53.393 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:53.393 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T01:24:53.393 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 1730, 2024-04-25T01:24:53.393 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 1730, 2024-04-25T01:24:53.393 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 1730, 2024-04-25T01:24:53.393 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 1730, 2024-04-25T01:24:53.393 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T01:24:53.393 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.393 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.393 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.393 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:53.393 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.393 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle": { 2024-04-25T01:24:53.394 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T01:24:53.394 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-25T01:24:53.394 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T01:24:53.394 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 50, 2024-04-25T01:24:53.394 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 5050, 2024-04-25T01:24:53.394 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:53.394 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 50, 2024-04-25T01:24:53.394 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T01:24:53.394 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T01:24:53.394 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 2, 2024-04-25T01:24:53.394 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 5050, 2024-04-25T01:24:53.394 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T01:24:53.394 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.394 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.395 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.395 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:53.395 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:53.395 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle-0x55ae039f4500": { 2024-04-25T01:24:53.395 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T01:24:53.395 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-25T01:24:53.395 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T01:24:53.395 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 5390, 2024-04-25T01:24:53.395 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 6617605, 2024-04-25T01:24:53.395 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:53.395 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 5390, 2024-04-25T01:24:53.395 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T01:24:53.395 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T01:24:53.395 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 244, 2024-04-25T01:24:53.395 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 6617605, 2024-04-25T01:24:53.396 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T01:24:53.396 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:53.396 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:53.396 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:53.396 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:53.396 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:53.396 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T01:24:54.089 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:53 smithi169 ceph-mon[32759]: pgmap v659: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T01:24:54.115 INFO:tasks.check_counter:Daemon mds.f mds.exported=3 2024-04-25T01:24:54.115 INFO:tasks.check_counter:Daemon mds.f mds.imported=3 2024-04-25T01:24:54.115 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-04-25T01:24:54.115 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T01:24:54.115 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T01:24:54.191 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:53 smithi040 ceph-mon[32518]: pgmap v659: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T01:24:54.249 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:53 smithi031 ceph-mon[28053]: pgmap v659: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T01:24:54.360 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:24:55.913 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T01:24:55.913 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T01:24:55.913 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 3520, 2024-04-25T01:24:55.913 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 3662, 2024-04-25T01:24:55.913 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 1558176, 2024-04-25T01:24:55.913 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 14197038, 2024-04-25T01:24:55.913 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 10, 2024-04-25T01:24:55.913 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 4, 2024-04-25T01:24:55.913 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.330596018, 2024-04-25T01:24:55.913 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.121793563, 2024-04-25T01:24:55.913 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.125539960, 2024-04-25T01:24:55.913 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.072012602, 2024-04-25T01:24:55.914 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:55.914 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 3660, 2024-04-25T01:24:55.914 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.256319797, 2024-04-25T01:24:55.914 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000070032 2024-04-25T01:24:55.914 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.914 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:55.914 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 3046, 2024-04-25T01:24:55.914 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000659325, 2024-04-25T01:24:55.914 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000216 2024-04-25T01:24:55.914 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.914 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T01:24:55.914 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T01:24:55.915 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.915 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T01:24:55.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 8271, 2024-04-25T01:24:55.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 10822, 2024-04-25T01:24:55.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 3271477, 2024-04-25T01:24:55.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 3904941, 2024-04-25T01:24:55.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 12, 2024-04-25T01:24:55.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 7, 2024-04-25T01:24:55.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.625516463, 2024-04-25T01:24:55.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.228149261, 2024-04-25T01:24:55.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.408268886, 2024-04-25T01:24:55.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.017414356, 2024-04-25T01:24:55.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:55.915 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 10822, 2024-04-25T01:24:55.915 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 1.103869069, 2024-04-25T01:24:55.916 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000102002 2024-04-25T01:24:55.916 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.916 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:55.916 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 6777, 2024-04-25T01:24:55.916 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.003276521, 2024-04-25T01:24:55.916 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000483 2024-04-25T01:24:55.916 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.916 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 552496, 2024-04-25T01:24:55.916 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 123488 2024-04-25T01:24:55.916 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.916 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T01:24:55.916 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 1191, 2024-04-25T01:24:55.916 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 1331, 2024-04-25T01:24:55.916 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 409253, 2024-04-25T01:24:55.916 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 2304439, 2024-04-25T01:24:55.917 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 8, 2024-04-25T01:24:55.917 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 6, 2024-04-25T01:24:55.917 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.119488358, 2024-04-25T01:24:55.917 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.059933937, 2024-04-25T01:24:55.917 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.044477886, 2024-04-25T01:24:55.917 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.012759121, 2024-04-25T01:24:55.917 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:55.917 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1329, 2024-04-25T01:24:55.917 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.104001964, 2024-04-25T01:24:55.917 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000078255 2024-04-25T01:24:55.917 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.917 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:55.917 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1435, 2024-04-25T01:24:55.918 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000269694, 2024-04-25T01:24:55.918 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000187 2024-04-25T01:24:55.918 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.918 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T01:24:55.918 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 331952 2024-04-25T01:24:55.918 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.918 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-04-25T01:24:55.918 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 1, 2024-04-25T01:24:55.918 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-04-25T01:24:55.918 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.918 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-MDSRank": { 2024-04-25T01:24:55.918 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-25T01:24:55.918 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-25T01:24:55.919 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2880, 2024-04-25T01:24:55.919 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 2.570370483, 2024-04-25T01:24:55.919 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000892489 2024-04-25T01:24:55.919 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:55.919 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.919 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-PurgeQueue": { 2024-04-25T01:24:55.919 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-25T01:24:55.919 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-25T01:24:55.919 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1571, 2024-04-25T01:24:55.919 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.060615110, 2024-04-25T01:24:55.919 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000038583 2024-04-25T01:24:55.919 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:55.919 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.919 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": { 2024-04-25T01:24:55.919 INFO:teuthology.orchestra.run.smithi169.stdout: "request": 646, 2024-04-25T01:24:55.920 INFO:teuthology.orchestra.run.smithi169.stdout: "reply": 637, 2024-04-25T01:24:55.920 INFO:teuthology.orchestra.run.smithi169.stdout: "reply_latency": { 2024-04-25T01:24:55.920 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 637, 2024-04-25T01:24:55.920 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 6.073729035, 2024-04-25T01:24:55.920 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.009534896 2024-04-25T01:24:55.920 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.920 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_reply": 0, 2024-04-25T01:24:55.920 INFO:teuthology.orchestra.run.smithi169.stdout: "forward": 9, 2024-04-25T01:24:55.920 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_complete": 0, 2024-04-25T01:24:55.920 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_keys": 0, 2024-04-25T01:24:55.920 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_commit": 20, 2024-04-25T01:24:55.920 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_split": 1, 2024-04-25T01:24:55.920 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_merge": 0, 2024-04-25T01:24:55.920 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes": 61, 2024-04-25T01:24:55.921 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_top": 2, 2024-04-25T01:24:55.921 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_bottom": 45, 2024-04-25T01:24:55.921 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pin_tail": 14, 2024-04-25T01:24:55.921 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pinned": 17, 2024-04-25T01:24:55.921 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_expired": 394, 2024-04-25T01:24:55.921 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_with_caps": 2, 2024-04-25T01:24:55.921 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": 2, 2024-04-25T01:24:55.921 INFO:teuthology.orchestra.run.smithi169.stdout: "subtrees": 5, 2024-04-25T01:24:55.921 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse": 888, 2024-04-25T01:24:55.921 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_hit": 842, 2024-04-25T01:24:55.921 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_forward": 9, 2024-04-25T01:24:55.921 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_discover": 6, 2024-04-25T01:24:55.921 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_dir_fetch": 0, 2024-04-25T01:24:55.921 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_remote_ino": 0, 2024-04-25T01:24:55.922 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_lock": 0, 2024-04-25T01:24:55.922 INFO:teuthology.orchestra.run.smithi169.stdout: "load_cent": 3, 2024-04-25T01:24:55.922 INFO:teuthology.orchestra.run.smithi169.stdout: "q": 0, 2024-04-25T01:24:55.922 INFO:teuthology.orchestra.run.smithi169.stdout: "exported": 10, 2024-04-25T01:24:55.922 INFO:teuthology.orchestra.run.smithi169.stdout: "exported_inodes": 378, 2024-04-25T01:24:55.922 INFO:teuthology.orchestra.run.smithi169.stdout: "imported": 12, 2024-04-25T01:24:55.922 INFO:teuthology.orchestra.run.smithi169.stdout: "imported_inodes": 736, 2024-04-25T01:24:55.922 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_dir_fetch": 0, 2024-04-25T01:24:55.922 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_backtrace_fetch": 0, 2024-04-25T01:24:55.922 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_peer_discover": 0, 2024-04-25T01:24:55.923 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rfiles": 325, 2024-04-25T01:24:55.923 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rbytes": 13399302255, 2024-04-25T01:24:55.923 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rsnaps": 0, 2024-04-25T01:24:55.923 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T01:24:55.923 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_set_tag": 0, 2024-04-25T01:24:55.923 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T01:24:55.923 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_inotable_repaired": 0, 2024-04-25T01:24:55.923 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_inodes": 0, 2024-04-25T01:24:55.923 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T01:24:55.923 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T01:24:55.923 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_file_inodes": 0, 2024-04-25T01:24:55.923 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_inode_file_caps": 4, 2024-04-25T01:24:55.923 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_revoke": 177, 2024-04-25T01:24:55.923 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_grant": 1874, 2024-04-25T01:24:55.924 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T01:24:55.924 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T01:24:55.924 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T01:24:55.924 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps": 5914, 2024-04-25T01:24:55.924 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps_dirty": 5257, 2024-04-25T01:24:55.924 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_cap_release": 7, 2024-04-25T01:24:55.924 INFO:teuthology.orchestra.run.smithi169.stdout: "process_request_cap_release": 444 2024-04-25T01:24:55.924 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.924 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache": { 2024-04-25T01:24:55.924 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update": 0, 2024-04-25T01:24:55.924 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update_receipt": 3, 2024-04-25T01:24:55.924 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_try_discover": 2, 2024-04-25T01:24:55.924 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_send_discover": 8, 2024-04-25T01:24:55.924 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_handle_discover": 6, 2024-04-25T01:24:55.924 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays": 2, 2024-04-25T01:24:55.925 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_delayed": 0, 2024-04-25T01:24:55.925 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_enqueuing": 0, 2024-04-25T01:24:55.925 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_created": 331, 2024-04-25T01:24:55.925 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_enqueued": 329, 2024-04-25T01:24:55.925 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_reintegrated": 0, 2024-04-25T01:24:55.925 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_migrated": 0, 2024-04-25T01:24:55.925 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_processing": 0, 2024-04-25T01:24:55.925 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_enqueued": 0, 2024-04-25T01:24:55.925 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_prioritized": 0, 2024-04-25T01:24:55.925 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_started": 0, 2024-04-25T01:24:55.925 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_completed": 0, 2024-04-25T01:24:55.925 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_path": 0, 2024-04-25T01:24:55.925 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_inode": 0, 2024-04-25T01:24:55.925 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T01:24:55.925 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_exportdir": 10, 2024-04-25T01:24:55.926 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_flush": 0, 2024-04-25T01:24:55.926 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragmentdir": 1, 2024-04-25T01:24:55.926 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragstats": 0, 2024-04-25T01:24:55.926 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_inodestats": 0 2024-04-25T01:24:55.926 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.926 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log": { 2024-04-25T01:24:55.926 INFO:teuthology.orchestra.run.smithi169.stdout: "evlrg": 0, 2024-04-25T01:24:55.926 INFO:teuthology.orchestra.run.smithi169.stdout: "evadd": 6713, 2024-04-25T01:24:55.926 INFO:teuthology.orchestra.run.smithi169.stdout: "evex": 0, 2024-04-25T01:24:55.926 INFO:teuthology.orchestra.run.smithi169.stdout: "evtrm": 0, 2024-04-25T01:24:55.926 INFO:teuthology.orchestra.run.smithi169.stdout: "ev": 6713, 2024-04-25T01:24:55.926 INFO:teuthology.orchestra.run.smithi169.stdout: "evexg": 0, 2024-04-25T01:24:55.926 INFO:teuthology.orchestra.run.smithi169.stdout: "evexd": 0, 2024-04-25T01:24:55.927 INFO:teuthology.orchestra.run.smithi169.stdout: "segadd": 8, 2024-04-25T01:24:55.927 INFO:teuthology.orchestra.run.smithi169.stdout: "segex": 0, 2024-04-25T01:24:55.927 INFO:teuthology.orchestra.run.smithi169.stdout: "segtrm": 0, 2024-04-25T01:24:55.927 INFO:teuthology.orchestra.run.smithi169.stdout: "seg": 8, 2024-04-25T01:24:55.927 INFO:teuthology.orchestra.run.smithi169.stdout: "segmjr": 2, 2024-04-25T01:24:55.927 INFO:teuthology.orchestra.run.smithi169.stdout: "segexg": 0, 2024-04-25T01:24:55.927 INFO:teuthology.orchestra.run.smithi169.stdout: "segexd": 0, 2024-04-25T01:24:55.927 INFO:teuthology.orchestra.run.smithi169.stdout: "expos": 4194304, 2024-04-25T01:24:55.927 INFO:teuthology.orchestra.run.smithi169.stdout: "wrpos": 18206829, 2024-04-25T01:24:55.927 INFO:teuthology.orchestra.run.smithi169.stdout: "rdpos": 0, 2024-04-25T01:24:55.927 INFO:teuthology.orchestra.run.smithi169.stdout: "jlat": { 2024-04-25T01:24:55.927 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1349, 2024-04-25T01:24:55.927 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 97.701882361, 2024-04-25T01:24:55.927 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.072425413 2024-04-25T01:24:55.927 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.928 INFO:teuthology.orchestra.run.smithi169.stdout: "replayed": 0 2024-04-25T01:24:55.928 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.928 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_mem": { 2024-04-25T01:24:55.928 INFO:teuthology.orchestra.run.smithi169.stdout: "ino": 23, 2024-04-25T01:24:55.928 INFO:teuthology.orchestra.run.smithi169.stdout: "ino+": 628, 2024-04-25T01:24:55.928 INFO:teuthology.orchestra.run.smithi169.stdout: "ino-": 605, 2024-04-25T01:24:55.928 INFO:teuthology.orchestra.run.smithi169.stdout: "dir": 29, 2024-04-25T01:24:55.928 INFO:teuthology.orchestra.run.smithi169.stdout: "dir+": 188, 2024-04-25T01:24:55.928 INFO:teuthology.orchestra.run.smithi169.stdout: "dir-": 159, 2024-04-25T01:24:55.928 INFO:teuthology.orchestra.run.smithi169.stdout: "dn": 61, 2024-04-25T01:24:55.928 INFO:teuthology.orchestra.run.smithi169.stdout: "dn+": 1231, 2024-04-25T01:24:55.928 INFO:teuthology.orchestra.run.smithi169.stdout: "dn-": 1170, 2024-04-25T01:24:55.928 INFO:teuthology.orchestra.run.smithi169.stdout: "cap": 2, 2024-04-25T01:24:55.928 INFO:teuthology.orchestra.run.smithi169.stdout: "cap+": 620, 2024-04-25T01:24:55.928 INFO:teuthology.orchestra.run.smithi169.stdout: "cap-": 618, 2024-04-25T01:24:55.929 INFO:teuthology.orchestra.run.smithi169.stdout: "rss": 62192, 2024-04-25T01:24:55.929 INFO:teuthology.orchestra.run.smithi169.stdout: "heap": 223612 2024-04-25T01:24:55.929 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.929 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_server": { 2024-04-25T01:24:55.929 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_client_request": 1025, 2024-04-25T01:24:55.929 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_server_request": 100, 2024-04-25T01:24:55.929 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_request": 646, 2024-04-25T01:24:55.929 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_session": 16, 2024-04-25T01:24:55.929 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_peer_request": 245, 2024-04-25T01:24:55.929 INFO:teuthology.orchestra.run.smithi169.stdout: "req_create_latency": { 2024-04-25T01:24:55.929 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 133, 2024-04-25T01:24:55.929 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.146699441, 2024-04-25T01:24:55.929 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.001103003 2024-04-25T01:24:55.929 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.929 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getattr_latency": { 2024-04-25T01:24:55.930 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1, 2024-04-25T01:24:55.930 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000932169, 2024-04-25T01:24:55.930 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000932169 2024-04-25T01:24:55.930 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.930 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getfilelock_latency": { 2024-04-25T01:24:55.930 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.930 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.930 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.930 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.930 INFO:teuthology.orchestra.run.smithi169.stdout: "req_link_latency": { 2024-04-25T01:24:55.930 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.930 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.930 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.931 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.931 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookup_latency": { 2024-04-25T01:24:55.931 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 155, 2024-04-25T01:24:55.931 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 2.408818688, 2024-04-25T01:24:55.931 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.015540765 2024-04-25T01:24:55.931 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.931 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookuphash_latency": { 2024-04-25T01:24:55.931 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.931 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.931 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.931 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.931 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupino_latency": { 2024-04-25T01:24:55.931 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1, 2024-04-25T01:24:55.931 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000094928, 2024-04-25T01:24:55.931 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000094928 2024-04-25T01:24:55.932 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.932 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupname_latency": { 2024-04-25T01:24:55.932 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.932 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.932 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.932 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.932 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupparent_latency": { 2024-04-25T01:24:55.932 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.932 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.932 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.932 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.932 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupsnap_latency": { 2024-04-25T01:24:55.932 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.933 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.933 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.933 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.933 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lssnap_latency": { 2024-04-25T01:24:55.933 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.933 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.933 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.934 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.934 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mkdir_latency": { 2024-04-25T01:24:55.934 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.934 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.934 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.934 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.934 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mknod_latency": { 2024-04-25T01:24:55.934 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.934 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.934 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.934 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.935 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mksnap_latency": { 2024-04-25T01:24:55.935 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.935 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.935 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.935 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.935 INFO:teuthology.orchestra.run.smithi169.stdout: "req_open_latency": { 2024-04-25T01:24:55.935 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.935 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.935 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.935 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.935 INFO:teuthology.orchestra.run.smithi169.stdout: "req_readdir_latency": { 2024-04-25T01:24:55.935 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 16, 2024-04-25T01:24:55.935 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.059664875, 2024-04-25T01:24:55.936 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.003729054 2024-04-25T01:24:55.936 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.936 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rename_latency": { 2024-04-25T01:24:55.936 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 18, 2024-04-25T01:24:55.936 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.042819780, 2024-04-25T01:24:55.936 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.002378876 2024-04-25T01:24:55.936 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.936 INFO:teuthology.orchestra.run.smithi169.stdout: "req_renamesnap_latency": { 2024-04-25T01:24:55.936 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.936 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.936 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.936 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.936 INFO:teuthology.orchestra.run.smithi169.stdout: "req_snapdiff_latency": { 2024-04-25T01:24:55.936 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.937 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.937 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.937 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.937 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmdir_latency": { 2024-04-25T01:24:55.937 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 83, 2024-04-25T01:24:55.937 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 2.982753890, 2024-04-25T01:24:55.937 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.035936793 2024-04-25T01:24:55.937 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.937 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmsnap_latency": { 2024-04-25T01:24:55.937 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.937 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.937 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.937 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.938 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmxattr_latency": { 2024-04-25T01:24:55.938 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.938 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.938 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.938 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.938 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setattr_latency": { 2024-04-25T01:24:55.938 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.938 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.938 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.938 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.938 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setdirlayout_latency": { 2024-04-25T01:24:55.938 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.938 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.939 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.939 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.939 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setfilelock_latency": { 2024-04-25T01:24:55.939 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.939 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.939 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.939 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.939 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setlayout_latency": { 2024-04-25T01:24:55.939 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.939 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.939 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.939 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.939 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setxattr_latency": { 2024-04-25T01:24:55.939 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.940 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.940 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.940 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.940 INFO:teuthology.orchestra.run.smithi169.stdout: "req_symlink_latency": { 2024-04-25T01:24:55.940 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.940 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.940 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.940 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.940 INFO:teuthology.orchestra.run.smithi169.stdout: "req_unlink_latency": { 2024-04-25T01:24:55.940 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 230, 2024-04-25T01:24:55.940 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.431945264, 2024-04-25T01:24:55.940 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.001878022 2024-04-25T01:24:55.940 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.940 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_revoke_eviction": 0, 2024-04-25T01:24:55.941 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_acquisition_throttle": 0, 2024-04-25T01:24:55.941 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getvxattr_latency": { 2024-04-25T01:24:55.941 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.941 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.941 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.941 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:55.941 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.941 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_sessions": { 2024-04-25T01:24:55.941 INFO:teuthology.orchestra.run.smithi169.stdout: "session_count": 1, 2024-04-25T01:24:55.941 INFO:teuthology.orchestra.run.smithi169.stdout: "session_add": 1, 2024-04-25T01:24:55.941 INFO:teuthology.orchestra.run.smithi169.stdout: "session_remove": 0, 2024-04-25T01:24:55.941 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_open": 1, 2024-04-25T01:24:55.941 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_stale": 0, 2024-04-25T01:24:55.941 INFO:teuthology.orchestra.run.smithi169.stdout: "total_load": 372, 2024-04-25T01:24:55.942 INFO:teuthology.orchestra.run.smithi169.stdout: "average_load": 372, 2024-04-25T01:24:55.942 INFO:teuthology.orchestra.run.smithi169.stdout: "avg_session_uptime": 533, 2024-04-25T01:24:55.942 INFO:teuthology.orchestra.run.smithi169.stdout: "mdthresh_evicted": 0 2024-04-25T01:24:55.942 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.942 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-04-25T01:24:55.942 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-04-25T01:24:55.942 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-04-25T01:24:55.942 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T01:24:55.942 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-04-25T01:24:55.942 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T01:24:55.942 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-04-25T01:24:55.942 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T01:24:55.942 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T01:24:55.943 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T01:24:55.943 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T01:24:55.943 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T01:24:55.943 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-04-25T01:24:55.943 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T01:24:55.943 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T01:24:55.943 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-04-25T01:24:55.943 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-04-25T01:24:55.943 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-04-25T01:24:55.943 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-04-25T01:24:55.943 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T01:24:55.943 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T01:24:55.943 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T01:24:55.943 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T01:24:55.944 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T01:24:55.944 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-04-25T01:24:55.944 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-04-25T01:24:55.944 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-04-25T01:24:55.944 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T01:24:55.944 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T01:24:55.944 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-04-25T01:24:55.944 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-04-25T01:24:55.944 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-04-25T01:24:55.944 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-04-25T01:24:55.944 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T01:24:55.944 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-04-25T01:24:55.944 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T01:24:55.944 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-04-25T01:24:55.945 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 301606, 2024-04-25T01:24:55.945 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 131, 2024-04-25T01:24:55.945 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 704, 2024-04-25T01:24:55.945 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 8, 2024-04-25T01:24:55.945 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-04-25T01:24:55.945 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-04-25T01:24:55.945 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-04-25T01:24:55.945 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-04-25T01:24:55.945 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-04-25T01:24:55.945 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-04-25T01:24:55.945 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 7344, 2024-04-25T01:24:55.945 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 155, 2024-04-25T01:24:55.945 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T01:24:55.946 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-04-25T01:24:55.946 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-04-25T01:24:55.946 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-04-25T01:24:55.946 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 136730, 2024-04-25T01:24:55.946 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 981, 2024-04-25T01:24:55.946 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-04-25T01:24:55.946 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-04-25T01:24:55.946 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-04-25T01:24:55.946 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-04-25T01:24:55.946 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.946 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter": { 2024-04-25T01:24:55.946 INFO:teuthology.orchestra.run.smithi169.stdout: "op_active": 0, 2024-04-25T01:24:55.946 INFO:teuthology.orchestra.run.smithi169.stdout: "op_laggy": 0, 2024-04-25T01:24:55.947 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send": 2911, 2024-04-25T01:24:55.947 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send_bytes": 14260024, 2024-04-25T01:24:55.947 INFO:teuthology.orchestra.run.smithi169.stdout: "op_resend": 31, 2024-04-25T01:24:55.947 INFO:teuthology.orchestra.run.smithi169.stdout: "op_reply": 2911, 2024-04-25T01:24:55.947 INFO:teuthology.orchestra.run.smithi169.stdout: "op_latency": { 2024-04-25T01:24:55.947 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2911, 2024-04-25T01:24:55.947 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 173.971329416, 2024-04-25T01:24:55.947 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.059763424 2024-04-25T01:24:55.947 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.947 INFO:teuthology.orchestra.run.smithi169.stdout: "op_inflight": 0, 2024-04-25T01:24:55.947 INFO:teuthology.orchestra.run.smithi169.stdout: "oplen_avg": { 2024-04-25T01:24:55.947 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2911, 2024-04-25T01:24:55.947 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 3196 2024-04-25T01:24:55.947 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.947 INFO:teuthology.orchestra.run.smithi169.stdout: "op": 2911, 2024-04-25T01:24:55.948 INFO:teuthology.orchestra.run.smithi169.stdout: "op_r": 8, 2024-04-25T01:24:55.948 INFO:teuthology.orchestra.run.smithi169.stdout: "op_w": 2903, 2024-04-25T01:24:55.948 INFO:teuthology.orchestra.run.smithi169.stdout: "op_rmw": 0, 2024-04-25T01:24:55.948 INFO:teuthology.orchestra.run.smithi169.stdout: "op_pg": 0, 2024-04-25T01:24:55.948 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_stat": 0, 2024-04-25T01:24:55.948 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_create": 10, 2024-04-25T01:24:55.948 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_read": 8, 2024-04-25T01:24:55.948 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_write": 1357, 2024-04-25T01:24:55.948 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writefull": 41, 2024-04-25T01:24:55.948 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writesame": 0, 2024-04-25T01:24:55.948 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_append": 0, 2024-04-25T01:24:55.948 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_zero": 0, 2024-04-25T01:24:55.948 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_truncate": 1, 2024-04-25T01:24:55.948 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_delete": 1284, 2024-04-25T01:24:55.949 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_mapext": 0, 2024-04-25T01:24:55.949 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_sparse_read": 0, 2024-04-25T01:24:55.949 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_clonerange": 0, 2024-04-25T01:24:55.949 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_getxattr": 0, 2024-04-25T01:24:55.949 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_setxattr": 20, 2024-04-25T01:24:55.949 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_cmpxattr": 0, 2024-04-25T01:24:55.949 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_rmxattr": 0, 2024-04-25T01:24:55.949 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_resetxattrs": 0, 2024-04-25T01:24:55.949 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_call": 0, 2024-04-25T01:24:55.949 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_watch": 0, 2024-04-25T01:24:55.949 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_notify": 0, 2024-04-25T01:24:55.949 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T01:24:55.949 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls": 0, 2024-04-25T01:24:55.949 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls_filter": 0, 2024-04-25T01:24:55.950 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_other": 0, 2024-04-25T01:24:55.950 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_active": 0, 2024-04-25T01:24:55.950 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_send": 0, 2024-04-25T01:24:55.950 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_resend": 0, 2024-04-25T01:24:55.950 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_ping": 0, 2024-04-25T01:24:55.950 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_active": 0, 2024-04-25T01:24:55.950 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_send": 0, 2024-04-25T01:24:55.950 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_resend": 0, 2024-04-25T01:24:55.950 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_active": 0, 2024-04-25T01:24:55.950 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_send": 0, 2024-04-25T01:24:55.950 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_resend": 0, 2024-04-25T01:24:55.950 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_active": 0, 2024-04-25T01:24:55.950 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_send": 0, 2024-04-25T01:24:55.950 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_resend": 0, 2024-04-25T01:24:55.951 INFO:teuthology.orchestra.run.smithi169.stdout: "command_active": 0, 2024-04-25T01:24:55.951 INFO:teuthology.orchestra.run.smithi169.stdout: "command_send": 0, 2024-04-25T01:24:55.951 INFO:teuthology.orchestra.run.smithi169.stdout: "command_resend": 0, 2024-04-25T01:24:55.951 INFO:teuthology.orchestra.run.smithi169.stdout: "map_epoch": 89, 2024-04-25T01:24:55.951 INFO:teuthology.orchestra.run.smithi169.stdout: "map_full": 0, 2024-04-25T01:24:55.951 INFO:teuthology.orchestra.run.smithi169.stdout: "map_inc": 2, 2024-04-25T01:24:55.951 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_sessions": 12, 2024-04-25T01:24:55.951 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_open": 12, 2024-04-25T01:24:55.951 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_close": 0, 2024-04-25T01:24:55.951 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_laggy": 0, 2024-04-25T01:24:55.951 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_wr": 339, 2024-04-25T01:24:55.951 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_rd": 0, 2024-04-25T01:24:55.951 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_del": 136 2024-04-25T01:24:55.951 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.952 INFO:teuthology.orchestra.run.smithi169.stdout: "oft": { 2024-04-25T01:24:55.952 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_objs": 1, 2024-04-25T01:24:55.952 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_kv_pairs": 0, 2024-04-25T01:24:55.952 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_updates": 0, 2024-04-25T01:24:55.952 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_removes": 0 2024-04-25T01:24:55.952 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.952 INFO:teuthology.orchestra.run.smithi169.stdout: "osd-slow-ops": { 2024-04-25T01:24:55.952 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_ops_count": 0 2024-04-25T01:24:55.952 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.952 INFO:teuthology.orchestra.run.smithi169.stdout: "purge_queue": { 2024-04-25T01:24:55.952 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops": 0, 2024-04-25T01:24:55.952 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops_high_water": 51, 2024-04-25T01:24:55.952 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing": 0, 2024-04-25T01:24:55.952 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_high_water": 10, 2024-04-25T01:24:55.952 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed_ops": 1269, 2024-04-25T01:24:55.953 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed": 329, 2024-04-25T01:24:55.953 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_item_in_journal": 0 2024-04-25T01:24:55.953 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.953 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T01:24:55.953 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T01:24:55.953 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T01:24:55.953 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T01:24:55.953 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 12978, 2024-04-25T01:24:55.953 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 4175666, 2024-04-25T01:24:55.953 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:55.953 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 12978, 2024-04-25T01:24:55.953 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T01:24:55.953 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T01:24:55.953 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 12978, 2024-04-25T01:24:55.954 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 4175666, 2024-04-25T01:24:55.954 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T01:24:55.954 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.954 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.954 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.954 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:55.954 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.954 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_bytes": { 2024-04-25T01:24:55.954 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T01:24:55.954 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T01:24:55.954 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T01:24:55.954 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T01:24:55.954 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T01:24:55.955 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:55.955 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T01:24:55.955 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 2911, 2024-04-25T01:24:55.955 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 14293253, 2024-04-25T01:24:55.955 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 2911, 2024-04-25T01:24:55.955 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 14293253, 2024-04-25T01:24:55.955 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T01:24:55.955 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.955 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.955 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.955 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:55.955 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.955 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_ops": { 2024-04-25T01:24:55.955 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T01:24:55.955 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 1024, 2024-04-25T01:24:55.956 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T01:24:55.956 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T01:24:55.956 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T01:24:55.956 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:55.956 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T01:24:55.956 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 2911, 2024-04-25T01:24:55.956 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 2911, 2024-04-25T01:24:55.956 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 2911, 2024-04-25T01:24:55.956 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 2911, 2024-04-25T01:24:55.956 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T01:24:55.956 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.956 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.956 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.956 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:55.957 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.957 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle": { 2024-04-25T01:24:55.957 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T01:24:55.957 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-25T01:24:55.957 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T01:24:55.957 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 329, 2024-04-25T01:24:55.957 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 33229, 2024-04-25T01:24:55.957 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:55.957 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 329, 2024-04-25T01:24:55.957 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T01:24:55.957 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T01:24:55.957 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 8, 2024-04-25T01:24:55.957 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 33229, 2024-04-25T01:24:55.957 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T01:24:55.958 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.958 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.958 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.958 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:55.958 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:55.958 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle-0x55cdf0c8c500": { 2024-04-25T01:24:55.958 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T01:24:55.958 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-25T01:24:55.958 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T01:24:55.958 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 6713, 2024-04-25T01:24:55.958 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 14012525, 2024-04-25T01:24:55.958 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:55.958 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 6713, 2024-04-25T01:24:55.959 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T01:24:55.959 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T01:24:55.959 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 1349, 2024-04-25T01:24:55.959 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 14012525, 2024-04-25T01:24:55.959 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T01:24:55.959 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:55.959 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:55.959 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:55.959 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:55.959 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:55.959 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T01:24:55.959 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:55 smithi169 ceph-mon[32759]: pgmap v660: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T01:24:56.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:55 smithi040 ceph-mon[32518]: pgmap v660: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T01:24:56.249 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:55 smithi031 ceph-mon[28053]: pgmap v660: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T01:24:56.699 INFO:tasks.check_counter:Daemon mds.i mds.exported=10 2024-04-25T01:24:56.699 INFO:tasks.check_counter:Daemon mds.i mds.imported=12 2024-04-25T01:24:56.699 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=1 2024-04-25T01:24:56.699 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T01:24:56.699 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 95849b94-029f-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T01:24:56.945 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:24:57.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:57 smithi169 ceph-mon[32759]: pgmap v661: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T01:24:58.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:57 smithi040 ceph-mon[32518]: pgmap v661: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T01:24:58.249 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:57 smithi031 ceph-mon[28053]: pgmap v661: 97 pgs: 97 active+clean; 24 MiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T01:24:58.483 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T01:24:58.484 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T01:24:58.484 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 0, 2024-04-25T01:24:58.484 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 2, 2024-04-25T01:24:58.484 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 0, 2024-04-25T01:24:58.484 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 0, 2024-04-25T01:24:58.484 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 4, 2024-04-25T01:24:58.484 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 0, 2024-04-25T01:24:58.484 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.000905363, 2024-04-25T01:24:58.484 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T01:24:58.484 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.000121629, 2024-04-25T01:24:58.484 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T01:24:58.485 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:58.485 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:58.485 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:58.485 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:58.485 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:58.485 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:58.485 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:58.485 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:58.485 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:58.485 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:58.485 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T01:24:58.485 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T01:24:58.485 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:58.485 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T01:24:58.486 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 204, 2024-04-25T01:24:58.486 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 199, 2024-04-25T01:24:58.486 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 476000, 2024-04-25T01:24:58.486 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 121312, 2024-04-25T01:24:58.486 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 2, 2024-04-25T01:24:58.486 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 1, 2024-04-25T01:24:58.486 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.030512402, 2024-04-25T01:24:58.486 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.011242245, 2024-04-25T01:24:58.486 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.014328730, 2024-04-25T01:24:58.486 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.005166722, 2024-04-25T01:24:58.486 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:58.486 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 199, 2024-04-25T01:24:58.486 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.008973863, 2024-04-25T01:24:58.486 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000045094 2024-04-25T01:24:58.487 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:58.487 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:58.487 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:58.487 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:58.487 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:58.487 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:58.487 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 476000, 2024-04-25T01:24:58.487 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 121312 2024-04-25T01:24:58.487 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:58.487 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T01:24:58.487 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 1, 2024-04-25T01:24:58.487 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 150, 2024-04-25T01:24:58.487 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 160, 2024-04-25T01:24:58.487 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 103952, 2024-04-25T01:24:58.488 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 3, 2024-04-25T01:24:58.488 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 2, 2024-04-25T01:24:58.488 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.007794036, 2024-04-25T01:24:58.488 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.005833855, 2024-04-25T01:24:58.488 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.000479727, 2024-04-25T01:24:58.488 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T01:24:58.488 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:24:58.488 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 148, 2024-04-25T01:24:58.488 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.008062368, 2024-04-25T01:24:58.488 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000054475 2024-04-25T01:24:58.488 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:58.488 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:24:58.488 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:58.488 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:58.489 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:58.489 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:58.489 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T01:24:58.489 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 103952 2024-04-25T01:24:58.489 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:58.489 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-04-25T01:24:58.489 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 0, 2024-04-25T01:24:58.489 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-04-25T01:24:58.489 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:58.489 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-04-25T01:24:58.489 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-04-25T01:24:58.489 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-04-25T01:24:58.489 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T01:24:58.489 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-04-25T01:24:58.490 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T01:24:58.490 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-04-25T01:24:58.490 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T01:24:58.490 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T01:24:58.490 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T01:24:58.490 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T01:24:58.490 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T01:24:58.490 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-04-25T01:24:58.490 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T01:24:58.490 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T01:24:58.490 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-04-25T01:24:58.490 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-04-25T01:24:58.490 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-04-25T01:24:58.490 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-04-25T01:24:58.490 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T01:24:58.491 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T01:24:58.491 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T01:24:58.491 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T01:24:58.491 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T01:24:58.491 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-04-25T01:24:58.491 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-04-25T01:24:58.491 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-04-25T01:24:58.491 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T01:24:58.491 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T01:24:58.491 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-04-25T01:24:58.491 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-04-25T01:24:58.491 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-04-25T01:24:58.491 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-04-25T01:24:58.492 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T01:24:58.492 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-04-25T01:24:58.492 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T01:24:58.492 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-04-25T01:24:58.492 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 148923, 2024-04-25T01:24:58.492 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 29, 2024-04-25T01:24:58.492 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 0, 2024-04-25T01:24:58.492 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 0, 2024-04-25T01:24:58.492 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-04-25T01:24:58.492 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-04-25T01:24:58.492 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-04-25T01:24:58.492 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-04-25T01:24:58.492 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-04-25T01:24:58.492 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-04-25T01:24:58.493 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 0, 2024-04-25T01:24:58.493 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 0, 2024-04-25T01:24:58.493 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T01:24:58.493 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-04-25T01:24:58.493 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-04-25T01:24:58.493 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-04-25T01:24:58.493 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 928, 2024-04-25T01:24:58.493 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 2, 2024-04-25T01:24:58.493 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-04-25T01:24:58.493 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-04-25T01:24:58.493 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-04-25T01:24:58.493 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-04-25T01:24:58.493 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T01:24:58.493 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T01:24:58.493 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T01:24:58.494 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T01:24:58.494 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T01:24:58.494 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 201, 2024-04-25T01:24:58.494 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 444754, 2024-04-25T01:24:58.494 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T01:24:58.494 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 201, 2024-04-25T01:24:58.494 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T01:24:58.494 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T01:24:58.494 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 201, 2024-04-25T01:24:58.494 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 444754, 2024-04-25T01:24:58.494 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T01:24:58.494 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T01:24:58.494 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T01:24:58.494 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T01:24:58.495 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:58.495 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T01:24:58.495 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T01:24:59.138 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-25T01:24:59.138 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-25T01:24:59.138 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-04-25T01:24:59.138 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T01:24:59.152 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T01:24:59.152 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T01:24:59.152 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:24:59.214 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T01:24:59.214 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:24:59.273 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:24:59.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.499+0000 7f821ca00640 1 -- 172.21.15.31:0/1256981556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82181015e0 msgr2=0x7f8218101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:59.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.499+0000 7f821ca00640 1 --2- 172.21.15.31:0/1256981556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82181015e0 0x7f8218101a00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8208009900 tx=0x7f820802a3f0 comp rx=0 tx=0).stop 2024-04-25T01:24:59.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.499+0000 7f821ca00640 1 -- 172.21.15.31:0/1256981556 shutdown_connections 2024-04-25T01:24:59.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.499+0000 7f821ca00640 1 --2- 172.21.15.31:0/1256981556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8218103190 0x7f8218109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:59.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.499+0000 7f821ca00640 1 --2- 172.21.15.31:0/1256981556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82181027d0 0x7f8218102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:59.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.499+0000 7f821ca00640 1 --2- 172.21.15.31:0/1256981556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82181015e0 0x7f8218101a00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:59.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.499+0000 7f821ca00640 1 -- 172.21.15.31:0/1256981556 >> 172.21.15.31:0/1256981556 conn(0x7f82180fcb50 msgr2=0x7f82180fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:24:59.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.499+0000 7f821ca00640 1 -- 172.21.15.31:0/1256981556 shutdown_connections 2024-04-25T01:24:59.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.499+0000 7f821ca00640 1 -- 172.21.15.31:0/1256981556 wait complete. 2024-04-25T01:24:59.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.500+0000 7f821ca00640 1 Processor -- start 2024-04-25T01:24:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.500+0000 7f821ca00640 1 -- start start 2024-04-25T01:24:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.501+0000 7f821ca00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82181015e0 0x7f8218199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.501+0000 7f821ca00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82181027d0 0x7f821819a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.501+0000 7f821ca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8218103190 0x7f82181a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.501+0000 7f821ca00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f821810d2b0 con 0x7f82181027d0 2024-04-25T01:24:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.501+0000 7f821ca00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f821810d130 con 0x7f82181015e0 2024-04-25T01:24:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.501+0000 7f821ca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f821810d430 con 0x7f8218103190 2024-04-25T01:24:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.501+0000 7f8216800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82181015e0 0x7f8218199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.501+0000 7f8217200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8218103190 0x7f82181a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.501+0000 7f8216800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82181015e0 0x7f8218199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:57460/0 (socket says 172.21.15.31:57460) 2024-04-25T01:24:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.501+0000 7f8216800640 1 -- 172.21.15.31:0/586831573 learned_addr learned my addr 172.21.15.31:0/586831573 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:24:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.501+0000 7f820fe00640 1 --2- 172.21.15.31:0/586831573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82181027d0 0x7f821819a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.501+0000 7f8217200640 1 -- 172.21.15.31:0/586831573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82181015e0 msgr2=0x7f8218199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.501+0000 7f8217200640 1 --2- 172.21.15.31:0/586831573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82181015e0 0x7f8218199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:59.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.501+0000 7f8217200640 1 -- 172.21.15.31:0/586831573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82181027d0 msgr2=0x7f821819a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:59.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.501+0000 7f8217200640 1 --2- 172.21.15.31:0/586831573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82181027d0 0x7f821819a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:59.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.501+0000 7f8217200640 1 -- 172.21.15.31:0/586831573 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82181a1730 con 0x7f8218103190 2024-04-25T01:24:59.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.501+0000 7f8216800640 1 --2- 172.21.15.31:0/586831573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82181015e0 0x7f8218199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:24:59.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.502+0000 7f8217200640 1 --2- 172.21.15.31:0/586831573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8218103190 0x7f82181a10c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8210009fd0 tx=0x7f821000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:24:59.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.502+0000 7f820f400640 1 -- 172.21.15.31:0/586831573 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821000ce20 con 0x7f8218103190 2024-04-25T01:24:59.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.502+0000 7f820f400640 1 -- 172.21.15.31:0/586831573 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8210005020 con 0x7f8218103190 2024-04-25T01:24:59.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.502+0000 7f820f400640 1 -- 172.21.15.31:0/586831573 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210010720 con 0x7f8218103190 2024-04-25T01:24:59.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.502+0000 7f821ca00640 1 -- 172.21.15.31:0/586831573 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82181a1a20 con 0x7f8218103190 2024-04-25T01:24:59.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.502+0000 7f821ca00640 1 -- 172.21.15.31:0/586831573 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82181a1f70 con 0x7f8218103190 2024-04-25T01:24:59.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.503+0000 7f821ca00640 1 -- 172.21.15.31:0/586831573 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82181a2200 con 0x7f8218103190 2024-04-25T01:24:59.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.505+0000 7f820f400640 1 -- 172.21.15.31:0/586831573 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8210005540 con 0x7f8218103190 2024-04-25T01:24:59.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.506+0000 7f820f400640 1 --2- 172.21.15.31:0/586831573 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f81fc0a2b90 0x7f81fc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:59.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.507+0000 7f820f400640 1 -- 172.21.15.31:0/586831573 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f82100cf650 con 0x7f8218103190 2024-04-25T01:24:59.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.507+0000 7f8216800640 1 --2- 172.21.15.31:0/586831573 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f81fc0a2b90 0x7f81fc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:59.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.508+0000 7f8216800640 1 --2- 172.21.15.31:0/586831573 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f81fc0a2b90 0x7f81fc0a5050 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8208000c00 tx=0x7f82080315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:24:59.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.510+0000 7f820f400640 1 -- 172.21.15.31:0/586831573 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f821009c5a0 con 0x7f8218103190 2024-04-25T01:24:59.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.718+0000 7f821ca00640 1 -- 172.21.15.31:0/586831573 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82181a23d0 con 0x7f8218103190 2024-04-25T01:24:59.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.720+0000 7f820f400640 1 -- 172.21.15.31:0/586831573 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22530 (secure 0 0 0) 0x7f821009bea0 con 0x7f8218103190 2024-04-25T01:24:59.701 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:24:59.702 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":89,"fsid":"95849b94-029f-11ef-bc93-c7b262605968","created":"2024-04-25T01:04:23.310856+0000","modified":"2024-04-25T01:14:33.793638+0000","last_up_change":"2024-04-25T01:12:29.393907+0000","last_in_change":"2024-04-25T01:12:08.978148+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:18.662391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:13:29.258771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:13:30.480996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa08e4c-fe6b-4116-bdf5-1adf5b465d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6803","nonce":788174091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6805","nonce":788174091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6809","nonce":788174091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6807","nonce":788174091}]},"public_addr":"172.21.15.31:6803/788174091","cluster_addr":"172.21.15.31:6805/788174091","heartbeat_back_addr":"172.21.15.31:6809/788174091","heartbeat_front_addr":"172.21.15.31:6807/788174091","state":["exists","up"]},{"osd":1,"uuid":"85339fe7-0dbc-4485-9334-2b039993c1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6801","nonce":3196394509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6803","nonce":3196394509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6807","nonce":3196394509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6805","nonce":3196394509}]},"public_addr":"172.21.15.40:6801/3196394509","cluster_addr":"172.21.15.40:6803/3196394509","heartbeat_back_addr":"172.21.15.40:6807/3196394509","heartbeat_front_addr":"172.21.15.40:6805/3196394509","state":["exists","up"]},{"osd":2,"uuid":"ab62ecb2-8aae-4987-9e2e-589e0f2fee5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6801","nonce":1955211197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6803","nonce":1955211197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6807","nonce":1955211197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6805","nonce":1955211197}]},"public_addr":"172.21.15.169:6801/1955211197","cluster_addr":"172.21.15.169:6803/1955211197","heartbeat_back_addr":"172.21.15.169:6807/1955211197","heartbeat_front_addr":"172.21.15.169:6805/1955211197","state":["exists","up"]},{"osd":3,"uuid":"8b5a7881-3401-495c-8ca4-c90685919d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6811","nonce":1228688643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6813","nonce":1228688643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6817","nonce":1228688643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6815","nonce":1228688643}]},"public_addr":"172.21.15.31:6811/1228688643","cluster_addr":"172.21.15.31:6813/1228688643","heartbeat_back_addr":"172.21.15.31:6817/1228688643","heartbeat_front_addr":"172.21.15.31:6815/1228688643","state":["exists","up"]},{"osd":4,"uuid":"56deb61d-a334-4cf2-8ac7-fbb6e6947092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6809","nonce":1913779091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6811","nonce":1913779091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6815","nonce":1913779091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6813","nonce":1913779091}]},"public_addr":"172.21.15.40:6809/1913779091","cluster_addr":"172.21.15.40:6811/1913779091","heartbeat_back_addr":"172.21.15.40:6815/1913779091","heartbeat_front_addr":"172.21.15.40:6813/1913779091","state":["exists","up"]},{"osd":5,"uuid":"8da55693-f9c7-49a7-99e0-165078f8001b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6809","nonce":2549650781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6811","nonce":2549650781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6815","nonce":2549650781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6813","nonce":2549650781}]},"public_addr":"172.21.15.169:6809/2549650781","cluster_addr":"172.21.15.169:6811/2549650781","heartbeat_back_addr":"172.21.15.169:6815/2549650781","heartbeat_front_addr":"172.21.15.169:6813/2549650781","state":["exists","up"]},{"osd":6,"uuid":"fc8350d2-1ad6-43cb-89f4-d336afc99ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6819","nonce":611507624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6821","nonce":611507624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6825","nonce":611507624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6823","nonce":611507624}]},"public_addr":"172.21.15.31:6819/611507624","cluster_addr":"172.21.15.31:6821/611507624","heartbeat_back_addr":"172.21.15.31:6825/611507624","heartbeat_front_addr":"172.21.15.31:6823/611507624","state":["exists","up"]},{"osd":7,"uuid":"3121b58b-8c94-40b5-9d48-f56552d7c39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6817","nonce":3776085849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6819","nonce":3776085849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6823","nonce":3776085849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6821","nonce":3776085849}]},"public_addr":"172.21.15.40:6817/3776085849","cluster_addr":"172.21.15.40:6819/3776085849","heartbeat_back_addr":"172.21.15.40:6823/3776085849","heartbeat_front_addr":"172.21.15.40:6821/3776085849","state":["exists","up"]},{"osd":8,"uuid":"f60407df-91f5-477a-9702-ddaef24faf82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6817","nonce":2736546122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6819","nonce":2736546122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6823","nonce":2736546122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6821","nonce":2736546122}]},"public_addr":"172.21.15.169:6817/2736546122","cluster_addr":"172.21.15.169:6819/2736546122","heartbeat_back_addr":"172.21.15.169:6823/2736546122","heartbeat_front_addr":"172.21.15.169:6821/2736546122","state":["exists","up"]},{"osd":9,"uuid":"7506a407-1960-49bc-9e38-7f2a8bf996ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6827","nonce":1187207880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6829","nonce":1187207880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6833","nonce":1187207880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6831","nonce":1187207880}]},"public_addr":"172.21.15.31:6827/1187207880","cluster_addr":"172.21.15.31:6829/1187207880","heartbeat_back_addr":"172.21.15.31:6833/1187207880","heartbeat_front_addr":"172.21.15.31:6831/1187207880","state":["exists","up"]},{"osd":10,"uuid":"831c48e2-b0d0-451f-959b-228e927542cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6825","nonce":3050148001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6827","nonce":3050148001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6831","nonce":3050148001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6829","nonce":3050148001}]},"public_addr":"172.21.15.40:6825/3050148001","cluster_addr":"172.21.15.40:6827/3050148001","heartbeat_back_addr":"172.21.15.40:6831/3050148001","heartbeat_front_addr":"172.21.15.40:6829/3050148001","state":["exists","up"]},{"osd":11,"uuid":"144fe0aa-f757-4b3c-90ad-99775e17d57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6825","nonce":262401944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6827","nonce":262401944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6831","nonce":262401944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6829","nonce":262401944}]},"public_addr":"172.21.15.169:6825/262401944","cluster_addr":"172.21.15.169:6827/262401944","heartbeat_back_addr":"172.21.15.169:6831/262401944","heartbeat_front_addr":"172.21.15.169:6829/262401944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:25.640995+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:51.036909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.635633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:48.467912+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:14.454971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:37.609315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:12.441807+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:38.239215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:02.230758+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:35.762553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:02.051405+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:26.569593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6801/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4004014066":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/1930101646":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/2638309002":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2238638505":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2902841144":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4190696323":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/480558521":"2024-04-26T01:04:57.149272+0000","172.21.15.31:6801/2638309002":"2024-04-26T01:04:57.149272+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:24:59.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.723+0000 7f821ca00640 1 -- 172.21.15.31:0/586831573 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f81fc0a2b90 msgr2=0x7f81fc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:59.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.723+0000 7f821ca00640 1 --2- 172.21.15.31:0/586831573 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f81fc0a2b90 0x7f81fc0a5050 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8208000c00 tx=0x7f82080315b0 comp rx=0 tx=0).stop 2024-04-25T01:24:59.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.723+0000 7f821ca00640 1 -- 172.21.15.31:0/586831573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8218103190 msgr2=0x7f82181a10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:59.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.723+0000 7f821ca00640 1 --2- 172.21.15.31:0/586831573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8218103190 0x7f82181a10c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8210009fd0 tx=0x7f821000ef10 comp rx=0 tx=0).stop 2024-04-25T01:24:59.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.723+0000 7f821ca00640 1 -- 172.21.15.31:0/586831573 shutdown_connections 2024-04-25T01:24:59.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.723+0000 7f821ca00640 1 --2- 172.21.15.31:0/586831573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8218103190 0x7f82181a10c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:59.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.723+0000 7f821ca00640 1 --2- 172.21.15.31:0/586831573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82181027d0 0x7f821819a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:59.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.723+0000 7f821ca00640 1 --2- 172.21.15.31:0/586831573 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f81fc0a2b90 0x7f81fc0a5050 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:59.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.723+0000 7f821ca00640 1 --2- 172.21.15.31:0/586831573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82181015e0 0x7f8218199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:59.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.723+0000 7f821ca00640 1 -- 172.21.15.31:0/586831573 >> 172.21.15.31:0/586831573 conn(0x7f82180fcb50 msgr2=0x7f8218073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:24:59.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.723+0000 7f821ca00640 1 -- 172.21.15.31:0/586831573 shutdown_connections 2024-04-25T01:24:59.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.723+0000 7f821ca00640 1 -- 172.21.15.31:0/586831573 wait complete. 2024-04-25T01:24:59.724 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:18.662391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:13:29.258771+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:13:30.480996+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:24:59.724 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:24:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.953+0000 7f8a0a800640 1 -- 172.21.15.31:0/676350959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a04105430 msgr2=0x7f8a0410bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.953+0000 7f8a0a800640 1 --2- 172.21.15.31:0/676350959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a04105430 0x7f8a0410bcc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009ca0 tx=0x7f89fc02a8b0 comp rx=0 tx=0).stop 2024-04-25T01:24:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.953+0000 7f8a0a800640 1 -- 172.21.15.31:0/676350959 shutdown_connections 2024-04-25T01:24:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.953+0000 7f8a0a800640 1 --2- 172.21.15.31:0/676350959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a04105430 0x7f8a0410bcc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.953+0000 7f8a0a800640 1 --2- 172.21.15.31:0/676350959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a04104a70 0x7f8a04104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.953+0000 7f8a0a800640 1 --2- 172.21.15.31:0/676350959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a04103880 0x7f8a04103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.953+0000 7f8a0a800640 1 -- 172.21.15.31:0/676350959 >> 172.21.15.31:0/676350959 conn(0x7f8a040fedf0 msgr2=0x7f8a04101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:24:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.953+0000 7f8a0a800640 1 -- 172.21.15.31:0/676350959 shutdown_connections 2024-04-25T01:24:59.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.953+0000 7f8a0a800640 1 -- 172.21.15.31:0/676350959 wait complete. 2024-04-25T01:24:59.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.954+0000 7f8a0a800640 1 Processor -- start 2024-04-25T01:24:59.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.955+0000 7f8a0a800640 1 -- start start 2024-04-25T01:24:59.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.955+0000 7f8a0a800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a04103880 0x7f8a04199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:59.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.955+0000 7f8a0a800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a04104a70 0x7f8a0419a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:59.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.955+0000 7f8a0a800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a04105430 0x7f8a041a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:59.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.955+0000 7f8a0a800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0410f550 con 0x7f8a04104a70 2024-04-25T01:24:59.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.955+0000 7f8a0a800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0410f3d0 con 0x7f8a04105430 2024-04-25T01:24:59.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.955+0000 7f8a0a800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0410f6d0 con 0x7f8a04103880 2024-04-25T01:24:59.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.955+0000 7f8a09400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a04103880 0x7f8a04199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:59.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.955+0000 7f8a09e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a04105430 0x7f8a041a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:59.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.955+0000 7f8a09400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a04103880 0x7f8a04199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:58900/0 (socket says 172.21.15.31:58900) 2024-04-25T01:24:59.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.955+0000 7f8a09400640 1 -- 172.21.15.31:0/3155180730 learned_addr learned my addr 172.21.15.31:0/3155180730 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:24:59.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.956+0000 7f8a08a00640 1 --2- 172.21.15.31:0/3155180730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a04104a70 0x7f8a0419a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:59.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.956+0000 7f8a09400640 1 -- 172.21.15.31:0/3155180730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a04105430 msgr2=0x7f8a041a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:59.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.956+0000 7f8a09400640 1 --2- 172.21.15.31:0/3155180730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a04105430 0x7f8a041a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:59.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.956+0000 7f8a09400640 1 -- 172.21.15.31:0/3155180730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a04104a70 msgr2=0x7f8a0419a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:24:59.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.956+0000 7f8a09400640 1 --2- 172.21.15.31:0/3155180730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a04104a70 0x7f8a0419a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:24:59.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.956+0000 7f8a09400640 1 -- 172.21.15.31:0/3155180730 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a041a1720 con 0x7f8a04103880 2024-04-25T01:24:59.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.956+0000 7f8a08a00640 1 --2- 172.21.15.31:0/3155180730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a04104a70 0x7f8a0419a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:24:59.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.956+0000 7f8a09400640 1 --2- 172.21.15.31:0/3155180730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a04103880 0x7f8a04199b00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f89f800e9c0 tx=0x7f89f800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:24:59.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.956+0000 7f8a02000640 1 -- 172.21.15.31:0/3155180730 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f800cd80 con 0x7f8a04103880 2024-04-25T01:24:59.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.957+0000 7f8a02000640 1 -- 172.21.15.31:0/3155180730 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f80035f0 con 0x7f8a04103880 2024-04-25T01:24:59.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.957+0000 7f8a0a800640 1 -- 172.21.15.31:0/3155180730 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a041a1a10 con 0x7f8a04103880 2024-04-25T01:24:59.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.957+0000 7f8a02000640 1 -- 172.21.15.31:0/3155180730 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8010690 con 0x7f8a04103880 2024-04-25T01:24:59.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.957+0000 7f8a0a800640 1 -- 172.21.15.31:0/3155180730 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a041a1f10 con 0x7f8a04103880 2024-04-25T01:24:59.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.958+0000 7f8a0a800640 1 -- 172.21.15.31:0/3155180730 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a0410ab40 con 0x7f8a04103880 2024-04-25T01:24:59.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.960+0000 7f8a02000640 1 -- 172.21.15.31:0/3155180730 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f89f80037a0 con 0x7f8a04103880 2024-04-25T01:24:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.961+0000 7f8a02000640 1 --2- 172.21.15.31:0/3155180730 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f89ec0a2b90 0x7f89ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:24:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.961+0000 7f8a02000640 1 -- 172.21.15.31:0/3155180730 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f89f80cf450 con 0x7f8a04103880 2024-04-25T01:24:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.961+0000 7f8a08a00640 1 --2- 172.21.15.31:0/3155180730 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f89ec0a2b90 0x7f89ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:24:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.962+0000 7f8a08a00640 1 --2- 172.21.15.31:0/3155180730 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f89ec0a2b90 0x7f89ec0a5050 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8a0419b010 tx=0x7f89f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:24:59.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:24:59.965+0000 7f8a02000640 1 -- 172.21.15.31:0/3155180730 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f89f809c3a0 con 0x7f8a04103880 2024-04-25T01:24:59.999 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:59 smithi031 ceph-mon[28053]: pgmap v662: 97 pgs: 97 active+clean; 24 MiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T01:25:00.000 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:24:59 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/586831573' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:25:00.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.198+0000 7f8a0a800640 1 -- 172.21.15.31:0/3155180730 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8a0402d170 con 0x7f8a04103880 2024-04-25T01:25:00.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.199+0000 7f8a02000640 1 -- 172.21.15.31:0/3155180730 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f89f809bca0 con 0x7f8a04103880 2024-04-25T01:25:00.180 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T01:25:00.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.202+0000 7f8a0a800640 1 -- 172.21.15.31:0/3155180730 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f89ec0a2b90 msgr2=0x7f89ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:00.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.202+0000 7f8a0a800640 1 --2- 172.21.15.31:0/3155180730 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f89ec0a2b90 0x7f89ec0a5050 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8a0419b010 tx=0x7f89f0005f90 comp rx=0 tx=0).stop 2024-04-25T01:25:00.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.202+0000 7f8a0a800640 1 -- 172.21.15.31:0/3155180730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a04103880 msgr2=0x7f8a04199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:00.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.202+0000 7f8a0a800640 1 --2- 172.21.15.31:0/3155180730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a04103880 0x7f8a04199b00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f89f800e9c0 tx=0x7f89f800ee80 comp rx=0 tx=0).stop 2024-04-25T01:25:00.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.202+0000 7f8a0a800640 1 -- 172.21.15.31:0/3155180730 shutdown_connections 2024-04-25T01:25:00.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.202+0000 7f8a0a800640 1 --2- 172.21.15.31:0/3155180730 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f89ec0a2b90 0x7f89ec0a5050 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:00.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.202+0000 7f8a0a800640 1 --2- 172.21.15.31:0/3155180730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a04105430 0x7f8a041a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:00.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.202+0000 7f8a0a800640 1 --2- 172.21.15.31:0/3155180730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a04104a70 0x7f8a0419a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:00.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.202+0000 7f8a0a800640 1 --2- 172.21.15.31:0/3155180730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a04103880 0x7f8a04199b00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:00.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.202+0000 7f8a0a800640 1 -- 172.21.15.31:0/3155180730 >> 172.21.15.31:0/3155180730 conn(0x7f8a040fedf0 msgr2=0x7f8a04107d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:00.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.203+0000 7f8a0a800640 1 -- 172.21.15.31:0/3155180730 shutdown_connections 2024-04-25T01:25:00.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.203+0000 7f8a0a800640 1 -- 172.21.15.31:0/3155180730 wait complete. 2024-04-25T01:25:00.188 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:59 smithi169 ceph-mon[32759]: pgmap v662: 97 pgs: 97 active+clean; 24 MiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T01:25:00.188 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:24:59 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/586831573' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:25:00.191 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:59 smithi040 ceph-mon[32518]: pgmap v662: 97 pgs: 97 active+clean; 24 MiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T01:25:00.191 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:24:59 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/586831573' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:25:00.203 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:25:00.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.427+0000 7eff59e00640 1 -- 172.21.15.31:0/3469257639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff54074120 msgr2=0x7eff54074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:00.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.427+0000 7eff59e00640 1 --2- 172.21.15.31:0/3469257639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff54074120 0x7eff54074580 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7eff40009900 tx=0x7eff4002a480 comp rx=0 tx=0).stop 2024-04-25T01:25:00.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.427+0000 7eff59e00640 1 -- 172.21.15.31:0/3469257639 shutdown_connections 2024-04-25T01:25:00.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.427+0000 7eff59e00640 1 --2- 172.21.15.31:0/3469257639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff54109640 0x7eff5410bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:00.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.427+0000 7eff59e00640 1 --2- 172.21.15.31:0/3469257639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff54074120 0x7eff54074580 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:00.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.427+0000 7eff59e00640 1 --2- 172.21.15.31:0/3469257639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff54075700 0x7eff54073b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:00.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.427+0000 7eff59e00640 1 -- 172.21.15.31:0/3469257639 >> 172.21.15.31:0/3469257639 conn(0x7eff540fcd20 msgr2=0x7eff540ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:00.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.427+0000 7eff59e00640 1 -- 172.21.15.31:0/3469257639 shutdown_connections 2024-04-25T01:25:00.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.427+0000 7eff59e00640 1 -- 172.21.15.31:0/3469257639 wait complete. 2024-04-25T01:25:00.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.428+0000 7eff59e00640 1 Processor -- start 2024-04-25T01:25:00.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.428+0000 7eff59e00640 1 -- start start 2024-04-25T01:25:00.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.429+0000 7eff59e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff54074120 0x7eff54199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:00.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.429+0000 7eff59e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff54075700 0x7eff5419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:00.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.429+0000 7eff59e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff54109640 0x7eff541a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:00.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.429+0000 7eff59e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7eff5410f660 con 0x7eff54074120 2024-04-25T01:25:00.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.429+0000 7eff59e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7eff5410f4e0 con 0x7eff54075700 2024-04-25T01:25:00.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.429+0000 7eff59e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7eff5410f7e0 con 0x7eff54109640 2024-04-25T01:25:00.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.429+0000 7eff52a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff54074120 0x7eff54199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:00.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.429+0000 7eff52000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff54075700 0x7eff5419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:00.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.429+0000 7eff52a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff54074120 0x7eff54199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60476/0 (socket says 172.21.15.31:60476) 2024-04-25T01:25:00.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.429+0000 7eff52a00640 1 -- 172.21.15.31:0/3779776182 learned_addr learned my addr 172.21.15.31:0/3779776182 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:25:00.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.429+0000 7eff53400640 1 --2- 172.21.15.31:0/3779776182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff54109640 0x7eff541a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:00.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.429+0000 7eff52a00640 1 -- 172.21.15.31:0/3779776182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff54109640 msgr2=0x7eff541a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:00.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.429+0000 7eff52a00640 1 --2- 172.21.15.31:0/3779776182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff54109640 0x7eff541a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:00.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.429+0000 7eff52a00640 1 -- 172.21.15.31:0/3779776182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff54075700 msgr2=0x7eff5419a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:00.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.429+0000 7eff52a00640 1 --2- 172.21.15.31:0/3779776182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff54075700 0x7eff5419a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:00.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.429+0000 7eff52a00640 1 -- 172.21.15.31:0/3779776182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff541a18f0 con 0x7eff54074120 2024-04-25T01:25:00.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.430+0000 7eff52000640 1 --2- 172.21.15.31:0/3779776182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff54075700 0x7eff5419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:25:00.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.430+0000 7eff52a00640 1 --2- 172.21.15.31:0/3779776182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff54074120 0x7eff54199cb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7eff4400e9f0 tx=0x7eff4400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:00.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.430+0000 7eff58a00640 1 -- 172.21.15.31:0/3779776182 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4400cd80 con 0x7eff54074120 2024-04-25T01:25:00.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.430+0000 7eff58a00640 1 -- 172.21.15.31:0/3779776182 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff44005500 con 0x7eff54074120 2024-04-25T01:25:00.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.430+0000 7eff59e00640 1 -- 172.21.15.31:0/3779776182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff541a1be0 con 0x7eff54074120 2024-04-25T01:25:00.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.430+0000 7eff58a00640 1 -- 172.21.15.31:0/3779776182 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff44010690 con 0x7eff54074120 2024-04-25T01:25:00.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.430+0000 7eff59e00640 1 -- 172.21.15.31:0/3779776182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff541a20b0 con 0x7eff54074120 2024-04-25T01:25:00.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.431+0000 7eff59e00640 1 -- 172.21.15.31:0/3779776182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff54108400 con 0x7eff54074120 2024-04-25T01:25:00.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.434+0000 7eff58a00640 1 -- 172.21.15.31:0/3779776182 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7eff440046c0 con 0x7eff54074120 2024-04-25T01:25:00.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.435+0000 7eff58a00640 1 --2- 172.21.15.31:0/3779776182 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7eff380a2b90 0x7eff380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:00.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.435+0000 7eff58a00640 1 -- 172.21.15.31:0/3779776182 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7eff440cf470 con 0x7eff54074120 2024-04-25T01:25:00.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.435+0000 7eff52000640 1 --2- 172.21.15.31:0/3779776182 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7eff380a2b90 0x7eff380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:00.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.436+0000 7eff52000640 1 --2- 172.21.15.31:0/3779776182 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7eff380a2b90 0x7eff380a5050 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7eff40031440 tx=0x7eff400315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:00.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.441+0000 7eff58a00640 1 -- 172.21.15.31:0/3779776182 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7eff4409c3c0 con 0x7eff54074120 2024-04-25T01:25:00.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.658+0000 7eff59e00640 1 -- 172.21.15.31:0/3779776182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7eff540623e0 con 0x7eff54074120 2024-04-25T01:25:00.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.659+0000 7eff58a00640 1 -- 172.21.15.31:0/3779776182 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7eff4409bcc0 con 0x7eff54074120 2024-04-25T01:25:00.640 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-25T01:25:00.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.662+0000 7eff59e00640 1 -- 172.21.15.31:0/3779776182 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7eff380a2b90 msgr2=0x7eff380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:00.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.662+0000 7eff59e00640 1 --2- 172.21.15.31:0/3779776182 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7eff380a2b90 0x7eff380a5050 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7eff40031440 tx=0x7eff400315b0 comp rx=0 tx=0).stop 2024-04-25T01:25:00.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.662+0000 7eff59e00640 1 -- 172.21.15.31:0/3779776182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff54074120 msgr2=0x7eff54199cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:00.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.662+0000 7eff59e00640 1 --2- 172.21.15.31:0/3779776182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff54074120 0x7eff54199cb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7eff4400e9f0 tx=0x7eff4400edf0 comp rx=0 tx=0).stop 2024-04-25T01:25:00.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.662+0000 7eff59e00640 1 -- 172.21.15.31:0/3779776182 shutdown_connections 2024-04-25T01:25:00.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.662+0000 7eff59e00640 1 --2- 172.21.15.31:0/3779776182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff54109640 0x7eff541a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:00.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.662+0000 7eff59e00640 1 --2- 172.21.15.31:0/3779776182 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7eff380a2b90 0x7eff380a5050 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:00.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.662+0000 7eff59e00640 1 --2- 172.21.15.31:0/3779776182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff54075700 0x7eff5419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:00.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.662+0000 7eff59e00640 1 --2- 172.21.15.31:0/3779776182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff54074120 0x7eff54199cb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:00.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.662+0000 7eff59e00640 1 -- 172.21.15.31:0/3779776182 >> 172.21.15.31:0/3779776182 conn(0x7eff540fcd20 msgr2=0x7eff541055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:00.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.662+0000 7eff59e00640 1 -- 172.21.15.31:0/3779776182 shutdown_connections 2024-04-25T01:25:00.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.663+0000 7eff59e00640 1 -- 172.21.15.31:0/3779776182 wait complete. 2024-04-25T01:25:00.663 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:25:00.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.881+0000 7fd6d2000640 1 -- 172.21.15.31:0/1671785016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6cc100550 msgr2=0x7fd6cc100970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:00.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.881+0000 7fd6d2000640 1 --2- 172.21.15.31:0/1671785016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6cc100550 0x7fd6cc100970 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc009900 tx=0x7fd6bc02a3f0 comp rx=0 tx=0).stop 2024-04-25T01:25:00.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.881+0000 7fd6d2000640 1 -- 172.21.15.31:0/1671785016 shutdown_connections 2024-04-25T01:25:00.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.881+0000 7fd6d2000640 1 --2- 172.21.15.31:0/1671785016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6cc102100 0x7fd6cc108990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:00.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.881+0000 7fd6d2000640 1 --2- 172.21.15.31:0/1671785016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6cc101740 0x7fd6cc101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:00.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.881+0000 7fd6d2000640 1 --2- 172.21.15.31:0/1671785016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6cc100550 0x7fd6cc100970 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:00.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.881+0000 7fd6d2000640 1 -- 172.21.15.31:0/1671785016 >> 172.21.15.31:0/1671785016 conn(0x7fd6cc0fbae0 msgr2=0x7fd6cc0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:00.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.881+0000 7fd6d2000640 1 -- 172.21.15.31:0/1671785016 shutdown_connections 2024-04-25T01:25:00.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.881+0000 7fd6d2000640 1 -- 172.21.15.31:0/1671785016 wait complete. 2024-04-25T01:25:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.882+0000 7fd6d2000640 1 Processor -- start 2024-04-25T01:25:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.882+0000 7fd6d2000640 1 -- start start 2024-04-25T01:25:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.882+0000 7fd6d2000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6cc100550 0x7fd6cc1044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.882+0000 7fd6d2000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6cc101740 0x7fd6cc102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.883+0000 7fd6d2000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6cc102100 0x7fd6cc103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.883+0000 7fd6d2000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc10c230 con 0x7fd6cc101740 2024-04-25T01:25:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.883+0000 7fd6d2000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc10c0b0 con 0x7fd6cc100550 2024-04-25T01:25:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.883+0000 7fd6d2000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc10c3b0 con 0x7fd6cc102100 2024-04-25T01:25:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.883+0000 7fd6d0c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6cc100550 0x7fd6cc1044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.883+0000 7fd6c3e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6cc101740 0x7fd6cc102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.883+0000 7fd6d0c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6cc100550 0x7fd6cc1044e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:57506/0 (socket says 172.21.15.31:57506) 2024-04-25T01:25:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.883+0000 7fd6d0c00640 1 -- 172.21.15.31:0/3126402569 learned_addr learned my addr 172.21.15.31:0/3126402569 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:25:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.883+0000 7fd6d1600640 1 --2- 172.21.15.31:0/3126402569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6cc102100 0x7fd6cc103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.883+0000 7fd6d0c00640 1 -- 172.21.15.31:0/3126402569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6cc102100 msgr2=0x7fd6cc103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.883+0000 7fd6d0c00640 1 --2- 172.21.15.31:0/3126402569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6cc102100 0x7fd6cc103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.883+0000 7fd6d0c00640 1 -- 172.21.15.31:0/3126402569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6cc101740 msgr2=0x7fd6cc102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.883+0000 7fd6d0c00640 1 --2- 172.21.15.31:0/3126402569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6cc101740 0x7fd6cc102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.883+0000 7fd6d0c00640 1 -- 172.21.15.31:0/3126402569 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6cc1035a0 con 0x7fd6cc100550 2024-04-25T01:25:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.884+0000 7fd6d0c00640 1 --2- 172.21.15.31:0/3126402569 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6cc100550 0x7fd6cc1044e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc031440 tx=0x7fd6bc032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.884+0000 7fd6ca000640 1 -- 172.21.15.31:0/3126402569 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc03f030 con 0x7fd6cc100550 2024-04-25T01:25:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.884+0000 7fd6d2000640 1 -- 172.21.15.31:0/3126402569 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6cc103830 con 0x7fd6cc100550 2024-04-25T01:25:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.884+0000 7fd6ca000640 1 -- 172.21.15.31:0/3126402569 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6bc003ce0 con 0x7fd6cc100550 2024-04-25T01:25:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.884+0000 7fd6ca000640 1 -- 172.21.15.31:0/3126402569 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc035070 con 0x7fd6cc100550 2024-04-25T01:25:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.884+0000 7fd6d2000640 1 -- 172.21.15.31:0/3126402569 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6cc1a63b0 con 0x7fd6cc100550 2024-04-25T01:25:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.885+0000 7fd6d2000640 1 -- 172.21.15.31:0/3126402569 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6cc1a66a0 con 0x7fd6cc100550 2024-04-25T01:25:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.887+0000 7fd6ca000640 1 -- 172.21.15.31:0/3126402569 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd6bc042030 con 0x7fd6cc100550 2024-04-25T01:25:00.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.889+0000 7fd6ca000640 1 --2- 172.21.15.31:0/3126402569 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd6b00a2b90 0x7fd6b00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:00.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.889+0000 7fd6ca000640 1 -- 172.21.15.31:0/3126402569 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fd6bc0ef4a0 con 0x7fd6cc100550 2024-04-25T01:25:00.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.889+0000 7fd6c3e00640 1 --2- 172.21.15.31:0/3126402569 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd6b00a2b90 0x7fd6b00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:00.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.890+0000 7fd6c3e00640 1 --2- 172.21.15.31:0/3126402569 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd6b00a2b90 0x7fd6b00a5050 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc104250 tx=0x7fd6b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:00.893+0000 7fd6ca000640 1 -- 172.21.15.31:0/3126402569 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd6bc0bbda0 con 0x7fd6cc100550 2024-04-25T01:25:00.999 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:00 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3155180730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:25:00.999 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:00 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3779776182' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:25:01.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.099+0000 7fd6d2000640 1 -- 172.21.15.31:0/3126402569 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd6cc1a6870 con 0x7fd6cc100550 2024-04-25T01:25:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.099+0000 7fd6ca000640 1 -- 172.21.15.31:0/3126402569 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fd6bc0c00a0 con 0x7fd6cc100550 2024-04-25T01:25:01.080 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-04-25T01:25:01.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.102+0000 7fd6d2000640 1 -- 172.21.15.31:0/3126402569 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd6b00a2b90 msgr2=0x7fd6b00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:01.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.102+0000 7fd6d2000640 1 --2- 172.21.15.31:0/3126402569 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd6b00a2b90 0x7fd6b00a5050 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc104250 tx=0x7fd6b4005f90 comp rx=0 tx=0).stop 2024-04-25T01:25:01.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.102+0000 7fd6d2000640 1 -- 172.21.15.31:0/3126402569 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6cc100550 msgr2=0x7fd6cc1044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:01.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.102+0000 7fd6d2000640 1 --2- 172.21.15.31:0/3126402569 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6cc100550 0x7fd6cc1044e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc031440 tx=0x7fd6bc032ce0 comp rx=0 tx=0).stop 2024-04-25T01:25:01.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.102+0000 7fd6d2000640 1 -- 172.21.15.31:0/3126402569 shutdown_connections 2024-04-25T01:25:01.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.102+0000 7fd6d2000640 1 --2- 172.21.15.31:0/3126402569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6cc102100 0x7fd6cc103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:01.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.102+0000 7fd6d2000640 1 --2- 172.21.15.31:0/3126402569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6cc101740 0x7fd6cc102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:01.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.102+0000 7fd6d2000640 1 --2- 172.21.15.31:0/3126402569 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd6b00a2b90 0x7fd6b00a5050 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:01.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.102+0000 7fd6d2000640 1 --2- 172.21.15.31:0/3126402569 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6cc100550 0x7fd6cc1044e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:01.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.102+0000 7fd6d2000640 1 -- 172.21.15.31:0/3126402569 >> 172.21.15.31:0/3126402569 conn(0x7fd6cc0fbae0 msgr2=0x7fd6cc0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:01.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.102+0000 7fd6d2000640 1 -- 172.21.15.31:0/3126402569 shutdown_connections 2024-04-25T01:25:01.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.102+0000 7fd6d2000640 1 -- 172.21.15.31:0/3126402569 wait complete. 2024-04-25T01:25:01.102 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T01:25:01.188 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:00 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3155180730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:25:01.188 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:00 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3779776182' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:25:01.191 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:00 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3155180730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:25:01.191 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:00 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3779776182' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:25:01.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.325+0000 7f88c6c00640 1 -- 172.21.15.31:0/1495156056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88c0102860 msgr2=0x7f88c0102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:01.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.325+0000 7f88c6c00640 1 --2- 172.21.15.31:0/1495156056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88c0102860 0x7f88c0102ce0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f88ac009900 tx=0x7f88ac02a410 comp rx=0 tx=0).stop 2024-04-25T01:25:01.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.325+0000 7f88c6c00640 1 -- 172.21.15.31:0/1495156056 shutdown_connections 2024-04-25T01:25:01.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.325+0000 7f88c6c00640 1 --2- 172.21.15.31:0/1495156056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88c0103220 0x7f88c0109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:01.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.325+0000 7f88c6c00640 1 --2- 172.21.15.31:0/1495156056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88c0102860 0x7f88c0102ce0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:01.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.325+0000 7f88c6c00640 1 --2- 172.21.15.31:0/1495156056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c0101670 0x7f88c0101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:01.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.325+0000 7f88c6c00640 1 -- 172.21.15.31:0/1495156056 >> 172.21.15.31:0/1495156056 conn(0x7f88c00fcbe0 msgr2=0x7f88c00ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:01.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.325+0000 7f88c6c00640 1 -- 172.21.15.31:0/1495156056 shutdown_connections 2024-04-25T01:25:01.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.325+0000 7f88c6c00640 1 -- 172.21.15.31:0/1495156056 wait complete. 2024-04-25T01:25:01.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.326+0000 7f88c6c00640 1 Processor -- start 2024-04-25T01:25:01.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.327+0000 7f88c6c00640 1 -- start start 2024-04-25T01:25:01.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.327+0000 7f88c6c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88c0101670 0x7f88c0199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:01.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.327+0000 7f88c6c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88c0102860 0x7f88c019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:01.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.327+0000 7f88c6c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c0103220 0x7f88c01a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:01.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.327+0000 7f88c6c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f88c010d320 con 0x7f88c0103220 2024-04-25T01:25:01.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.327+0000 7f88c6c00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f88c010d1a0 con 0x7f88c0102860 2024-04-25T01:25:01.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.327+0000 7f88c6c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f88c010d4a0 con 0x7f88c0101670 2024-04-25T01:25:01.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.327+0000 7f88c5800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88c0101670 0x7f88c0199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:01.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.327+0000 7f88c5800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88c0101670 0x7f88c0199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:58970/0 (socket says 172.21.15.31:58970) 2024-04-25T01:25:01.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.327+0000 7f88c5800640 1 -- 172.21.15.31:0/1819478366 learned_addr learned my addr 172.21.15.31:0/1819478366 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:25:01.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.327+0000 7f88c4e00640 1 --2- 172.21.15.31:0/1819478366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88c0102860 0x7f88c019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:01.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.328+0000 7f88c6200640 1 --2- 172.21.15.31:0/1819478366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c0103220 0x7f88c01a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:01.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.328+0000 7f88c5800640 1 -- 172.21.15.31:0/1819478366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88c0102860 msgr2=0x7f88c019a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:01.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.328+0000 7f88c5800640 1 --2- 172.21.15.31:0/1819478366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88c0102860 0x7f88c019a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:01.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.328+0000 7f88c5800640 1 -- 172.21.15.31:0/1819478366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c0103220 msgr2=0x7f88c01a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:01.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.328+0000 7f88c5800640 1 --2- 172.21.15.31:0/1819478366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c0103220 0x7f88c01a10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:01.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.328+0000 7f88c5800640 1 -- 172.21.15.31:0/1819478366 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c01a1730 con 0x7f88c0101670 2024-04-25T01:25:01.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.328+0000 7f88c4e00640 1 --2- 172.21.15.31:0/1819478366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88c0102860 0x7f88c019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:25:01.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.328+0000 7f88c5800640 1 --2- 172.21.15.31:0/1819478366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88c0101670 0x7f88c0199b10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f88b400e9c0 tx=0x7f88b400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:01.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.328+0000 7f88be000640 1 -- 172.21.15.31:0/1819478366 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b400cd80 con 0x7f88c0101670 2024-04-25T01:25:01.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.328+0000 7f88c6c00640 1 -- 172.21.15.31:0/1819478366 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c01a1a20 con 0x7f88c0101670 2024-04-25T01:25:01.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.328+0000 7f88be000640 1 -- 172.21.15.31:0/1819478366 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b40035f0 con 0x7f88c0101670 2024-04-25T01:25:01.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.328+0000 7f88be000640 1 -- 172.21.15.31:0/1819478366 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4010690 con 0x7f88c0101670 2024-04-25T01:25:01.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.329+0000 7f88c6c00640 1 -- 172.21.15.31:0/1819478366 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c01a1f90 con 0x7f88c0101670 2024-04-25T01:25:01.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.330+0000 7f88c6c00640 1 -- 172.21.15.31:0/1819478366 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88c0068480 con 0x7f88c0101670 2024-04-25T01:25:01.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.331+0000 7f88be000640 1 -- 172.21.15.31:0/1819478366 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f88b40037a0 con 0x7f88c0101670 2024-04-25T01:25:01.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.333+0000 7f88be000640 1 --2- 172.21.15.31:0/1819478366 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f88a80a2b90 0x7f88a80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:01.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.333+0000 7f88be000640 1 -- 172.21.15.31:0/1819478366 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f88b40cf5a0 con 0x7f88c0101670 2024-04-25T01:25:01.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.333+0000 7f88c4e00640 1 --2- 172.21.15.31:0/1819478366 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f88a80a2b90 0x7f88a80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:01.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.334+0000 7f88c4e00640 1 --2- 172.21.15.31:0/1819478366 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f88a80a2b90 0x7f88a80a5050 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f88ac006010 tx=0x7f88ac000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:01.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.337+0000 7f88be000640 1 -- 172.21.15.31:0/1819478366 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f88b409c4f0 con 0x7f88c0101670 2024-04-25T01:25:01.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.575+0000 7f88c6c00640 1 -- 172.21.15.31:0/1819478366 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f88c01a21c0 con 0x7f88c0101670 2024-04-25T01:25:01.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.576+0000 7f88be000640 1 -- 172.21.15.31:0/1819478366 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+591 (secure 0 0 0) 0x7f88b409bdf0 con 0x7f88c0101670 2024-04-25T01:25:01.557 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/1901973432 2024-04-26T01:05:31.932309+0000 2024-04-25T01:25:01.557 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/4004014066 2024-04-26T01:05:31.932309+0000 2024-04-25T01:25:01.557 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/1901973432 2024-04-26T01:05:31.932309+0000 2024-04-25T01:25:01.557 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1930101646 2024-04-26T01:05:31.932309+0000 2024-04-25T01:25:01.557 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/2638309002 2024-04-26T01:04:57.149272+0000 2024-04-25T01:25:01.557 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2238638505 2024-04-26T01:04:57.149272+0000 2024-04-25T01:25:01.557 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2902841144 2024-04-26T01:05:31.932309+0000 2024-04-25T01:25:01.557 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/4190696323 2024-04-26T01:04:57.149272+0000 2024-04-25T01:25:01.558 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/480558521 2024-04-26T01:04:57.149272+0000 2024-04-25T01:25:01.558 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/2638309002 2024-04-26T01:04:57.149272+0000 2024-04-25T01:25:01.558 INFO:teuthology.orchestra.run.smithi031.stderr:listed 10 entries 2024-04-25T01:25:01.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.579+0000 7f88c6c00640 1 -- 172.21.15.31:0/1819478366 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f88a80a2b90 msgr2=0x7f88a80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:01.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.579+0000 7f88c6c00640 1 --2- 172.21.15.31:0/1819478366 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f88a80a2b90 0x7f88a80a5050 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f88ac006010 tx=0x7f88ac000bc0 comp rx=0 tx=0).stop 2024-04-25T01:25:01.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.579+0000 7f88c6c00640 1 -- 172.21.15.31:0/1819478366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88c0101670 msgr2=0x7f88c0199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:01.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.579+0000 7f88c6c00640 1 --2- 172.21.15.31:0/1819478366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88c0101670 0x7f88c0199b10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f88b400e9c0 tx=0x7f88b400ee80 comp rx=0 tx=0).stop 2024-04-25T01:25:01.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.579+0000 7f88c6c00640 1 -- 172.21.15.31:0/1819478366 shutdown_connections 2024-04-25T01:25:01.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.579+0000 7f88c6c00640 1 --2- 172.21.15.31:0/1819478366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c0103220 0x7f88c01a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:01.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.579+0000 7f88c6c00640 1 --2- 172.21.15.31:0/1819478366 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f88a80a2b90 0x7f88a80a5050 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:01.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.579+0000 7f88c6c00640 1 --2- 172.21.15.31:0/1819478366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88c0102860 0x7f88c019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:01.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.579+0000 7f88c6c00640 1 --2- 172.21.15.31:0/1819478366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88c0101670 0x7f88c0199b10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:01.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.579+0000 7f88c6c00640 1 -- 172.21.15.31:0/1819478366 >> 172.21.15.31:0/1819478366 conn(0x7f88c00fcbe0 msgr2=0x7f88c0073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:01.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.579+0000 7f88c6c00640 1 -- 172.21.15.31:0/1819478366 shutdown_connections 2024-04-25T01:25:01.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:01.579+0000 7f88c6c00640 1 -- 172.21.15.31:0/1819478366 wait complete. 2024-04-25T01:25:01.580 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T01:25:01.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:25:01.580 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T01:25:01.782 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T01:25:01.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:25:01.782 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T01:25:01.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:25:01.864 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-04-25T01:25:01.908 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:01 smithi031 ceph-mon[28053]: pgmap v663: 97 pgs: 97 active+clean; 24 MiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T01:25:01.908 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:01 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3126402569' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:25:01.947 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T01:25:01.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:25:01.947 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:25:02.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:25:02.116 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2024-04-25T01:25:02.188 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:01 smithi169 ceph-mon[32759]: pgmap v663: 97 pgs: 97 active+clean; 24 MiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T01:25:02.188 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:01 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3126402569' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:25:02.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:01 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1819478366' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T01:25:02.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:01 smithi040 ceph-mon[32518]: pgmap v663: 97 pgs: 97 active+clean; 24 MiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T01:25:02.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:01 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3126402569' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:25:02.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:01 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1819478366' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T01:25:02.203 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:01 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1819478366' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T01:25:02.250 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T01:25:02.266 INFO:tasks.cephadm:Teardown begin 2024-04-25T01:25:02.266 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T01:25:02.266 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:25:02.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.456+0000 7fd4f3200640 1 -- 172.21.15.31:0/2144405242 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4ec102040 msgr2=0x7fd4ec104430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:02.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.456+0000 7fd4f3200640 1 --2- 172.21.15.31:0/2144405242 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4ec102040 0x7fd4ec104430 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009900 tx=0x7fd4e002a3f0 comp rx=0 tx=0).stop 2024-04-25T01:25:02.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.456+0000 7fd4f3200640 1 -- 172.21.15.31:0/2144405242 shutdown_connections 2024-04-25T01:25:02.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.456+0000 7fd4f3200640 1 --2- 172.21.15.31:0/2144405242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4ec1072a0 0x7fd4ec1096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:02.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.456+0000 7fd4f3200640 1 --2- 172.21.15.31:0/2144405242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4ec104970 0x7fd4ec106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:02.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.456+0000 7fd4f3200640 1 --2- 172.21.15.31:0/2144405242 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4ec102040 0x7fd4ec104430 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:02.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.456+0000 7fd4f3200640 1 -- 172.21.15.31:0/2144405242 >> 172.21.15.31:0/2144405242 conn(0x7fd4ec0fba10 msgr2=0x7fd4ec0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:02.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.456+0000 7fd4f3200640 1 -- 172.21.15.31:0/2144405242 shutdown_connections 2024-04-25T01:25:02.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.456+0000 7fd4f3200640 1 -- 172.21.15.31:0/2144405242 wait complete. 2024-04-25T01:25:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.457+0000 7fd4f3200640 1 Processor -- start 2024-04-25T01:25:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.457+0000 7fd4f3200640 1 -- start start 2024-04-25T01:25:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.457+0000 7fd4f3200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4ec102040 0x7fd4ec199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.457+0000 7fd4f3200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4ec104970 0x7fd4ec19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.457+0000 7fd4f3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4ec1072a0 0x7fd4ec1a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.457+0000 7fd4f3200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec10d2a0 con 0x7fd4ec102040 2024-04-25T01:25:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.457+0000 7fd4f3200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec10d120 con 0x7fd4ec104970 2024-04-25T01:25:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.457+0000 7fd4f3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec10d420 con 0x7fd4ec1072a0 2024-04-25T01:25:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.458+0000 7fd4f1e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4ec102040 0x7fd4ec199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.458+0000 7fd4f1400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4ec104970 0x7fd4ec19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.458+0000 7fd4f2800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4ec1072a0 0x7fd4ec1a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.458+0000 7fd4f1400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4ec104970 0x7fd4ec19a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:57534/0 (socket says 172.21.15.31:57534) 2024-04-25T01:25:02.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.458+0000 7fd4f1400640 1 -- 172.21.15.31:0/740542168 learned_addr learned my addr 172.21.15.31:0/740542168 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:25:02.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.458+0000 7fd4f2800640 1 -- 172.21.15.31:0/740542168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4ec104970 msgr2=0x7fd4ec19a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:02.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.458+0000 7fd4f2800640 1 --2- 172.21.15.31:0/740542168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4ec104970 0x7fd4ec19a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:02.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.458+0000 7fd4f2800640 1 -- 172.21.15.31:0/740542168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4ec102040 msgr2=0x7fd4ec199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:02.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.458+0000 7fd4f2800640 1 --2- 172.21.15.31:0/740542168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4ec102040 0x7fd4ec199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:02.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.458+0000 7fd4f2800640 1 -- 172.21.15.31:0/740542168 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ec1a1710 con 0x7fd4ec1072a0 2024-04-25T01:25:02.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.458+0000 7fd4f2800640 1 --2- 172.21.15.31:0/740542168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4ec1072a0 0x7fd4ec1a10a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8009fd0 tx=0x7fd4e800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:02.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.459+0000 7fd4e6a00640 1 -- 172.21.15.31:0/740542168 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e800ce20 con 0x7fd4ec1072a0 2024-04-25T01:25:02.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.459+0000 7fd4e6a00640 1 -- 172.21.15.31:0/740542168 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e8005020 con 0x7fd4ec1072a0 2024-04-25T01:25:02.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.459+0000 7fd4f3200640 1 -- 172.21.15.31:0/740542168 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4ec1a1a00 con 0x7fd4ec1072a0 2024-04-25T01:25:02.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.459+0000 7fd4e6a00640 1 -- 172.21.15.31:0/740542168 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8010720 con 0x7fd4ec1072a0 2024-04-25T01:25:02.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.459+0000 7fd4f3200640 1 -- 172.21.15.31:0/740542168 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4ec1a1fa0 con 0x7fd4ec1072a0 2024-04-25T01:25:02.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.460+0000 7fd4f3200640 1 -- 172.21.15.31:0/740542168 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4ec108df0 con 0x7fd4ec1072a0 2024-04-25T01:25:02.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.462+0000 7fd4e6a00640 1 -- 172.21.15.31:0/740542168 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd4e8005540 con 0x7fd4ec1072a0 2024-04-25T01:25:02.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.463+0000 7fd4e6a00640 1 --2- 172.21.15.31:0/740542168 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd4d40a2c10 0x7fd4d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:02.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.463+0000 7fd4e6a00640 1 -- 172.21.15.31:0/740542168 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fd4e80cf5c0 con 0x7fd4ec1072a0 2024-04-25T01:25:02.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.463+0000 7fd4f1e00640 1 --2- 172.21.15.31:0/740542168 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd4d40a2c10 0x7fd4d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:02.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.464+0000 7fd4f1e00640 1 --2- 172.21.15.31:0/740542168 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd4d40a2c10 0x7fd4d40a50d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0031440 tx=0x7fd4e00315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:02.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.465+0000 7fd4e6a00640 1 -- 172.21.15.31:0/740542168 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd4e809c510 con 0x7fd4ec1072a0 2024-04-25T01:25:02.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.704+0000 7fd4f3200640 1 -- 172.21.15.31:0/740542168 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4ec02d3c0 con 0x7fd4ec1072a0 2024-04-25T01:25:02.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.705+0000 7fd4e6a00640 1 -- 172.21.15.31:0/740542168 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11860 (secure 0 0 0) 0x7fd4e809be10 con 0x7fd4ec1072a0 2024-04-25T01:25:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:25:02.686 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/498218714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":498218714},{"type":"v1","addr":"172.21.15.31:6835","nonce":498218714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2647199475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2647199475},{"type":"v1","addr":"172.21.15.31:6839","nonce":2647199475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/30518311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":30518311},{"type":"v1","addr":"172.21.15.31:6841","nonce":30518311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/420362823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":420362823},{"type":"v1","addr":"172.21.15.169:6833","nonce":420362823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1943710438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1943710438},{"type":"v1","addr":"172.21.15.40:6833","nonce":1943710438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/4191344115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":4191344115},{"type":"v1","addr":"172.21.15.31:6837","nonce":4191344115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:13:30.975848+0000","modified":"2024-04-25T01:24:24.699113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24565,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6835/2916305801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2916305801},{"type":"v1","addr":"172.21.15.40:6835","nonce":2916305801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.169:6835/2253264784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2253264784},{"type":"v1","addr":"172.21.15.169:6835","nonce":2253264784}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6837/2883308641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2883308641},{"type":"v1","addr":"172.21.15.169:6837","nonce":2883308641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/590109562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":590109562},{"type":"v1","addr":"172.21.15.40:6837","nonce":590109562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/4185710218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":4185710218},{"type":"v1","addr":"172.21.15.169:6839","nonce":4185710218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6839/927194004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":927194004},{"type":"v1","addr":"172.21.15.40:6839","nonce":927194004}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24565,24596,24532]},"id":1}]} 2024-04-25T01:25:02.686 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 34 2024-04-25T01:25:02.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.707+0000 7fd4f3200640 1 -- 172.21.15.31:0/740542168 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd4d40a2c10 msgr2=0x7fd4d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:02.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.707+0000 7fd4f3200640 1 --2- 172.21.15.31:0/740542168 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd4d40a2c10 0x7fd4d40a50d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0031440 tx=0x7fd4e00315b0 comp rx=0 tx=0).stop 2024-04-25T01:25:02.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.707+0000 7fd4f3200640 1 -- 172.21.15.31:0/740542168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4ec1072a0 msgr2=0x7fd4ec1a10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:02.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.707+0000 7fd4f3200640 1 --2- 172.21.15.31:0/740542168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4ec1072a0 0x7fd4ec1a10a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8009fd0 tx=0x7fd4e800ef10 comp rx=0 tx=0).stop 2024-04-25T01:25:02.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.708+0000 7fd4f3200640 1 -- 172.21.15.31:0/740542168 shutdown_connections 2024-04-25T01:25:02.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.708+0000 7fd4f3200640 1 --2- 172.21.15.31:0/740542168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4ec1072a0 0x7fd4ec1a10a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:02.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.708+0000 7fd4f3200640 1 --2- 172.21.15.31:0/740542168 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fd4d40a2c10 0x7fd4d40a50d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:02.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.708+0000 7fd4f3200640 1 --2- 172.21.15.31:0/740542168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4ec104970 0x7fd4ec19a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:02.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.708+0000 7fd4f3200640 1 --2- 172.21.15.31:0/740542168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4ec102040 0x7fd4ec199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:02.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.708+0000 7fd4f3200640 1 -- 172.21.15.31:0/740542168 >> 172.21.15.31:0/740542168 conn(0x7fd4ec0fba10 msgr2=0x7fd4ec101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:02.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.708+0000 7fd4f3200640 1 -- 172.21.15.31:0/740542168 shutdown_connections 2024-04-25T01:25:02.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.708+0000 7fd4f3200640 1 -- 172.21.15.31:0/740542168 wait complete. 2024-04-25T01:25:02.705 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:25:02.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.906+0000 7fbe9be00640 1 -- 172.21.15.31:0/8569877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe9c104970 msgr2=0x7fbe9c106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:02.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.906+0000 7fbe9be00640 1 --2- 172.21.15.31:0/8569877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe9c104970 0x7fbe9c106d60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009900 tx=0x7fbe8402a3f0 comp rx=0 tx=0).stop 2024-04-25T01:25:02.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.906+0000 7fbe9be00640 1 -- 172.21.15.31:0/8569877 shutdown_connections 2024-04-25T01:25:02.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.906+0000 7fbe9be00640 1 --2- 172.21.15.31:0/8569877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe9c1072a0 0x7fbe9c1096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:02.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.906+0000 7fbe9be00640 1 --2- 172.21.15.31:0/8569877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe9c104970 0x7fbe9c106d60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:02.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.906+0000 7fbe9be00640 1 --2- 172.21.15.31:0/8569877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe9c102040 0x7fbe9c104430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:02.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.906+0000 7fbe9be00640 1 -- 172.21.15.31:0/8569877 >> 172.21.15.31:0/8569877 conn(0x7fbe9c0fba10 msgr2=0x7fbe9c0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:02.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.906+0000 7fbe9be00640 1 -- 172.21.15.31:0/8569877 shutdown_connections 2024-04-25T01:25:02.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.906+0000 7fbe9be00640 1 -- 172.21.15.31:0/8569877 wait complete. 2024-04-25T01:25:02.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.907+0000 7fbe9be00640 1 Processor -- start 2024-04-25T01:25:02.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.907+0000 7fbe9be00640 1 -- start start 2024-04-25T01:25:02.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.908+0000 7fbe9be00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe9c102040 0x7fbe9c199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:02.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.908+0000 7fbe9be00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe9c104970 0x7fbe9c19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:02.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.908+0000 7fbe9be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe9c1072a0 0x7fbe9c1a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:02.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.908+0000 7fbe9be00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c10d270 con 0x7fbe9c104970 2024-04-25T01:25:02.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.908+0000 7fbe9be00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c10d0f0 con 0x7fbe9c102040 2024-04-25T01:25:02.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.908+0000 7fbe9be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c10d3f0 con 0x7fbe9c1072a0 2024-04-25T01:25:02.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.908+0000 7fbe9a000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe9c104970 0x7fbe9c19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:02.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.908+0000 7fbe9a000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe9c104970 0x7fbe9c19a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60536/0 (socket says 172.21.15.31:60536) 2024-04-25T01:25:02.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.908+0000 7fbe9aa00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe9c102040 0x7fbe9c199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:02.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.908+0000 7fbe9a000640 1 -- 172.21.15.31:0/1822972288 learned_addr learned my addr 172.21.15.31:0/1822972288 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:25:02.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.908+0000 7fbe9b400640 1 --2- 172.21.15.31:0/1822972288 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe9c1072a0 0x7fbe9c1a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:02.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.909+0000 7fbe9b400640 1 -- 172.21.15.31:0/1822972288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe9c102040 msgr2=0x7fbe9c199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:02.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.909+0000 7fbe9b400640 1 --2- 172.21.15.31:0/1822972288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe9c102040 0x7fbe9c199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:02.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.909+0000 7fbe9b400640 1 -- 172.21.15.31:0/1822972288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe9c104970 msgr2=0x7fbe9c19a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:02.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.909+0000 7fbe9b400640 1 --2- 172.21.15.31:0/1822972288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe9c104970 0x7fbe9c19a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:02.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.909+0000 7fbe9b400640 1 -- 172.21.15.31:0/1822972288 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9c1a1700 con 0x7fbe9c1072a0 2024-04-25T01:25:02.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.909+0000 7fbe9a000640 1 --2- 172.21.15.31:0/1822972288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe9c104970 0x7fbe9c19a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:25:02.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.909+0000 7fbe9b400640 1 --2- 172.21.15.31:0/1822972288 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe9c1072a0 0x7fbe9c1a1090 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400e970 tx=0x7fbe9400ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:02.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.909+0000 7fbe93400640 1 -- 172.21.15.31:0/1822972288 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9400ccc0 con 0x7fbe9c1072a0 2024-04-25T01:25:02.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.909+0000 7fbe93400640 1 -- 172.21.15.31:0/1822972288 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe94005020 con 0x7fbe9c1072a0 2024-04-25T01:25:02.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.909+0000 7fbe9be00640 1 -- 172.21.15.31:0/1822972288 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe9c1a19f0 con 0x7fbe9c1072a0 2024-04-25T01:25:02.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.909+0000 7fbe93400640 1 -- 172.21.15.31:0/1822972288 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94004a00 con 0x7fbe9c1072a0 2024-04-25T01:25:02.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.909+0000 7fbe9be00640 1 -- 172.21.15.31:0/1822972288 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe9c1a1f10 con 0x7fbe9c1072a0 2024-04-25T01:25:02.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.910+0000 7fbe9be00640 1 -- 172.21.15.31:0/1822972288 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe9c108df0 con 0x7fbe9c1072a0 2024-04-25T01:25:02.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.913+0000 7fbe93400640 1 -- 172.21.15.31:0/1822972288 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbe940051d0 con 0x7fbe9c1072a0 2024-04-25T01:25:02.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.914+0000 7fbe93400640 1 --2- 172.21.15.31:0/1822972288 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbe800a2c10 0x7fbe800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:02.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.914+0000 7fbe93400640 1 -- 172.21.15.31:0/1822972288 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7fbe940cef40 con 0x7fbe9c1072a0 2024-04-25T01:25:02.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.914+0000 7fbe9aa00640 1 --2- 172.21.15.31:0/1822972288 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbe800a2c10 0x7fbe800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:02.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.915+0000 7fbe9aa00640 1 --2- 172.21.15.31:0/1822972288 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbe800a2c10 0x7fbe800a50d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c005fd0 tx=0x7fbe8c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:02.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:02.918+0000 7fbe93400640 1 -- 172.21.15.31:0/1822972288 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbe9409be90 con 0x7fbe9c1072a0 2024-04-25T01:25:02.999 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:02 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/740542168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:25:03.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.165+0000 7fbe9be00640 1 -- 172.21.15.31:0/1822972288 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe9c062340 con 0x7fbe9c1072a0 2024-04-25T01:25:03.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.166+0000 7fbe93400640 1 -- 172.21.15.31:0/1822972288 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11860 (secure 0 0 0) 0x7fbe9409b790 con 0x7fbe9c1072a0 2024-04-25T01:25:03.147 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:25:03.148 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/498218714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":498218714},{"type":"v1","addr":"172.21.15.31:6835","nonce":498218714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2647199475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2647199475},{"type":"v1","addr":"172.21.15.31:6839","nonce":2647199475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/30518311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":30518311},{"type":"v1","addr":"172.21.15.31:6841","nonce":30518311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/420362823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":420362823},{"type":"v1","addr":"172.21.15.169:6833","nonce":420362823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1943710438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1943710438},{"type":"v1","addr":"172.21.15.40:6833","nonce":1943710438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/4191344115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":4191344115},{"type":"v1","addr":"172.21.15.31:6837","nonce":4191344115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:13:30.975848+0000","modified":"2024-04-25T01:24:24.699113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24565,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6835/2916305801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2916305801},{"type":"v1","addr":"172.21.15.40:6835","nonce":2916305801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.169:6835/2253264784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2253264784},{"type":"v1","addr":"172.21.15.169:6835","nonce":2253264784}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6837/2883308641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2883308641},{"type":"v1","addr":"172.21.15.169:6837","nonce":2883308641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/590109562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":590109562},{"type":"v1","addr":"172.21.15.40:6837","nonce":590109562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.169:6839/4185710218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":4185710218},{"type":"v1","addr":"172.21.15.169:6839","nonce":4185710218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6839/927194004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":927194004},{"type":"v1","addr":"172.21.15.40:6839","nonce":927194004}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24565,24596,24532]},"id":1}]} 2024-04-25T01:25:03.148 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 34 2024-04-25T01:25:03.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.169+0000 7fbe9be00640 1 -- 172.21.15.31:0/1822972288 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbe800a2c10 msgr2=0x7fbe800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:03.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.169+0000 7fbe9be00640 1 --2- 172.21.15.31:0/1822972288 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbe800a2c10 0x7fbe800a50d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c005fd0 tx=0x7fbe8c009500 comp rx=0 tx=0).stop 2024-04-25T01:25:03.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.169+0000 7fbe9be00640 1 -- 172.21.15.31:0/1822972288 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe9c1072a0 msgr2=0x7fbe9c1a1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:03.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.169+0000 7fbe9be00640 1 --2- 172.21.15.31:0/1822972288 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe9c1072a0 0x7fbe9c1a1090 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400e970 tx=0x7fbe9400ee30 comp rx=0 tx=0).stop 2024-04-25T01:25:03.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.170+0000 7fbe9be00640 1 -- 172.21.15.31:0/1822972288 shutdown_connections 2024-04-25T01:25:03.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.170+0000 7fbe9be00640 1 --2- 172.21.15.31:0/1822972288 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe9c1072a0 0x7fbe9c1a1090 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:03.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.170+0000 7fbe9be00640 1 --2- 172.21.15.31:0/1822972288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe9c104970 0x7fbe9c19a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:03.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.170+0000 7fbe9be00640 1 --2- 172.21.15.31:0/1822972288 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fbe800a2c10 0x7fbe800a50d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:03.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.170+0000 7fbe9be00640 1 --2- 172.21.15.31:0/1822972288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe9c102040 0x7fbe9c199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:03.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.170+0000 7fbe9be00640 1 -- 172.21.15.31:0/1822972288 >> 172.21.15.31:0/1822972288 conn(0x7fbe9c0fba10 msgr2=0x7fbe9c101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:03.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.170+0000 7fbe9be00640 1 -- 172.21.15.31:0/1822972288 shutdown_connections 2024-04-25T01:25:03.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.170+0000 7fbe9be00640 1 -- 172.21.15.31:0/1822972288 wait complete. 2024-04-25T01:25:03.169 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:25:03.188 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:02 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/740542168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:25:03.191 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:02 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/740542168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:25:03.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.373+0000 7f5d5fe00640 1 -- 172.21.15.31:0/1183576912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d60104970 msgr2=0x7f5d60106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:03.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.373+0000 7f5d5fe00640 1 --2- 172.21.15.31:0/1183576912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d60104970 0x7f5d60106d60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c009900 tx=0x7f5d4c02a3f0 comp rx=0 tx=0).stop 2024-04-25T01:25:03.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.373+0000 7f5d5fe00640 1 -- 172.21.15.31:0/1183576912 shutdown_connections 2024-04-25T01:25:03.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.373+0000 7f5d5fe00640 1 --2- 172.21.15.31:0/1183576912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d601072a0 0x7f5d601096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:03.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.373+0000 7f5d5fe00640 1 --2- 172.21.15.31:0/1183576912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d60104970 0x7f5d60106d60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:03.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.373+0000 7f5d5fe00640 1 --2- 172.21.15.31:0/1183576912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d60102040 0x7f5d60104430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:03.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.373+0000 7f5d5fe00640 1 -- 172.21.15.31:0/1183576912 >> 172.21.15.31:0/1183576912 conn(0x7f5d600fba10 msgr2=0x7f5d600fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:03.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.373+0000 7f5d5fe00640 1 -- 172.21.15.31:0/1183576912 shutdown_connections 2024-04-25T01:25:03.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.373+0000 7f5d5fe00640 1 -- 172.21.15.31:0/1183576912 wait complete. 2024-04-25T01:25:03.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.374+0000 7f5d5fe00640 1 Processor -- start 2024-04-25T01:25:03.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.374+0000 7f5d5fe00640 1 -- start start 2024-04-25T01:25:03.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.375+0000 7f5d5fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d60102040 0x7f5d601978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:03.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.375+0000 7f5d5fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d60104970 0x7f5d60197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:03.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.375+0000 7f5d5fe00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d601072a0 0x7f5d6019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:03.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.375+0000 7f5d5fe00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6010d250 con 0x7f5d60102040 2024-04-25T01:25:03.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.375+0000 7f5d5fe00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6010d0d0 con 0x7f5d601072a0 2024-04-25T01:25:03.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.375+0000 7f5d5fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6010d3d0 con 0x7f5d60104970 2024-04-25T01:25:03.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.375+0000 7f5d5ea00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d60102040 0x7f5d601978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:03.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.375+0000 7f5d5ea00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d60102040 0x7f5d601978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60560/0 (socket says 172.21.15.31:60560) 2024-04-25T01:25:03.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.375+0000 7f5d5ea00640 1 -- 172.21.15.31:0/1433545246 learned_addr learned my addr 172.21.15.31:0/1433545246 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:25:03.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.375+0000 7f5d5e000640 1 --2- 172.21.15.31:0/1433545246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d60104970 0x7f5d60197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:03.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.375+0000 7f5d5f400640 1 --2- 172.21.15.31:0/1433545246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d601072a0 0x7f5d6019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:03.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.376+0000 7f5d5ea00640 1 -- 172.21.15.31:0/1433545246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d60104970 msgr2=0x7f5d60197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:03.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.376+0000 7f5d5ea00640 1 --2- 172.21.15.31:0/1433545246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d60104970 0x7f5d60197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:03.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.376+0000 7f5d5ea00640 1 -- 172.21.15.31:0/1433545246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d601072a0 msgr2=0x7f5d6019ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:03.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.376+0000 7f5d5ea00640 1 --2- 172.21.15.31:0/1433545246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d601072a0 0x7f5d6019ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:03.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.376+0000 7f5d5ea00640 1 -- 172.21.15.31:0/1433545246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6019f4f0 con 0x7f5d60102040 2024-04-25T01:25:03.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.376+0000 7f5d5ea00640 1 --2- 172.21.15.31:0/1433545246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d60102040 0x7f5d601978d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5d5000e9f0 tx=0x7f5d5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:03.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.376+0000 7f5d57400640 1 -- 172.21.15.31:0/1433545246 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5000cd80 con 0x7f5d60102040 2024-04-25T01:25:03.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.376+0000 7f5d57400640 1 -- 172.21.15.31:0/1433545246 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d50010910 con 0x7f5d60102040 2024-04-25T01:25:03.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.376+0000 7f5d5fe00640 1 -- 172.21.15.31:0/1433545246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d6019f7e0 con 0x7f5d60102040 2024-04-25T01:25:03.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.376+0000 7f5d57400640 1 -- 172.21.15.31:0/1433545246 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d50018990 con 0x7f5d60102040 2024-04-25T01:25:03.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.376+0000 7f5d5fe00640 1 -- 172.21.15.31:0/1433545246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d6019fd00 con 0x7f5d60102040 2024-04-25T01:25:03.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.377+0000 7f5d5fe00640 1 -- 172.21.15.31:0/1433545246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d600fd500 con 0x7f5d60102040 2024-04-25T01:25:03.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.379+0000 7f5d57400640 1 -- 172.21.15.31:0/1433545246 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5d50010ac0 con 0x7f5d60102040 2024-04-25T01:25:03.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.381+0000 7f5d57400640 1 --2- 172.21.15.31:0/1433545246 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5d440a2b90 0x7f5d440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:03.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.381+0000 7f5d57400640 1 -- 172.21.15.31:0/1433545246 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f5d500cf360 con 0x7f5d60102040 2024-04-25T01:25:03.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.381+0000 7f5d5e000640 1 --2- 172.21.15.31:0/1433545246 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5d440a2b90 0x7f5d440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:03.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.382+0000 7f5d5e000640 1 --2- 172.21.15.31:0/1433545246 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5d440a2b90 0x7f5d440a5050 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c005ab0 tx=0x7f5d4c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:03.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.386+0000 7f5d57400640 1 -- 172.21.15.31:0/1433545246 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5d5009bc60 con 0x7f5d60102040 2024-04-25T01:25:03.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.603+0000 7f5d5fe00640 1 -- 172.21.15.31:0/1433545246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d60062340 con 0x7f5d60102040 2024-04-25T01:25:03.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.605+0000 7f5d57400640 1 -- 172.21.15.31:0/1433545246 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22530 (secure 0 0 0) 0x7f5d5009ff60 con 0x7f5d60102040 2024-04-25T01:25:03.586 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T01:25:03.586 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":89,"fsid":"95849b94-029f-11ef-bc93-c7b262605968","created":"2024-04-25T01:04:23.310856+0000","modified":"2024-04-25T01:14:33.793638+0000","last_up_change":"2024-04-25T01:12:29.393907+0000","last_in_change":"2024-04-25T01:12:08.978148+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:18.662391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:13:29.258771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:13:30.480996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa08e4c-fe6b-4116-bdf5-1adf5b465d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6803","nonce":788174091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6805","nonce":788174091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6809","nonce":788174091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":788174091},{"type":"v1","addr":"172.21.15.31:6807","nonce":788174091}]},"public_addr":"172.21.15.31:6803/788174091","cluster_addr":"172.21.15.31:6805/788174091","heartbeat_back_addr":"172.21.15.31:6809/788174091","heartbeat_front_addr":"172.21.15.31:6807/788174091","state":["exists","up"]},{"osd":1,"uuid":"85339fe7-0dbc-4485-9334-2b039993c1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6801","nonce":3196394509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6803","nonce":3196394509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6807","nonce":3196394509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3196394509},{"type":"v1","addr":"172.21.15.40:6805","nonce":3196394509}]},"public_addr":"172.21.15.40:6801/3196394509","cluster_addr":"172.21.15.40:6803/3196394509","heartbeat_back_addr":"172.21.15.40:6807/3196394509","heartbeat_front_addr":"172.21.15.40:6805/3196394509","state":["exists","up"]},{"osd":2,"uuid":"ab62ecb2-8aae-4987-9e2e-589e0f2fee5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6801","nonce":1955211197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6803","nonce":1955211197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6807","nonce":1955211197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":1955211197},{"type":"v1","addr":"172.21.15.169:6805","nonce":1955211197}]},"public_addr":"172.21.15.169:6801/1955211197","cluster_addr":"172.21.15.169:6803/1955211197","heartbeat_back_addr":"172.21.15.169:6807/1955211197","heartbeat_front_addr":"172.21.15.169:6805/1955211197","state":["exists","up"]},{"osd":3,"uuid":"8b5a7881-3401-495c-8ca4-c90685919d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6811","nonce":1228688643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6813","nonce":1228688643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6817","nonce":1228688643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1228688643},{"type":"v1","addr":"172.21.15.31:6815","nonce":1228688643}]},"public_addr":"172.21.15.31:6811/1228688643","cluster_addr":"172.21.15.31:6813/1228688643","heartbeat_back_addr":"172.21.15.31:6817/1228688643","heartbeat_front_addr":"172.21.15.31:6815/1228688643","state":["exists","up"]},{"osd":4,"uuid":"56deb61d-a334-4cf2-8ac7-fbb6e6947092","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6809","nonce":1913779091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6811","nonce":1913779091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6815","nonce":1913779091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1913779091},{"type":"v1","addr":"172.21.15.40:6813","nonce":1913779091}]},"public_addr":"172.21.15.40:6809/1913779091","cluster_addr":"172.21.15.40:6811/1913779091","heartbeat_back_addr":"172.21.15.40:6815/1913779091","heartbeat_front_addr":"172.21.15.40:6813/1913779091","state":["exists","up"]},{"osd":5,"uuid":"8da55693-f9c7-49a7-99e0-165078f8001b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6809","nonce":2549650781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6811","nonce":2549650781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6815","nonce":2549650781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2549650781},{"type":"v1","addr":"172.21.15.169:6813","nonce":2549650781}]},"public_addr":"172.21.15.169:6809/2549650781","cluster_addr":"172.21.15.169:6811/2549650781","heartbeat_back_addr":"172.21.15.169:6815/2549650781","heartbeat_front_addr":"172.21.15.169:6813/2549650781","state":["exists","up"]},{"osd":6,"uuid":"fc8350d2-1ad6-43cb-89f4-d336afc99ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6819","nonce":611507624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6821","nonce":611507624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6825","nonce":611507624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":611507624},{"type":"v1","addr":"172.21.15.31:6823","nonce":611507624}]},"public_addr":"172.21.15.31:6819/611507624","cluster_addr":"172.21.15.31:6821/611507624","heartbeat_back_addr":"172.21.15.31:6825/611507624","heartbeat_front_addr":"172.21.15.31:6823/611507624","state":["exists","up"]},{"osd":7,"uuid":"3121b58b-8c94-40b5-9d48-f56552d7c39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6817","nonce":3776085849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6819","nonce":3776085849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6823","nonce":3776085849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3776085849},{"type":"v1","addr":"172.21.15.40:6821","nonce":3776085849}]},"public_addr":"172.21.15.40:6817/3776085849","cluster_addr":"172.21.15.40:6819/3776085849","heartbeat_back_addr":"172.21.15.40:6823/3776085849","heartbeat_front_addr":"172.21.15.40:6821/3776085849","state":["exists","up"]},{"osd":8,"uuid":"f60407df-91f5-477a-9702-ddaef24faf82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6817","nonce":2736546122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6819","nonce":2736546122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6823","nonce":2736546122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2736546122},{"type":"v1","addr":"172.21.15.169:6821","nonce":2736546122}]},"public_addr":"172.21.15.169:6817/2736546122","cluster_addr":"172.21.15.169:6819/2736546122","heartbeat_back_addr":"172.21.15.169:6823/2736546122","heartbeat_front_addr":"172.21.15.169:6821/2736546122","state":["exists","up"]},{"osd":9,"uuid":"7506a407-1960-49bc-9e38-7f2a8bf996ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6827","nonce":1187207880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6829","nonce":1187207880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6833","nonce":1187207880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1187207880},{"type":"v1","addr":"172.21.15.31:6831","nonce":1187207880}]},"public_addr":"172.21.15.31:6827/1187207880","cluster_addr":"172.21.15.31:6829/1187207880","heartbeat_back_addr":"172.21.15.31:6833/1187207880","heartbeat_front_addr":"172.21.15.31:6831/1187207880","state":["exists","up"]},{"osd":10,"uuid":"831c48e2-b0d0-451f-959b-228e927542cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6825","nonce":3050148001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6827","nonce":3050148001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6831","nonce":3050148001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3050148001},{"type":"v1","addr":"172.21.15.40:6829","nonce":3050148001}]},"public_addr":"172.21.15.40:6825/3050148001","cluster_addr":"172.21.15.40:6827/3050148001","heartbeat_back_addr":"172.21.15.40:6831/3050148001","heartbeat_front_addr":"172.21.15.40:6829/3050148001","state":["exists","up"]},{"osd":11,"uuid":"144fe0aa-f757-4b3c-90ad-99775e17d57f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6825","nonce":262401944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6827","nonce":262401944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6831","nonce":262401944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":262401944},{"type":"v1","addr":"172.21.15.169:6829","nonce":262401944}]},"public_addr":"172.21.15.169:6825/262401944","cluster_addr":"172.21.15.169:6827/262401944","heartbeat_back_addr":"172.21.15.169:6831/262401944","heartbeat_front_addr":"172.21.15.169:6829/262401944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:25.640995+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:07:51.036909+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.635633+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:48.467912+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:14.454971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:09:37.609315+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:12.441807+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:10:38.239215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:02.230758+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:11:35.762553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:02.051405+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:12:26.569593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6801/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4004014066":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/1901973432":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/1930101646":"2024-04-26T01:05:31.932309+0000","172.21.15.31:6800/2638309002":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2238638505":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/2902841144":"2024-04-26T01:05:31.932309+0000","172.21.15.31:0/4190696323":"2024-04-26T01:04:57.149272+0000","172.21.15.31:0/480558521":"2024-04-26T01:04:57.149272+0000","172.21.15.31:6801/2638309002":"2024-04-26T01:04:57.149272+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:25:03.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.608+0000 7f5d5fe00640 1 -- 172.21.15.31:0/1433545246 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5d440a2b90 msgr2=0x7f5d440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:03.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.608+0000 7f5d5fe00640 1 --2- 172.21.15.31:0/1433545246 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5d440a2b90 0x7f5d440a5050 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c005ab0 tx=0x7f5d4c000bc0 comp rx=0 tx=0).stop 2024-04-25T01:25:03.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.608+0000 7f5d5fe00640 1 -- 172.21.15.31:0/1433545246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d60102040 msgr2=0x7f5d601978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:03.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.608+0000 7f5d5fe00640 1 --2- 172.21.15.31:0/1433545246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d60102040 0x7f5d601978d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5d5000e9f0 tx=0x7f5d5000eeb0 comp rx=0 tx=0).stop 2024-04-25T01:25:03.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.608+0000 7f5d5fe00640 1 -- 172.21.15.31:0/1433545246 shutdown_connections 2024-04-25T01:25:03.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.608+0000 7f5d5fe00640 1 --2- 172.21.15.31:0/1433545246 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f5d440a2b90 0x7f5d440a5050 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:03.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.608+0000 7f5d5fe00640 1 --2- 172.21.15.31:0/1433545246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d601072a0 0x7f5d6019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:03.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.608+0000 7f5d5fe00640 1 --2- 172.21.15.31:0/1433545246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d60104970 0x7f5d60197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:03.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.608+0000 7f5d5fe00640 1 --2- 172.21.15.31:0/1433545246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d60102040 0x7f5d601978d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:03.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.608+0000 7f5d5fe00640 1 -- 172.21.15.31:0/1433545246 >> 172.21.15.31:0/1433545246 conn(0x7f5d600fba10 msgr2=0x7f5d60102900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:03.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.608+0000 7f5d5fe00640 1 -- 172.21.15.31:0/1433545246 shutdown_connections 2024-04-25T01:25:03.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.608+0000 7f5d5fe00640 1 -- 172.21.15.31:0/1433545246 wait complete. 2024-04-25T01:25:03.607 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T01:25:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.803+0000 7f2200c00640 1 -- 172.21.15.31:0/3004943029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc1020d0 msgr2=0x7f21fc1044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.803+0000 7f2200c00640 1 --2- 172.21.15.31:0/3004943029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc1020d0 0x7f21fc1044c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009ca0 tx=0x7f21ec02a8c0 comp rx=0 tx=0).stop 2024-04-25T01:25:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.804+0000 7f2200c00640 1 -- 172.21.15.31:0/3004943029 shutdown_connections 2024-04-25T01:25:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.804+0000 7f2200c00640 1 --2- 172.21.15.31:0/3004943029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21fc107330 0x7f21fc109780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.804+0000 7f2200c00640 1 --2- 172.21.15.31:0/3004943029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21fc104a00 0x7f21fc106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.804+0000 7f2200c00640 1 --2- 172.21.15.31:0/3004943029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc1020d0 0x7f21fc1044c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.804+0000 7f2200c00640 1 -- 172.21.15.31:0/3004943029 >> 172.21.15.31:0/3004943029 conn(0x7f21fc0fbae0 msgr2=0x7f21fc0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.804+0000 7f2200c00640 1 -- 172.21.15.31:0/3004943029 shutdown_connections 2024-04-25T01:25:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.804+0000 7f2200c00640 1 -- 172.21.15.31:0/3004943029 wait complete. 2024-04-25T01:25:03.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.805+0000 7f2200c00640 1 Processor -- start 2024-04-25T01:25:03.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.805+0000 7f2200c00640 1 -- start start 2024-04-25T01:25:03.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.806+0000 7f2200c00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21fc1020d0 0x7f21fc1019f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:03.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.806+0000 7f2200c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc104a00 0x7f21fc100080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:03.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.806+0000 7f2200c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21fc107330 0x7f21fc1005c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:03.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.806+0000 7f2200c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc10d330 con 0x7f21fc107330 2024-04-25T01:25:03.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.806+0000 7f2200c00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc10d1b0 con 0x7f21fc1020d0 2024-04-25T01:25:03.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.806+0000 7f2200c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc10d4b0 con 0x7f21fc104a00 2024-04-25T01:25:03.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.806+0000 7f21faa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc104a00 0x7f21fc100080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:03.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.806+0000 7f21fb400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21fc1020d0 0x7f21fc1019f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:03.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.806+0000 7f21fbe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21fc107330 0x7f21fc1005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:03.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.806+0000 7f21faa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc104a00 0x7f21fc100080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:59040/0 (socket says 172.21.15.31:59040) 2024-04-25T01:25:03.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.806+0000 7f21faa00640 1 -- 172.21.15.31:0/2750596460 learned_addr learned my addr 172.21.15.31:0/2750596460 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:25:03.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.806+0000 7f21faa00640 1 -- 172.21.15.31:0/2750596460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21fc1020d0 msgr2=0x7f21fc1019f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:03.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.806+0000 7f21faa00640 1 --2- 172.21.15.31:0/2750596460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21fc1020d0 0x7f21fc1019f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:03.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.806+0000 7f21faa00640 1 -- 172.21.15.31:0/2750596460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21fc107330 msgr2=0x7f21fc1005c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:03.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.806+0000 7f21faa00640 1 --2- 172.21.15.31:0/2750596460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21fc107330 0x7f21fc1005c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:03.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.806+0000 7f21faa00640 1 -- 172.21.15.31:0/2750596460 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21fc100e80 con 0x7f21fc104a00 2024-04-25T01:25:03.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.807+0000 7f21faa00640 1 --2- 172.21.15.31:0/2750596460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc104a00 0x7f21fc100080 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f21e400d740 tx=0x7f21e400dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:03.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.807+0000 7f21f3e00640 1 -- 172.21.15.31:0/2750596460 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4017070 con 0x7f21fc104a00 2024-04-25T01:25:03.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.807+0000 7f2200c00640 1 -- 172.21.15.31:0/2750596460 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21fc1a1af0 con 0x7f21fc104a00 2024-04-25T01:25:03.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.807+0000 7f21f3e00640 1 -- 172.21.15.31:0/2750596460 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21e400de50 con 0x7f21fc104a00 2024-04-25T01:25:03.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.807+0000 7f21f3e00640 1 -- 172.21.15.31:0/2750596460 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4004e00 con 0x7f21fc104a00 2024-04-25T01:25:03.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.807+0000 7f2200c00640 1 -- 172.21.15.31:0/2750596460 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21fc1a2010 con 0x7f21fc104a00 2024-04-25T01:25:03.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.808+0000 7f2200c00640 1 -- 172.21.15.31:0/2750596460 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21fc0fd5d0 con 0x7f21fc104a00 2024-04-25T01:25:03.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.811+0000 7f21f3e00640 1 -- 172.21.15.31:0/2750596460 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f21e4004fa0 con 0x7f21fc104a00 2024-04-25T01:25:03.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.812+0000 7f21f3e00640 1 --2- 172.21.15.31:0/2750596460 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f21e00a2b90 0x7f21e00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:03.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.812+0000 7f21f3e00640 1 -- 172.21.15.31:0/2750596460 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8161+0+0 (secure 0 0 0) 0x7f21e40ce970 con 0x7f21fc104a00 2024-04-25T01:25:03.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.812+0000 7f21fb400640 1 --2- 172.21.15.31:0/2750596460 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f21e00a2b90 0x7f21e00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:03.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.813+0000 7f21fb400640 1 --2- 172.21.15.31:0/2750596460 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f21e00a2b90 0x7f21e00a5050 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f21ec005e30 tx=0x7f21ec005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:03.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:03.817+0000 7f21f3e00640 1 -- 172.21.15.31:0/2750596460 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f21e409b270 con 0x7f21fc104a00 2024-04-25T01:25:03.917 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:03 smithi031 ceph-mon[28053]: pgmap v664: 97 pgs: 97 active+clean; 24 MiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T01:25:03.918 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:03 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1822972288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:25:03.918 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:03 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1433545246' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:25:04.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.057+0000 7f2200c00640 1 -- 172.21.15.31:0/2750596460 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f21fc062340 con 0x7f21fc104a00 2024-04-25T01:25:04.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.071+0000 7f21f3e00640 1 -- 172.21.15.31:0/2750596460 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) ==== 161+0+0 (secure 0 0 0) 0x7f21e409f570 con 0x7f21fc104a00 2024-04-25T01:25:04.052 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T01:25:04.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.074+0000 7f2200c00640 1 -- 172.21.15.31:0/2750596460 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f21e00a2b90 msgr2=0x7f21e00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:04.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.074+0000 7f2200c00640 1 --2- 172.21.15.31:0/2750596460 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f21e00a2b90 0x7f21e00a5050 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f21ec005e30 tx=0x7f21ec005f50 comp rx=0 tx=0).stop 2024-04-25T01:25:04.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.074+0000 7f2200c00640 1 -- 172.21.15.31:0/2750596460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc104a00 msgr2=0x7f21fc100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:04.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.074+0000 7f2200c00640 1 --2- 172.21.15.31:0/2750596460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc104a00 0x7f21fc100080 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f21e400d740 tx=0x7f21e400dc00 comp rx=0 tx=0).stop 2024-04-25T01:25:04.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.074+0000 7f2200c00640 1 -- 172.21.15.31:0/2750596460 shutdown_connections 2024-04-25T01:25:04.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.074+0000 7f2200c00640 1 --2- 172.21.15.31:0/2750596460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21fc107330 0x7f21fc1005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:04.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.074+0000 7f2200c00640 1 --2- 172.21.15.31:0/2750596460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc104a00 0x7f21fc100080 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:04.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.074+0000 7f2200c00640 1 --2- 172.21.15.31:0/2750596460 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7f21e00a2b90 0x7f21e00a5050 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:04.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.074+0000 7f2200c00640 1 --2- 172.21.15.31:0/2750596460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21fc1020d0 0x7f21fc1019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:04.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.074+0000 7f2200c00640 1 -- 172.21.15.31:0/2750596460 >> 172.21.15.31:0/2750596460 conn(0x7f21fc0fbae0 msgr2=0x7f21fc1029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:04.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.074+0000 7f2200c00640 1 -- 172.21.15.31:0/2750596460 shutdown_connections 2024-04-25T01:25:04.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.074+0000 7f2200c00640 1 -- 172.21.15.31:0/2750596460 wait complete. 2024-04-25T01:25:04.074 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T01:25:04.188 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:03 smithi169 ceph-mon[32759]: pgmap v664: 97 pgs: 97 active+clean; 24 MiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T01:25:04.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:03 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1822972288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:25:04.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:03 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1433545246' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:25:04.189 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 01:25:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-c[63241]: ignoring --setuser ceph since I am not root 2024-04-25T01:25:04.189 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 01:25:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-c[63241]: ignoring --setgroup ceph since I am not root 2024-04-25T01:25:04.189 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 01:25:04 smithi169 ceph-mds[63245]: -- 172.21.15.169:0/3769630197 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563deddb9a40 con 0x563dedd8c800 2024-04-25T01:25:04.189 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 01:25:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-c[63241]: starting mds.c at 2024-04-25T01:25:04.190 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 01:25:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-f[62933]: ignoring --setuser ceph since I am not root 2024-04-25T01:25:04.190 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 01:25:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-f[62933]: ignoring --setgroup ceph since I am not root 2024-04-25T01:25:04.190 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 01:25:04 smithi169 ceph-mds[62937]: -- 172.21.15.169:0/904344886 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5570db299a40 con 0x5570db26c800 2024-04-25T01:25:04.190 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 01:25:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-f[62933]: starting mds.f at 2024-04-25T01:25:04.190 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 01:25:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-i[63552]: ignoring --setuser ceph since I am not root 2024-04-25T01:25:04.190 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 01:25:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-i[63552]: ignoring --setgroup ceph since I am not root 2024-04-25T01:25:04.190 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 01:25:04 smithi169 ceph-mds[63556]: -- 172.21.15.169:0/2998081926 <== mon.2 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561b8c8afa40 con 0x561b8c882800 2024-04-25T01:25:04.190 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 01:25:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-i[63552]: starting mds.i at 2024-04-25T01:25:04.191 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:03 smithi040 ceph-mon[32518]: pgmap v664: 97 pgs: 97 active+clean; 24 MiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T01:25:04.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:03 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1822972288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:25:04.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:03 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1433545246' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:25:04.192 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 01:25:04 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-b[63226]: ignoring --setuser ceph since I am not root 2024-04-25T01:25:04.192 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 01:25:04 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-b[63226]: ignoring --setgroup ceph since I am not root 2024-04-25T01:25:04.192 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 01:25:04 smithi040 ceph-mds[63230]: -- 172.21.15.40:0/275241806 <== mon.2 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b5fde0ba40 con 0x55b5fddde800 2024-04-25T01:25:04.192 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 01:25:04 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-b[63226]: starting mds.b at 2024-04-25T01:25:04.192 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 01:25:04 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-h[62917]: ignoring --setuser ceph since I am not root 2024-04-25T01:25:04.193 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 01:25:04 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-h[62917]: ignoring --setgroup ceph since I am not root 2024-04-25T01:25:04.193 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 01:25:04 smithi040 ceph-mds[62921]: -- 172.21.15.40:0/710076134 <== mon.2 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5603d3af7a40 con 0x5603d3aca800 2024-04-25T01:25:04.193 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 01:25:04 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-h[62917]: starting mds.h at 2024-04-25T01:25:04.193 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 01:25:04 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-e[62606]: ignoring --setuser ceph since I am not root 2024-04-25T01:25:04.193 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 01:25:04 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-e[62606]: ignoring --setgroup ceph since I am not root 2024-04-25T01:25:04.193 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 01:25:04 smithi040 ceph-mds[62610]: -- 172.21.15.40:0/2022675103 <== mon.2 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5557e50f9a40 con 0x5557e50cc800 2024-04-25T01:25:04.193 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 01:25:04 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-e[62606]: starting mds.e at 2024-04-25T01:25:04.249 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:04 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-a[28049]: 2024-04-25T01:25:04.058+0000 7ff7b4200700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T01:25:04.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.277+0000 7ff2aa600640 1 -- 172.21.15.31:0/1493992045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a4107330 msgr2=0x7ff2a4109780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:04.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.277+0000 7ff2aa600640 1 --2- 172.21.15.31:0/1493992045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a4107330 0x7ff2a4109780 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff29c009be0 tx=0x7ff29c02a6c0 comp rx=0 tx=0).stop 2024-04-25T01:25:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.278+0000 7ff2aa600640 1 -- 172.21.15.31:0/1493992045 shutdown_connections 2024-04-25T01:25:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.278+0000 7ff2aa600640 1 --2- 172.21.15.31:0/1493992045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a4107330 0x7ff2a4109780 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.278+0000 7ff2aa600640 1 --2- 172.21.15.31:0/1493992045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2a4104a00 0x7ff2a4106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.278+0000 7ff2aa600640 1 --2- 172.21.15.31:0/1493992045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2a41020d0 0x7ff2a41044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.278+0000 7ff2aa600640 1 -- 172.21.15.31:0/1493992045 >> 172.21.15.31:0/1493992045 conn(0x7ff2a40fbae0 msgr2=0x7ff2a40fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.278+0000 7ff2aa600640 1 -- 172.21.15.31:0/1493992045 shutdown_connections 2024-04-25T01:25:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.278+0000 7ff2aa600640 1 -- 172.21.15.31:0/1493992045 wait complete. 2024-04-25T01:25:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.279+0000 7ff2aa600640 1 Processor -- start 2024-04-25T01:25:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.279+0000 7ff2aa600640 1 -- start start 2024-04-25T01:25:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.279+0000 7ff2aa600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2a41020d0 0x7ff2a41978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.279+0000 7ff2aa600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a4104a00 0x7ff2a4197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.279+0000 7ff2aa600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2a4107330 0x7ff2a419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.279+0000 7ff2aa600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a410d2f0 con 0x7ff2a4107330 2024-04-25T01:25:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.279+0000 7ff2aa600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a410d170 con 0x7ff2a4104a00 2024-04-25T01:25:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.279+0000 7ff2aa600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a410d470 con 0x7ff2a41020d0 2024-04-25T01:25:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.279+0000 7ff2a9200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2a41020d0 0x7ff2a41978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.280+0000 7ff2a9c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2a4107330 0x7ff2a419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.280+0000 7ff2a3e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a4104a00 0x7ff2a4197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.280+0000 7ff2a9200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2a41020d0 0x7ff2a41978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.31:59056/0 (socket says 172.21.15.31:59056) 2024-04-25T01:25:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.280+0000 7ff2a9200640 1 -- 172.21.15.31:0/2116151112 learned_addr learned my addr 172.21.15.31:0/2116151112 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:25:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.280+0000 7ff2a3e00640 1 -- 172.21.15.31:0/2116151112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2a41020d0 msgr2=0x7ff2a41978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.280+0000 7ff2a3e00640 1 --2- 172.21.15.31:0/2116151112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2a41020d0 0x7ff2a41978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.280+0000 7ff2a3e00640 1 -- 172.21.15.31:0/2116151112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2a4107330 msgr2=0x7ff2a419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.280+0000 7ff2a3e00640 1 --2- 172.21.15.31:0/2116151112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2a4107330 0x7ff2a419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.280+0000 7ff2a3e00640 1 -- 172.21.15.31:0/2116151112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a419f500 con 0x7ff2a4104a00 2024-04-25T01:25:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.280+0000 7ff2a3e00640 1 --2- 172.21.15.31:0/2116151112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a4104a00 0x7ff2a4197e20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4069430 tx=0x7ff29000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:04.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.280+0000 7ff2a1600640 1 -- 172.21.15.31:0/2116151112 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29000ccd0 con 0x7ff2a4104a00 2024-04-25T01:25:04.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.280+0000 7ff2a1600640 1 -- 172.21.15.31:0/2116151112 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2900035f0 con 0x7ff2a4104a00 2024-04-25T01:25:04.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.281+0000 7ff2a1600640 1 -- 172.21.15.31:0/2116151112 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff290010670 con 0x7ff2a4104a00 2024-04-25T01:25:04.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.281+0000 7ff2aa600640 1 -- 172.21.15.31:0/2116151112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a419f7f0 con 0x7ff2a4104a00 2024-04-25T01:25:04.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.281+0000 7ff2aa600640 1 -- 172.21.15.31:0/2116151112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a419fd40 con 0x7ff2a4104a00 2024-04-25T01:25:04.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.282+0000 7ff2aa600640 1 -- 172.21.15.31:0/2116151112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2a40fd5d0 con 0x7ff2a4104a00 2024-04-25T01:25:04.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.284+0000 7ff2a1600640 1 -- 172.21.15.31:0/2116151112 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff2900037a0 con 0x7ff2a4104a00 2024-04-25T01:25:04.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.287+0000 7ff2a1600640 1 --2- 172.21.15.31:0/2116151112 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff28c0a2b90 0x7ff28c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:04.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.287+0000 7ff2a1600640 1 -- 172.21.15.31:0/2116151112 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8419+0+0 (secure 0 0 0) 0x7ff2900cf2f0 con 0x7ff2a4104a00 2024-04-25T01:25:04.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.287+0000 7ff2a9200640 1 --2- 172.21.15.31:0/2116151112 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff28c0a2b90 0x7ff28c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:04.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.288+0000 7ff2a9200640 1 --2- 172.21.15.31:0/2116151112 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff28c0a2b90 0x7ff28c0a5050 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff298009cd0 tx=0x7ff298005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:04.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.290+0000 7ff2a1600640 1 -- 172.21.15.31:0/2116151112 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff290009510 con 0x7ff2a4104a00 2024-04-25T01:25:04.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:04.535+0000 7ff2aa600640 1 -- 172.21.15.31:0/2116151112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff2a4062340 con 0x7ff2a4104a00 2024-04-25T01:25:05.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.067+0000 7ff2a1600640 1 -- 172.21.15.31:0/2116151112 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) ==== 104+0+0 (secure 0 0 0) 0x7ff29009c130 con 0x7ff2a4104a00 2024-04-25T01:25:05.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.071+0000 7ff2aa600640 1 -- 172.21.15.31:0/2116151112 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff28c0a2b90 msgr2=0x7ff28c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:05.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.071+0000 7ff2aa600640 1 --2- 172.21.15.31:0/2116151112 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff28c0a2b90 0x7ff28c0a5050 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff298009cd0 tx=0x7ff298005f90 comp rx=0 tx=0).stop 2024-04-25T01:25:05.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.072+0000 7ff2aa600640 1 -- 172.21.15.31:0/2116151112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a4104a00 msgr2=0x7ff2a4197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:05.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.072+0000 7ff2aa600640 1 --2- 172.21.15.31:0/2116151112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a4104a00 0x7ff2a4197e20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4069430 tx=0x7ff29000ee50 comp rx=0 tx=0).stop 2024-04-25T01:25:05.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.072+0000 7ff2aa600640 1 -- 172.21.15.31:0/2116151112 shutdown_connections 2024-04-25T01:25:05.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.072+0000 7ff2aa600640 1 --2- 172.21.15.31:0/2116151112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2a4107330 0x7ff2a419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:05.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.072+0000 7ff2aa600640 1 --2- 172.21.15.31:0/2116151112 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7ff28c0a2b90 0x7ff28c0a5050 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:05.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.072+0000 7ff2aa600640 1 --2- 172.21.15.31:0/2116151112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2a4104a00 0x7ff2a4197e20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:05.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.072+0000 7ff2aa600640 1 --2- 172.21.15.31:0/2116151112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2a41020d0 0x7ff2a41978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:05.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.072+0000 7ff2aa600640 1 -- 172.21.15.31:0/2116151112 >> 172.21.15.31:0/2116151112 conn(0x7ff2a40fbae0 msgr2=0x7ff2a4102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:05.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.072+0000 7ff2aa600640 1 -- 172.21.15.31:0/2116151112 shutdown_connections 2024-04-25T01:25:05.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.072+0000 7ff2aa600640 1 -- 172.21.15.31:0/2116151112 wait complete. 2024-04-25T01:25:05.069 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:04 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2750596460' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T01:25:05.069 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:04 smithi031 ceph-mon[28053]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T01:25:05.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:04 smithi031 ceph-mon[28053]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T01:25:05.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:04 smithi031 ceph-mon[28053]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T01:25:05.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:04 smithi031 ceph-mon[28053]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T01:25:05.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:04 smithi031 ceph-mon[28053]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T01:25:05.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:04 smithi031 ceph-mon[28053]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T01:25:05.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:04 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/2116151112' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T01:25:05.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:04 smithi031 ceph-mon[28053]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T01:25:05.072 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T01:25:05.188 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:04 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2750596460' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T01:25:05.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:04 smithi169 ceph-mon[32759]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T01:25:05.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:04 smithi169 ceph-mon[32759]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T01:25:05.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:04 smithi169 ceph-mon[32759]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T01:25:05.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:04 smithi169 ceph-mon[32759]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T01:25:05.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:04 smithi169 ceph-mon[32759]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T01:25:05.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:04 smithi169 ceph-mon[32759]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T01:25:05.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:04 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/2116151112' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T01:25:05.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:04 smithi169 ceph-mon[32759]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T01:25:05.191 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:04 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2750596460' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T01:25:05.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:04 smithi040 ceph-mon[32518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T01:25:05.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:04 smithi040 ceph-mon[32518]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T01:25:05.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:04 smithi040 ceph-mon[32518]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T01:25:05.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:04 smithi040 ceph-mon[32518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T01:25:05.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:04 smithi040 ceph-mon[32518]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T01:25:05.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:04 smithi040 ceph-mon[32518]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T01:25:05.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:04 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/2116151112' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T01:25:05.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:04 smithi040 ceph-mon[32518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T01:25:05.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.266+0000 7fee52800640 1 -- 172.21.15.31:0/216613284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee4c0ffec0 msgr2=0x7fee4c1022f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:05.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.266+0000 7fee52800640 1 --2- 172.21.15.31:0/216613284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee4c0ffec0 0x7fee4c1022f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fee40009860 tx=0x7fee4002a340 comp rx=0 tx=0).stop 2024-04-25T01:25:05.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.266+0000 7fee52800640 1 -- 172.21.15.31:0/216613284 shutdown_connections 2024-04-25T01:25:05.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.266+0000 7fee52800640 1 --2- 172.21.15.31:0/216613284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee4c1051e0 0x7fee4c105660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:05.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.266+0000 7fee52800640 1 --2- 172.21.15.31:0/216613284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee4c102830 0x7fee4c104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:05.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.266+0000 7fee52800640 1 --2- 172.21.15.31:0/216613284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee4c0ffec0 0x7fee4c1022f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:05.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.266+0000 7fee52800640 1 -- 172.21.15.31:0/216613284 >> 172.21.15.31:0/216613284 conn(0x7fee4c0fbae0 msgr2=0x7fee4c0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:05.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.266+0000 7fee52800640 1 -- 172.21.15.31:0/216613284 shutdown_connections 2024-04-25T01:25:05.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.266+0000 7fee52800640 1 -- 172.21.15.31:0/216613284 wait complete. 2024-04-25T01:25:05.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.267+0000 7fee52800640 1 Processor -- start 2024-04-25T01:25:05.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.267+0000 7fee52800640 1 -- start start 2024-04-25T01:25:05.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.268+0000 7fee52800640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee4c0ffec0 0x7fee4c19df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:05.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.268+0000 7fee52800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee4c102830 0x7fee4c19e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:05.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.268+0000 7fee52800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee4c1051e0 0x7fee4c1a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:05.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.268+0000 7fee52800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c1138a0 con 0x7fee4c102830 2024-04-25T01:25:05.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.268+0000 7fee52800640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c113720 con 0x7fee4c0ffec0 2024-04-25T01:25:05.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.268+0000 7fee52800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c113a20 con 0x7fee4c1051e0 2024-04-25T01:25:05.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.268+0000 7fee51400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee4c0ffec0 0x7fee4c19df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:05.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.268+0000 7fee50a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee4c102830 0x7fee4c19e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:05.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.268+0000 7fee51400640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee4c0ffec0 0x7fee4c19df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:57648/0 (socket says 172.21.15.31:57648) 2024-04-25T01:25:05.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.268+0000 7fee51400640 1 -- 172.21.15.31:0/3206061910 learned_addr learned my addr 172.21.15.31:0/3206061910 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:25:05.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.268+0000 7fee51e00640 1 --2- 172.21.15.31:0/3206061910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee4c1051e0 0x7fee4c1a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:05.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.268+0000 7fee51400640 1 -- 172.21.15.31:0/3206061910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee4c1051e0 msgr2=0x7fee4c1a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:05.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.268+0000 7fee51400640 1 --2- 172.21.15.31:0/3206061910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee4c1051e0 0x7fee4c1a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:05.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.269+0000 7fee51400640 1 -- 172.21.15.31:0/3206061910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee4c102830 msgr2=0x7fee4c19e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:05.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.269+0000 7fee51400640 1 --2- 172.21.15.31:0/3206061910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee4c102830 0x7fee4c19e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:05.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.269+0000 7fee51400640 1 -- 172.21.15.31:0/3206061910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4c1a5b30 con 0x7fee4c0ffec0 2024-04-25T01:25:05.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.269+0000 7fee51400640 1 --2- 172.21.15.31:0/3206061910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee4c0ffec0 0x7fee4c19df10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fee40005940 tx=0x7fee40033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:05.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.269+0000 7fee4a000640 1 -- 172.21.15.31:0/3206061910 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4003f020 con 0x7fee4c0ffec0 2024-04-25T01:25:05.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.269+0000 7fee4a000640 1 -- 172.21.15.31:0/3206061910 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee40030850 con 0x7fee4c0ffec0 2024-04-25T01:25:05.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.269+0000 7fee52800640 1 -- 172.21.15.31:0/3206061910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee4c1a5dc0 con 0x7fee4c0ffec0 2024-04-25T01:25:05.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.269+0000 7fee4a000640 1 -- 172.21.15.31:0/3206061910 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee40037040 con 0x7fee4c0ffec0 2024-04-25T01:25:05.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.269+0000 7fee52800640 1 -- 172.21.15.31:0/3206061910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee4c1a61f0 con 0x7fee4c0ffec0 2024-04-25T01:25:05.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.271+0000 7fee52800640 1 -- 172.21.15.31:0/3206061910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee4c0fd5d0 con 0x7fee4c0ffec0 2024-04-25T01:25:05.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.272+0000 7fee4a000640 1 -- 172.21.15.31:0/3206061910 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fee40042020 con 0x7fee4c0ffec0 2024-04-25T01:25:05.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.273+0000 7fee4a000640 1 --2- 172.21.15.31:0/3206061910 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fee340a2b90 0x7fee340a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:05.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.273+0000 7fee4a000640 1 -- 172.21.15.31:0/3206061910 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8419+0+0 (secure 0 0 0) 0x7fee400ef440 con 0x7fee4c0ffec0 2024-04-25T01:25:05.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.274+0000 7fee50a00640 1 --2- 172.21.15.31:0/3206061910 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fee340a2b90 0x7fee340a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:05.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.274+0000 7fee50a00640 1 --2- 172.21.15.31:0/3206061910 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fee340a2b90 0x7fee340a5050 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fee4c19f420 tx=0x7fee38009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.278+0000 7fee4a000640 1 -- 172.21.15.31:0/3206061910 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fee40049020 con 0x7fee4c0ffec0 2024-04-25T01:25:05.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:05.473+0000 7fee52800640 1 -- 172.21.15.31:0/3206061910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fee4c062340 con 0x7fee4c0ffec0 2024-04-25T01:25:06.063 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2024-04-25T01:25:06.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.081+0000 7fee4a000640 1 -- 172.21.15.31:0/3206061910 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) ==== 181+0+0 (secure 0 0 0) 0x7fee400bc280 con 0x7fee4c0ffec0 2024-04-25T01:25:06.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.084+0000 7fee52800640 1 -- 172.21.15.31:0/3206061910 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fee340a2b90 msgr2=0x7fee340a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:06.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.084+0000 7fee52800640 1 --2- 172.21.15.31:0/3206061910 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fee340a2b90 0x7fee340a5050 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fee4c19f420 tx=0x7fee38009260 comp rx=0 tx=0).stop 2024-04-25T01:25:06.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.085+0000 7fee52800640 1 -- 172.21.15.31:0/3206061910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee4c0ffec0 msgr2=0x7fee4c19df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:06.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.085+0000 7fee52800640 1 --2- 172.21.15.31:0/3206061910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee4c0ffec0 0x7fee4c19df10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fee40005940 tx=0x7fee40033d90 comp rx=0 tx=0).stop 2024-04-25T01:25:06.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.085+0000 7fee52800640 1 -- 172.21.15.31:0/3206061910 shutdown_connections 2024-04-25T01:25:06.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.085+0000 7fee52800640 1 --2- 172.21.15.31:0/3206061910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee4c1051e0 0x7fee4c1a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:06.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.085+0000 7fee52800640 1 --2- 172.21.15.31:0/3206061910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee4c102830 0x7fee4c19e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:06.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.085+0000 7fee52800640 1 --2- 172.21.15.31:0/3206061910 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fee340a2b90 0x7fee340a5050 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:06.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.085+0000 7fee52800640 1 --2- 172.21.15.31:0/3206061910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee4c0ffec0 0x7fee4c19df10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:06.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.085+0000 7fee52800640 1 -- 172.21.15.31:0/3206061910 >> 172.21.15.31:0/3206061910 conn(0x7fee4c0fbae0 msgr2=0x7fee4c1031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:06.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.085+0000 7fee52800640 1 -- 172.21.15.31:0/3206061910 shutdown_connections 2024-04-25T01:25:06.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.085+0000 7fee52800640 1 -- 172.21.15.31:0/3206061910 wait complete. 2024-04-25T01:25:06.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:05 smithi031 ceph-mon[28053]: pgmap v666: 97 pgs: 97 active+clean; 24 MiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-25T01:25:06.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:05 smithi031 ceph-mon[28053]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T01:25:06.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:05 smithi031 ceph-mon[28053]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T01:25:06.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:05 smithi031 ceph-mon[28053]: Cluster is now healthy 2024-04-25T01:25:06.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:05 smithi031 ceph-mon[28053]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T01:25:06.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:05 smithi031 ceph-mon[28053]: mds.? [v2:172.21.15.40:6834/413421286,v1:172.21.15.40:6835/413421286] up:boot 2024-04-25T01:25:06.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:05 smithi031 ceph-mon[28053]: mds.? [v2:172.21.15.169:6834/582127686,v1:172.21.15.169:6837/582127686] up:boot 2024-04-25T01:25:06.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:05 smithi031 ceph-mon[28053]: mds.? [v2:172.21.15.169:6835/3639221628,v1:172.21.15.169:6838/3639221628] up:boot 2024-04-25T01:25:06.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:05 smithi031 ceph-mon[28053]: mds.? [v2:172.21.15.169:6836/704924249,v1:172.21.15.169:6839/704924249] up:boot 2024-04-25T01:25:06.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:05 smithi031 ceph-mon[28053]: mds.? [v2:172.21.15.40:6836/1604723537,v1:172.21.15.40:6838/1604723537] up:boot 2024-04-25T01:25:06.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:05 smithi031 ceph-mon[28053]: mds.? [v2:172.21.15.40:6837/1211410971,v1:172.21.15.40:6839/1211410971] up:boot 2024-04-25T01:25:06.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:05 smithi031 ceph-mon[28053]: fsmap 12 up:standby 2024-04-25T01:25:06.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:05 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T01:25:06.088 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:05 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T01:25:06.088 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:05 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T01:25:06.088 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:05 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T01:25:06.088 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:05 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T01:25:06.088 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:05 smithi031 ceph-mon[28053]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T01:25:06.088 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:05 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/3206061910' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T01:25:06.088 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:05 smithi031 ceph-mon[28053]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T01:25:06.090 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T01:25:06.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:05 smithi169 ceph-mon[32759]: pgmap v666: 97 pgs: 97 active+clean; 24 MiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-25T01:25:06.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:05 smithi169 ceph-mon[32759]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T01:25:06.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:05 smithi169 ceph-mon[32759]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T01:25:06.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:05 smithi169 ceph-mon[32759]: Cluster is now healthy 2024-04-25T01:25:06.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:05 smithi169 ceph-mon[32759]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T01:25:06.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:05 smithi169 ceph-mon[32759]: mds.? [v2:172.21.15.40:6834/413421286,v1:172.21.15.40:6835/413421286] up:boot 2024-04-25T01:25:06.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:05 smithi169 ceph-mon[32759]: mds.? [v2:172.21.15.169:6834/582127686,v1:172.21.15.169:6837/582127686] up:boot 2024-04-25T01:25:06.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:05 smithi169 ceph-mon[32759]: mds.? [v2:172.21.15.169:6835/3639221628,v1:172.21.15.169:6838/3639221628] up:boot 2024-04-25T01:25:06.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:05 smithi169 ceph-mon[32759]: mds.? [v2:172.21.15.169:6836/704924249,v1:172.21.15.169:6839/704924249] up:boot 2024-04-25T01:25:06.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:05 smithi169 ceph-mon[32759]: mds.? [v2:172.21.15.40:6836/1604723537,v1:172.21.15.40:6838/1604723537] up:boot 2024-04-25T01:25:06.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:05 smithi169 ceph-mon[32759]: mds.? [v2:172.21.15.40:6837/1211410971,v1:172.21.15.40:6839/1211410971] up:boot 2024-04-25T01:25:06.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:05 smithi169 ceph-mon[32759]: fsmap 12 up:standby 2024-04-25T01:25:06.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:05 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T01:25:06.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:05 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T01:25:06.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:05 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T01:25:06.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:05 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T01:25:06.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:05 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T01:25:06.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:05 smithi169 ceph-mon[32759]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T01:25:06.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:05 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/3206061910' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T01:25:06.190 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:05 smithi169 ceph-mon[32759]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T01:25:06.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:05 smithi040 ceph-mon[32518]: pgmap v666: 97 pgs: 97 active+clean; 24 MiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-25T01:25:06.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:05 smithi040 ceph-mon[32518]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T01:25:06.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:05 smithi040 ceph-mon[32518]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T01:25:06.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:05 smithi040 ceph-mon[32518]: Cluster is now healthy 2024-04-25T01:25:06.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:05 smithi040 ceph-mon[32518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T01:25:06.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:05 smithi040 ceph-mon[32518]: mds.? [v2:172.21.15.40:6834/413421286,v1:172.21.15.40:6835/413421286] up:boot 2024-04-25T01:25:06.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:05 smithi040 ceph-mon[32518]: mds.? [v2:172.21.15.169:6834/582127686,v1:172.21.15.169:6837/582127686] up:boot 2024-04-25T01:25:06.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:05 smithi040 ceph-mon[32518]: mds.? [v2:172.21.15.169:6835/3639221628,v1:172.21.15.169:6838/3639221628] up:boot 2024-04-25T01:25:06.193 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:05 smithi040 ceph-mon[32518]: mds.? [v2:172.21.15.169:6836/704924249,v1:172.21.15.169:6839/704924249] up:boot 2024-04-25T01:25:06.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:05 smithi040 ceph-mon[32518]: mds.? [v2:172.21.15.40:6836/1604723537,v1:172.21.15.40:6838/1604723537] up:boot 2024-04-25T01:25:06.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:05 smithi040 ceph-mon[32518]: mds.? [v2:172.21.15.40:6837/1211410971,v1:172.21.15.40:6839/1211410971] up:boot 2024-04-25T01:25:06.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:05 smithi040 ceph-mon[32518]: fsmap 12 up:standby 2024-04-25T01:25:06.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:05 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T01:25:06.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:05 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T01:25:06.194 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:05 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T01:25:06.195 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:05 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T01:25:06.195 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:05 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T01:25:06.195 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:05 smithi040 ceph-mon[32518]: from='mgr.14152 172.21.15.31:0/2109879347' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T01:25:06.195 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:05 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/3206061910' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T01:25:06.195 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:05 smithi040 ceph-mon[32518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T01:25:06.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.294+0000 7fa891e00640 1 -- 172.21.15.31:0/1630695207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88c1020d0 msgr2=0x7fa88c1044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:06.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.294+0000 7fa891e00640 1 --2- 172.21.15.31:0/1630695207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88c1020d0 0x7fa88c1044c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa870009900 tx=0x7fa87002a400 comp rx=0 tx=0).stop 2024-04-25T01:25:06.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.294+0000 7fa891e00640 1 -- 172.21.15.31:0/1630695207 shutdown_connections 2024-04-25T01:25:06.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.294+0000 7fa891e00640 1 --2- 172.21.15.31:0/1630695207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa88c107330 0x7fa88c109780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:06.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.294+0000 7fa891e00640 1 --2- 172.21.15.31:0/1630695207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa88c104a00 0x7fa88c106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:06.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.294+0000 7fa891e00640 1 --2- 172.21.15.31:0/1630695207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88c1020d0 0x7fa88c1044c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:06.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.294+0000 7fa891e00640 1 -- 172.21.15.31:0/1630695207 >> 172.21.15.31:0/1630695207 conn(0x7fa88c0fbae0 msgr2=0x7fa88c0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:06.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.294+0000 7fa891e00640 1 -- 172.21.15.31:0/1630695207 shutdown_connections 2024-04-25T01:25:06.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.294+0000 7fa891e00640 1 -- 172.21.15.31:0/1630695207 wait complete. 2024-04-25T01:25:06.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.295+0000 7fa891e00640 1 Processor -- start 2024-04-25T01:25:06.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.295+0000 7fa891e00640 1 -- start start 2024-04-25T01:25:06.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.296+0000 7fa891e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa88c1020d0 0x7fa88c101a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:06.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.296+0000 7fa891e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88c104a00 0x7fa88c100080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:06.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.296+0000 7fa891e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa88c107330 0x7fa88c1005c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:06.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.296+0000 7fa891e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c10d320 con 0x7fa88c1020d0 2024-04-25T01:25:06.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.296+0000 7fa891e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c10d1a0 con 0x7fa88c104a00 2024-04-25T01:25:06.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.296+0000 7fa891e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c10d4a0 con 0x7fa88c107330 2024-04-25T01:25:06.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.296+0000 7fa890a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa88c1020d0 0x7fa88c101a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:06.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.296+0000 7fa890a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa88c1020d0 0x7fa88c101a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60632/0 (socket says 172.21.15.31:60632) 2024-04-25T01:25:06.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.296+0000 7fa890a00640 1 -- 172.21.15.31:0/1569978440 learned_addr learned my addr 172.21.15.31:0/1569978440 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T01:25:06.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.296+0000 7fa883e00640 1 --2- 172.21.15.31:0/1569978440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88c104a00 0x7fa88c100080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:06.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.296+0000 7fa891400640 1 --2- 172.21.15.31:0/1569978440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa88c107330 0x7fa88c1005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:06.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.297+0000 7fa883e00640 1 -- 172.21.15.31:0/1569978440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa88c107330 msgr2=0x7fa88c1005c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:06.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.297+0000 7fa883e00640 1 --2- 172.21.15.31:0/1569978440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa88c107330 0x7fa88c1005c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:06.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.297+0000 7fa883e00640 1 -- 172.21.15.31:0/1569978440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa88c1020d0 msgr2=0x7fa88c101a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:06.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.297+0000 7fa883e00640 1 --2- 172.21.15.31:0/1569978440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa88c1020d0 0x7fa88c101a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:06.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.297+0000 7fa883e00640 1 -- 172.21.15.31:0/1569978440 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88c100e50 con 0x7fa88c104a00 2024-04-25T01:25:06.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.297+0000 7fa890a00640 1 --2- 172.21.15.31:0/1569978440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa88c1020d0 0x7fa88c101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:25:06.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.297+0000 7fa883e00640 1 --2- 172.21.15.31:0/1569978440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88c104a00 0x7fa88c100080 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa87800e9c0 tx=0x7fa87800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:06.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.297+0000 7fa88a000640 1 -- 172.21.15.31:0/1569978440 <== mon.2 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87800cd80 con 0x7fa88c104a00 2024-04-25T01:25:06.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.297+0000 7fa88a000640 1 -- 172.21.15.31:0/1569978440 <== mon.2 v2:172.21.15.40:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8780035f0 con 0x7fa88c104a00 2024-04-25T01:25:06.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.297+0000 7fa891e00640 1 -- 172.21.15.31:0/1569978440 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa88c1a1a90 con 0x7fa88c104a00 2024-04-25T01:25:06.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.297+0000 7fa88a000640 1 -- 172.21.15.31:0/1569978440 <== mon.2 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878010690 con 0x7fa88c104a00 2024-04-25T01:25:06.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.297+0000 7fa891e00640 1 -- 172.21.15.31:0/1569978440 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa88c1a1fe0 con 0x7fa88c104a00 2024-04-25T01:25:06.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.298+0000 7fa891e00640 1 -- 172.21.15.31:0/1569978440 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa88c0fd5d0 con 0x7fa88c104a00 2024-04-25T01:25:06.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.300+0000 7fa88a000640 1 -- 172.21.15.31:0/1569978440 <== mon.2 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa8780037a0 con 0x7fa88c104a00 2024-04-25T01:25:06.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.302+0000 7fa88a000640 1 --2- 172.21.15.31:0/1569978440 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa8680a2b90 0x7fa8680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:25:06.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.303+0000 7fa88a000640 1 -- 172.21.15.31:0/1569978440 <== mon.2 v2:172.21.15.40:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 7915+0+0 (secure 0 0 0) 0x7fa8780cf260 con 0x7fa88c104a00 2024-04-25T01:25:06.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.303+0000 7fa890a00640 1 --2- 172.21.15.31:0/1569978440 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa8680a2b90 0x7fa8680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:25:06.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.303+0000 7fa890a00640 1 --2- 172.21.15.31:0/1569978440 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa8680a2b90 0x7fa8680a5050 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa8700096c0 tx=0x7fa8700315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:25:06.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.305+0000 7fa88a000640 1 -- 172.21.15.31:0/1569978440 <== mon.2 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa87809c2b0 con 0x7fa88c104a00 2024-04-25T01:25:06.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:06.514+0000 7fa891e00640 1 -- 172.21.15.31:0/1569978440 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa88c062340 con 0x7fa88c104a00 2024-04-25T01:25:07.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:07.106+0000 7fa88a000640 1 -- 172.21.15.31:0/1569978440 <== mon.2 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) ==== 169+0+0 (secure 0 0 0) 0x7fa87809bbb0 con 0x7fa88c104a00 2024-04-25T01:25:07.087 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2024-04-25T01:25:07.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:07.114+0000 7fa891e00640 1 -- 172.21.15.31:0/1569978440 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa8680a2b90 msgr2=0x7fa8680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:07.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:07.114+0000 7fa891e00640 1 --2- 172.21.15.31:0/1569978440 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa8680a2b90 0x7fa8680a5050 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa8700096c0 tx=0x7fa8700315b0 comp rx=0 tx=0).stop 2024-04-25T01:25:07.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:07.114+0000 7fa891e00640 1 -- 172.21.15.31:0/1569978440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88c104a00 msgr2=0x7fa88c100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:25:07.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:07.114+0000 7fa891e00640 1 --2- 172.21.15.31:0/1569978440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88c104a00 0x7fa88c100080 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa87800e9c0 tx=0x7fa87800ee80 comp rx=0 tx=0).stop 2024-04-25T01:25:07.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:07.114+0000 7fa891e00640 1 -- 172.21.15.31:0/1569978440 shutdown_connections 2024-04-25T01:25:07.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:07.114+0000 7fa891e00640 1 --2- 172.21.15.31:0/1569978440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa88c107330 0x7fa88c1005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:07.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:07.114+0000 7fa891e00640 1 --2- 172.21.15.31:0/1569978440 >> [v2:172.21.15.31:6800/2874594057,v1:172.21.15.31:6801/2874594057] conn(0x7fa8680a2b90 0x7fa8680a5050 secure :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa8700096c0 tx=0x7fa8700315b0 comp rx=0 tx=0).stop 2024-04-25T01:25:07.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:07.114+0000 7fa891e00640 1 --2- 172.21.15.31:0/1569978440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88c104a00 0x7fa88c100080 secure :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa87800e9c0 tx=0x7fa87800ee80 comp rx=0 tx=0).stop 2024-04-25T01:25:07.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:07.114+0000 7fa891e00640 1 --2- 172.21.15.31:0/1569978440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa88c1020d0 0x7fa88c101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:25:07.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:07.114+0000 7fa891e00640 1 -- 172.21.15.31:0/1569978440 >> 172.21.15.31:0/1569978440 conn(0x7fa88c0fbae0 msgr2=0x7fa88c102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:25:07.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:07.115+0000 7fa891e00640 1 -- 172.21.15.31:0/1569978440 shutdown_connections 2024-04-25T01:25:07.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T01:25:07.116+0000 7fa891e00640 1 -- 172.21.15.31:0/1569978440 wait complete. 2024-04-25T01:25:07.124 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T01:25:07.218 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T01:25:07.249 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T01:25:07.284 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T01:25:07.284 DEBUG:teuthology.orchestra.run.smithi031:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T01:25:07.330 DEBUG:teuthology.orchestra.run.smithi040:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T01:25:07.345 DEBUG:teuthology.orchestra.run.smithi169:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T01:25:07.360 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T01:25:07.361 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T01:25:07.361 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@mon.a 2024-04-25T01:25:07.433 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:07 smithi031 ceph-mon[28053]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T01:25:07.433 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:07 smithi031 ceph-mon[28053]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T01:25:07.433 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:07 smithi031 ceph-mon[28053]: from='client.? 172.21.15.31:0/1569978440' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T01:25:07.433 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:07 smithi031 ceph-mon[28053]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T01:25:07.438 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:07 smithi169 ceph-mon[32759]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T01:25:07.438 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:07 smithi169 ceph-mon[32759]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T01:25:07.439 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:07 smithi169 ceph-mon[32759]: from='client.? 172.21.15.31:0/1569978440' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T01:25:07.439 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:07 smithi169 ceph-mon[32759]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T01:25:07.441 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:07 smithi040 ceph-mon[32518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T01:25:07.441 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:07 smithi040 ceph-mon[32518]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T01:25:07.442 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:07 smithi040 ceph-mon[32518]: from='client.? 172.21.15.31:0/1569978440' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T01:25:07.442 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:07 smithi040 ceph-mon[32518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T01:25:07.749 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:07 smithi031 systemd[1]: Stopping Ceph mon.a for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:25:08.425 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:08 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-a[28049]: 2024-04-25T01:25:08.018+0000 7ff7bb000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:25:08.425 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:08 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-a[28049]: 2024-04-25T01:25:08.018+0000 7ff7bb000700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T01:25:08.425 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:08 smithi031 podman[87693]: 2024-04-25 01:25:08.081427739 +0000 UTC m=+0.241269778 container died b4a98098cf56696d4aed04fea09585ba196427b8ceae955658abde5b734336fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True) 2024-04-25T01:25:08.749 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:08 smithi031 podman[87693]: 2024-04-25 01:25:08.577017248 +0000 UTC m=+0.736859320 container cleanup b4a98098cf56696d4aed04fea09585ba196427b8ceae955658abde5b734336fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T01:25:08.749 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:08 smithi031 bash[87693]: ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-a 2024-04-25T01:25:09.162 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 01:25:08 smithi031 podman[87704]: 2024-04-25 01:25:08.799707226 +0000 UTC m=+0.703598914 container remove b4a98098cf56696d4aed04fea09585ba196427b8ceae955658abde5b734336fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:25:09.183 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T01:25:09.287 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:25:09.287 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T01:25:09.287 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T01:25:09.287 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@mon.b 2024-04-25T01:25:09.635 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:09 smithi040 systemd[1]: Stopping Ceph mon.b for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:25:09.905 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:09 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-b[32510]: 2024-04-25T01:25:09.655+0000 7f892751f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:25:09.905 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:09 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-b[32510]: 2024-04-25T01:25:09.655+0000 7f892751f700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T01:25:09.905 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:09 smithi040 podman[67257]: 2024-04-25 01:25:09.706775458 +0000 UTC m=+0.211433068 container died d9a4e55c713c52f1cc6917a4ab9d9e9b83387ff983a494e9a9ba73a56ca5a75f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9) 2024-04-25T01:25:10.191 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:10 smithi040 podman[67257]: 2024-04-25 01:25:10.015999274 +0000 UTC m=+0.520656881 container cleanup d9a4e55c713c52f1cc6917a4ab9d9e9b83387ff983a494e9a9ba73a56ca5a75f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-b, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-25T01:25:10.192 INFO:journalctl@ceph.mon.b.smithi040.stdout:Apr 25 01:25:10 smithi040 bash[67257]: ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-b 2024-04-25T01:25:10.352 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T01:25:10.401 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:25:10.401 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T01:25:10.401 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T01:25:10.402 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@mon.c 2024-04-25T01:25:10.688 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:10 smithi169 systemd[1]: Stopping Ceph mon.c for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:25:10.966 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:10 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-c[32741]: 2024-04-25T01:25:10.709+0000 7f446f547700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:25:10.966 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:10 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-c[32741]: 2024-04-25T01:25:10.709+0000 7f446f547700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T01:25:10.966 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:10 smithi169 podman[67429]: 2024-04-25 01:25:10.792985253 +0000 UTC m=+0.174279073 container died 1dac5730d76111be6cdc562dc47962f948ce3421a07a19c27d4df2ad293be3fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:25:11.300 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:11 smithi169 podman[67429]: 2024-04-25 01:25:11.069363861 +0000 UTC m=+0.450657668 container cleanup 1dac5730d76111be6cdc562dc47962f948ce3421a07a19c27d4df2ad293be3fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:25:11.300 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 01:25:11 smithi169 bash[67429]: ceph-95849b94-029f-11ef-bc93-c7b262605968-mon-c 2024-04-25T01:25:11.478 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T01:25:11.524 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:25:11.524 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T01:25:11.525 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T01:25:11.525 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@mgr.x 2024-04-25T01:25:11.999 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:25:11 smithi031 systemd[1]: Stopping Ceph mgr.x for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:25:12.419 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:25:12 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:25:12.064+0000 7fdab9000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:25:12.419 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:25:12 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-25T01:25:12.064+0000 7fdab9000700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-25T01:25:12.419 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:25:12 smithi031 podman[87811]: 2024-04-25 01:25:12.158888325 +0000 UTC m=+0.224302547 container died f54adae2952fdce4d53f5a18ad70c7cfb5ff481d3813ae338b2f20794c921f4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9) 2024-04-25T01:25:12.749 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:25:12 smithi031 podman[87811]: 2024-04-25 01:25:12.614921985 +0000 UTC m=+0.680336222 container cleanup f54adae2952fdce4d53f5a18ad70c7cfb5ff481d3813ae338b2f20794c921f4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-d2c45d9, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T01:25:12.749 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:25:12 smithi031 bash[87811]: ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x 2024-04-25T01:25:13.172 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 01:25:12 smithi031 podman[87825]: 2024-04-25 01:25:12.782296448 +0000 UTC m=+0.605361305 container remove f54adae2952fdce4d53f5a18ad70c7cfb5ff481d3813ae338b2f20794c921f4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-x, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:25:13.197 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T01:25:13.304 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:25:13.304 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T01:25:13.304 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T01:25:13.304 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@mgr.y 2024-04-25T01:25:13.621 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:25:13 smithi040 systemd[1]: Stopping Ceph mgr.y for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:25:13.941 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:25:13 smithi040 podman[67381]: 2024-04-25 01:25:13.641653063 +0000 UTC m=+0.131909733 container died e8346f709fab7619eb6505e9debb4390d9d6eeddab0d5ac2fbf865bcce720251 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-25T01:25:14.384 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:25:14 smithi040 podman[67381]: 2024-04-25 01:25:14.028948661 +0000 UTC m=+0.519205330 container cleanup e8346f709fab7619eb6505e9debb4390d9d6eeddab0d5ac2fbf865bcce720251 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux ) 2024-04-25T01:25:14.384 INFO:journalctl@ceph.mgr.y.smithi040.stdout:Apr 25 01:25:14 smithi040 bash[67381]: ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-y 2024-04-25T01:25:14.529 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T01:25:14.563 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:25:14.563 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T01:25:14.563 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T01:25:14.563 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@mgr.z 2024-04-25T01:25:14.865 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:25:14 smithi169 systemd[1]: Stopping Ceph mgr.z for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:25:15.128 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:25:14 smithi169 podman[67551]: 2024-04-25 01:25:14.886266461 +0000 UTC m=+0.121924399 container died 4a052505ea91141e28ded6f9007d8529579c45606fe1028a0f3f2367c6699892 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T01:25:15.439 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:25:15 smithi169 podman[67551]: 2024-04-25 01:25:15.249159733 +0000 UTC m=+0.484817670 container cleanup 4a052505ea91141e28ded6f9007d8529579c45606fe1028a0f3f2367c6699892 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T01:25:15.439 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 01:25:15 smithi169 bash[67551]: ceph-95849b94-029f-11ef-bc93-c7b262605968-mgr-z 2024-04-25T01:25:15.605 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T01:25:15.639 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:25:15.639 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T01:25:15.639 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T01:25:15.639 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.0 2024-04-25T01:25:15.999 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 01:25:15 smithi031 systemd[1]: Stopping Ceph osd.0 for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:25:16.499 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 01:25:16 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-0[42319]: 2024-04-25T01:25:16.157+0000 7f5567000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:25:16.500 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 01:25:16 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-0[42319]: 2024-04-25T01:25:16.157+0000 7f5567000700 -1 osd.0 92 *** Got signal Terminated *** 2024-04-25T01:25:16.500 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 01:25:16 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-0[42319]: 2024-04-25T01:25:16.157+0000 7f5567000700 -1 osd.0 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:25:46.408 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 01:25:46 smithi031 bash[87929]: time="2024-04-25T01:25:46Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-0 in 30 seconds, resorting to SIGKILL" 2024-04-25T01:25:46.677 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 01:25:46 smithi031 podman[87929]: 2024-04-25 01:25:46.427199973 +0000 UTC m=+30.388978373 container died 09f1b5c3759a8cb8f199793415f383bd4cbff4586bd649096a19d87b3fdbf20d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-0, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T01:25:46.997 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 01:25:46 smithi031 podman[87929]: 2024-04-25 01:25:46.841093374 +0000 UTC m=+30.802871762 container cleanup 09f1b5c3759a8cb8f199793415f383bd4cbff4586bd649096a19d87b3fdbf20d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:25:46.997 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 01:25:46 smithi031 bash[87929]: ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-0 2024-04-25T01:25:47.248 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 01:25:47 smithi031 podman[87951]: 2024-04-25 01:25:47.017277551 +0000 UTC m=+0.578089391 container remove 09f1b5c3759a8cb8f199793415f383bd4cbff4586bd649096a19d87b3fdbf20d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T01:25:47.249 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 01:25:47 smithi031 systemd[1]: ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.0.service: Main process exited, code=exited, status=137/n/a 2024-04-25T01:25:47.999 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 01:25:47 smithi031 podman[88044]: 2024-04-25 01:25:47.663875185 +0000 UTC m=+0.038318968 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:25:47.999 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 01:25:47 smithi031 podman[88044]: 2024-04-25 01:25:47.759591508 +0000 UTC m=+0.134035270 container create 9a8f6f063f8181e4304cab8e697d4fa78a16dcb76e34e623f28c9da5bd02a329 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:25:48.282 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 01:25:48 smithi031 podman[88044]: 2024-04-25 01:25:48.121593368 +0000 UTC m=+0.496037177 container init 9a8f6f063f8181e4304cab8e697d4fa78a16dcb76e34e623f28c9da5bd02a329 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:25:48.282 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 01:25:48 smithi031 podman[88044]: 2024-04-25 01:25:48.130528903 +0000 UTC m=+0.504972666 container start 9a8f6f063f8181e4304cab8e697d4fa78a16dcb76e34e623f28c9da5bd02a329 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:25:48.282 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 01:25:48 smithi031 podman[88044]: 2024-04-25 01:25:48.220602808 +0000 UTC m=+0.595046673 container attach 9a8f6f063f8181e4304cab8e697d4fa78a16dcb76e34e623f28c9da5bd02a329 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-25T01:25:48.999 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 01:25:48 smithi031 podman[88044]: 2024-04-25 01:25:48.574743288 +0000 UTC m=+0.949187137 container died 9a8f6f063f8181e4304cab8e697d4fa78a16dcb76e34e623f28c9da5bd02a329 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T01:25:49.435 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T01:25:49.530 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:25:49.531 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T01:25:49.531 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T01:25:49.531 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.1 2024-04-25T01:25:49.862 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:25:49 smithi040 systemd[1]: Stopping Ceph osd.1 for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:25:50.165 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:25:49 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-1[37137]: 2024-04-25T01:25:49.881+0000 7f1a53f00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:25:50.165 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:25:49 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-1[37137]: 2024-04-25T01:25:49.881+0000 7f1a53f00700 -1 osd.1 92 *** Got signal Terminated *** 2024-04-25T01:25:50.165 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:25:49 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-1[37137]: 2024-04-25T01:25:49.881+0000 7f1a53f00700 -1 osd.1 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:26:20.191 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:26:19 smithi040 bash[67510]: time="2024-04-25T01:26:19Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-1 in 30 seconds, resorting to SIGKILL" 2024-04-25T01:26:20.192 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:26:20 smithi040 podman[67510]: 2024-04-25 01:26:20.000041713 +0000 UTC m=+30.258152343 container died e071053113f3073182594748fbb5a52f2aa7ab554c932e9fb746e79b1856e5d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T01:26:20.569 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:26:20 smithi040 podman[67510]: 2024-04-25 01:26:20.439210012 +0000 UTC m=+30.697320630 container cleanup e071053113f3073182594748fbb5a52f2aa7ab554c932e9fb746e79b1856e5d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T01:26:20.569 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:26:20 smithi040 bash[67510]: ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-1 2024-04-25T01:26:20.941 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:26:20 smithi040 podman[67541]: 2024-04-25 01:26:20.590247326 +0000 UTC m=+0.590377023 container remove e071053113f3073182594748fbb5a52f2aa7ab554c932e9fb746e79b1856e5d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-1, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T01:26:20.941 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:26:20 smithi040 systemd[1]: ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.1.service: Main process exited, code=exited, status=137/n/a 2024-04-25T01:26:21.290 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:26:21 smithi040 podman[67631]: 2024-04-25 01:26:20.917188516 +0000 UTC m=+0.018666208 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:26:21.290 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:26:21 smithi040 podman[67631]: 2024-04-25 01:26:21.016749884 +0000 UTC m=+0.118227549 container create 4157d9b1158fec93a2907bed10e9ed93d21930df2a8255bb4378dfebac375974 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T01:26:21.290 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:26:21 smithi040 podman[67631]: 2024-04-25 01:26:21.211114981 +0000 UTC m=+0.312592659 container init 4157d9b1158fec93a2907bed10e9ed93d21930df2a8255bb4378dfebac375974 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T01:26:21.290 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:26:21 smithi040 podman[67631]: 2024-04-25 01:26:21.214652675 +0000 UTC m=+0.316130343 container start 4157d9b1158fec93a2907bed10e9ed93d21930df2a8255bb4378dfebac375974 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424) 2024-04-25T01:26:21.290 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:26:21 smithi040 podman[67631]: 2024-04-25 01:26:21.310673774 +0000 UTC m=+0.412151469 container attach 4157d9b1158fec93a2907bed10e9ed93d21930df2a8255bb4378dfebac375974 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:26:21.691 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:26:21 smithi040 podman[67631]: 2024-04-25 01:26:21.524216175 +0000 UTC m=+0.625693842 container died 4157d9b1158fec93a2907bed10e9ed93d21930df2a8255bb4378dfebac375974 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9) 2024-04-25T01:26:22.092 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T01:26:22.093 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:26:22 smithi040 podman[67657]: 2024-04-25 01:26:22.097901936 +0000 UTC m=+0.567983626 container remove 4157d9b1158fec93a2907bed10e9ed93d21930df2a8255bb4378dfebac375974 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T01:26:22.093 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:26:22 smithi040 systemd[1]: ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.1.service: Failed with result 'exit-code'. 2024-04-25T01:26:22.093 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:26:22 smithi040 systemd[1]: Stopped Ceph osd.1 for 95849b94-029f-11ef-bc93-c7b262605968. 2024-04-25T01:26:22.093 INFO:journalctl@ceph.osd.1.smithi040.stdout:Apr 25 01:26:22 smithi040 systemd[1]: ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.1.service: Consumed 8min 41.087s CPU time. 2024-04-25T01:26:22.132 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:26:22.132 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T01:26:22.132 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T01:26:22.132 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.2 2024-04-25T01:26:22.428 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:26:22 smithi169 systemd[1]: Stopping Ceph osd.2 for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:26:22.689 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:26:22 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-2[37904]: 2024-04-25T01:26:22.448+0000 7f6fbc3c4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:26:22.689 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:26:22 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-2[37904]: 2024-04-25T01:26:22.448+0000 7f6fbc3c4700 -1 osd.2 92 *** Got signal Terminated *** 2024-04-25T01:26:22.689 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:26:22 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-2[37904]: 2024-04-25T01:26:22.448+0000 7f6fbc3c4700 -1 osd.2 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:26:46.941 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:26:46 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:26:46.661+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:26:46.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:26:46 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:26:46.712+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:26:47.941 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:26:47 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:26:47.659+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:26:47.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:26:47 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:26:47.745+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:26:48.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:26:48 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:26:48.030+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:26:48.941 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:26:48 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:26:48.687+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:26:48.964 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:26:48 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:26:48.733+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:26:49.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:26:48 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:26:48.984+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:26:49.919 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:26:49 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:26:49.687+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:26:49.941 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:26:49 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:26:49.708+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:26:50.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:26:49 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:26:49.940+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:26:50.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:26:50 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:26:50.096+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:26:50.941 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:26:50 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:26:50.675+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:26:50.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:26:50 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:26:50.897+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:26:50.999 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:26:50 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:26:50.667+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:26:51.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:26:51 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:26:51.140+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:26:51.662 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:26:51 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-3[48821]: 2024-04-25T01:26:51.396+0000 7f7e19000700 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.683624+0000 front 2024-04-25T01:25:18.684511+0000 (oldest deadline 2024-04-25T01:26:50.493081+0000) 2024-04-25T01:26:51.941 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:26:51 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:26:51.646+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:26:51.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:26:51 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:26:51.887+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:26:51.999 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:26:51 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:26:51.682+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:26:52.433 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:26:52 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:26:52.166+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:26:52.434 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:26:52 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:26:52.154+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:26:52.688 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:26:52 smithi169 bash[67677]: time="2024-04-25T01:26:52Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-2 in 30 seconds, resorting to SIGKILL" 2024-04-25T01:26:52.688 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:26:52 smithi169 podman[67677]: 2024-04-25 01:26:52.560838815 +0000 UTC m=+30.214064538 container died 0627eecea4e51cdc985c80c51f9e49bc729a49e5a737dd82a6ab6d69385b700b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-2, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9) 2024-04-25T01:26:52.696 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:26:52 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-3[48821]: 2024-04-25T01:26:52.440+0000 7f7e19000700 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.683624+0000 front 2024-04-25T01:25:18.684511+0000 (oldest deadline 2024-04-25T01:26:50.493081+0000) 2024-04-25T01:26:52.941 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:26:52 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:26:52.695+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:26:52.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:26:52 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:26:52.717+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:26:52.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:26:52 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:26:52.863+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:26:53.077 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:26:52 smithi169 podman[67677]: 2024-04-25 01:26:52.897173203 +0000 UTC m=+30.550398978 container cleanup 0627eecea4e51cdc985c80c51f9e49bc729a49e5a737dd82a6ab6d69385b700b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T01:26:53.077 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:26:52 smithi169 bash[67677]: ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-2 2024-04-25T01:26:53.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:26:53 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:26:53.202+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:26:53.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:26:53 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:26:53.141+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:26:53.438 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:26:53 smithi169 podman[67710]: 2024-04-25 01:26:53.098653552 +0000 UTC m=+0.530432638 container remove 0627eecea4e51cdc985c80c51f9e49bc729a49e5a737dd82a6ab6d69385b700b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-25T01:26:53.439 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:26:53 smithi169 systemd[1]: ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.2.service: Main process exited, code=exited, status=137/n/a 2024-04-25T01:26:53.748 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:26:53 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-3[48821]: 2024-04-25T01:26:53.477+0000 7f7e19000700 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.683624+0000 front 2024-04-25T01:25:18.684511+0000 (oldest deadline 2024-04-25T01:26:50.493081+0000) 2024-04-25T01:26:53.749 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:26:53 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:26:53.716+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:26:53.751 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:26:53 smithi169 podman[67797]: 2024-04-25 01:26:53.41771584 +0000 UTC m=+0.018535214 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:26:53.751 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:26:53 smithi169 podman[67797]: 2024-04-25 01:26:53.601807302 +0000 UTC m=+0.202626649 container create cd7e2a93c11827946d7d45c25023e771372a45cf2a1b6c3507d1a2348a45d8ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T01:26:54.120 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:26:53 smithi169 podman[67797]: 2024-04-25 01:26:53.868665494 +0000 UTC m=+0.469484840 container init cd7e2a93c11827946d7d45c25023e771372a45cf2a1b6c3507d1a2348a45d8ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T01:26:54.121 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:26:53 smithi169 podman[67797]: 2024-04-25 01:26:53.871703096 +0000 UTC m=+0.472522448 container start cd7e2a93c11827946d7d45c25023e771372a45cf2a1b6c3507d1a2348a45d8ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:26:54.121 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:26:53 smithi169 podman[67797]: 2024-04-25 01:26:53.943621337 +0000 UTC m=+0.544440700 container attach cd7e2a93c11827946d7d45c25023e771372a45cf2a1b6c3507d1a2348a45d8ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T01:26:54.191 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:26:53 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:26:53.740+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:26:54.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:26:53 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:26:53.898+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:26:54.438 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 01:26:54 smithi169 podman[67797]: 2024-04-25 01:26:54.170346689 +0000 UTC m=+0.771166029 container died cd7e2a93c11827946d7d45c25023e771372a45cf2a1b6c3507d1a2348a45d8ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T01:26:54.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:26:54 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:26:54.197+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:26:54.439 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:26:54 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:26:54.140+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:26:54.742 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:26:54 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-3[48821]: 2024-04-25T01:26:54.485+0000 7f7e19000700 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.683624+0000 front 2024-04-25T01:25:18.684511+0000 (oldest deadline 2024-04-25T01:26:50.493081+0000) 2024-04-25T01:26:54.757 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T01:26:54.795 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:26:54.795 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T01:26:54.796 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T01:26:54.796 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.3 2024-04-25T01:26:54.998 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:26:54 smithi031 systemd[1]: Stopping Ceph osd.3 for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:26:54.999 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:26:54 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:26:54.867+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:26:54.999 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:26:54 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:26:54.761+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:26:55.191 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:26:54 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:26:54.783+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:26:55.429 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:26:55 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:26:55.243+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:26:55.443 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:26:55 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:26:55.118+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:26:55.748 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:26:55 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-3[48821]: 2024-04-25T01:26:55.351+0000 7f7e21e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:26:55.748 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:26:55 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-3[48821]: 2024-04-25T01:26:55.351+0000 7f7e21e00700 -1 osd.3 92 *** Got signal Terminated *** 2024-04-25T01:26:55.748 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:26:55 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-3[48821]: 2024-04-25T01:26:55.351+0000 7f7e21e00700 -1 osd.3 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:26:55.749 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:26:55 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-3[48821]: 2024-04-25T01:26:55.456+0000 7f7e19000700 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.683624+0000 front 2024-04-25T01:25:18.684511+0000 (oldest deadline 2024-04-25T01:26:50.493081+0000) 2024-04-25T01:26:56.191 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:26:55 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:26:55.769+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:26:56.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:26:55 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:26:55.861+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:26:56.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:26:55 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:26:55.795+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:26:56.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:26:56 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:26:56.155+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:26:56.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:26:56 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:26:56.234+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:26:56.748 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:26:56 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-3[48821]: 2024-04-25T01:26:56.483+0000 7f7e19000700 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.683624+0000 front 2024-04-25T01:25:18.684511+0000 (oldest deadline 2024-04-25T01:26:50.493081+0000) 2024-04-25T01:26:57.191 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:26:56 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:26:56.724+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:26:57.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:26:56 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:26:56.860+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:26:57.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:26:56 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:26:56.809+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:26:57.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:26:57 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:26:57.155+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:26:57.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:26:57 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:26:57.278+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:26:57.793 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:26:57 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-3[48821]: 2024-04-25T01:26:57.530+0000 7f7e19000700 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.683624+0000 front 2024-04-25T01:25:18.684511+0000 (oldest deadline 2024-04-25T01:26:50.493081+0000) 2024-04-25T01:26:57.941 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:26:57 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:26:57.703+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:26:58.122 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:26:57 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:26:57.904+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:26:58.122 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:26:57 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:26:57.812+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:26:58.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:26:58 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:26:58.256+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:26:58.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:26:58 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:26:58.139+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:26:58.748 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:26:58 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-3[48821]: 2024-04-25T01:26:58.483+0000 7f7e19000700 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.683624+0000 front 2024-04-25T01:25:18.684511+0000 (oldest deadline 2024-04-25T01:26:50.493081+0000) 2024-04-25T01:26:59.191 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:26:58 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:26:58.711+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:26:59.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:26:58 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:26:58.879+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:26:59.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:26:58 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:26:58.768+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:26:59.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:26:59 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:26:59.103+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:26:59.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:26:59 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:26:59.224+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:26:59.748 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:26:59 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-3[48821]: 2024-04-25T01:26:59.489+0000 7f7e19000700 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.683624+0000 front 2024-04-25T01:25:18.684511+0000 (oldest deadline 2024-04-25T01:26:50.493081+0000) 2024-04-25T01:26:59.748 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:26:59 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:26:59.723+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:00.191 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:26:59 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:26:59.757+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:00.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:26:59 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:26:59.834+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:00.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:00 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:00.136+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:00.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:00 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:00.266+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:00.999 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:00 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:00.698+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:00.999 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:00 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:00.797+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:01.191 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:00 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:00.805+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:01.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:01 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:01.269+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:01.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:01 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:01.131+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:01.941 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:01 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:01.767+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:01.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:01 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:01.648+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:01.999 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:01 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:01.750+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:02.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:02 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:02.144+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:02.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:02 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:02.300+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:02.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:02 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:02.612+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:02.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:02 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:02.720+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:03.191 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:02 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:02.741+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:03.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:03 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:03.299+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:03.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:03 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:03.123+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:03.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:03.691+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:03.999 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:03.650+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:04.190 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:03 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:03.778+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:04.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:04.156+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:04.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:04.284+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:04.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:04 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:04.705+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:04.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:04 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:04.604+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:05.187 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:05 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:05.141+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:05.190 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:04 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:04.758+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:05.495 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:05 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:05.263+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:06.002 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:05 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:05.615+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:06.002 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:05 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:05.681+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:06.191 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:05 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:05.781+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:06.437 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:06 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:06.263+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:06.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:06 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:06.166+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:06.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:06 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:06.645+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:06.999 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:06 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:06.617+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:07.190 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:06 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:06.802+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:07.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:07 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:07.246+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:07.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:07 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:07.148+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:07.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:07 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:07.682+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:07.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:07 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:07.574+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:08.191 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:07 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:07.828+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:08.437 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:08 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:08.232+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:08.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:08 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:08.125+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:08.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:08 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:08.672+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:08.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:08 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:08.571+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:09.191 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:08 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:08.860+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:09.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:09 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:09.257+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:09.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:09 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:09.135+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:09.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:09 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:09.597+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:09.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:09 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:09.682+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:10.191 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:09 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:09.910+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:10.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:10 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:10.253+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:10.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:10 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:10.138+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:10.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:10 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:10.591+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:10.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:10 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:10.683+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:11.191 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:10 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:10.926+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:11.437 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:11 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:11.296+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:11.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:11 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:11.089+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:11.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:11 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:11.691+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:11.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:11 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:11.562+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:12.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:12 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:12.307+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:12.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:12 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:12.104+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:12.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:11 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:11.964+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:12.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:12 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:12.605+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:12.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:12 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:12.699+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:13.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:13 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:13.105+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:13.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:13 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:13.340+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:13.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:12 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:12.968+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:13.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:13 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:13.716+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:13.999 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:13 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:13.561+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:14.347 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:14 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:14.098+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:14.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:13 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:13.994+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:14.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:14 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:14.368+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:14.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:14 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:14.729+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:14.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:14 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:14.558+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:15.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:15 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:15.322+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:15.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:15 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:15.080+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:15.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:14 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:14.972+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:15.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:15 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:15.774+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:15.999 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:15 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:15.575+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:16.190 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:15 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:15.948+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:16.322 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:16 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:16.090+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:16.688 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:16 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:16.343+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:16.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:16 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:16.813+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:16.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:16 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:16.606+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:17.437 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:17 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:17.325+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:17.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:17 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:17.078+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:17.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:16 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:16.983+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:17.900 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:17 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:17.827+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:17.900 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:17 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:17.616+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:18.190 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:17 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:17.937+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:18.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:18 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:18.102+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:18.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:18 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:18.309+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:18.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:18 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:18.780+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:18.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:18 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:18.643+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:19.190 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:18 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:18.948+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:19.437 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:19 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:19.331+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:19.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:19 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:19.129+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:19.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:19 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:19.774+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:19.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:19 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:19.681+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:20.190 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:19 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:19.898+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:20.437 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:20 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:20.312+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:20.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:20 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:20.106+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:20.941 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:20 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:20.850+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:20.941 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:27:20 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4[43687]: 2024-04-25T01:27:20.684+0000 7f8bfc9b2700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:51.512067+0000 front 2024-04-25T01:25:51.512096+0000 (oldest deadline 2024-04-25T01:27:20.214610+0000) 2024-04-25T01:27:20.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:20 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:20.737+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:20.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:20 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:20.737+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:20.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:20 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:20.662+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:20.999 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:20 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:20.662+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:21.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:21 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:21.323+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:21.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:21 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:21.097+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:21.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:21 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:21.097+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:21.941 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:27:21 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4[43687]: 2024-04-25T01:27:21.694+0000 7f8bfc9b2700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:51.512067+0000 front 2024-04-25T01:25:51.512096+0000 (oldest deadline 2024-04-25T01:27:20.214610+0000) 2024-04-25T01:27:21.941 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:21 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:21.868+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:21.942 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:21 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:21.868+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:21.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:21 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:21.787+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:21.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:21 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:21.787+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:21.999 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:21 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:21.684+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:21.999 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:21 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:21.684+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:22.338 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:22 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:22.299+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:22.338 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:22 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:22.082+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:22.339 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:22 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:22.082+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:22.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:22 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:22.826+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:22.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:22 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:22.826+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:22.999 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:22 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:22.643+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:22.999 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:22 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:22.643+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:23.190 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:27:22 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4[43687]: 2024-04-25T01:27:22.725+0000 7f8bfc9b2700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:51.512067+0000 front 2024-04-25T01:25:51.512096+0000 (oldest deadline 2024-04-25T01:27:20.214610+0000) 2024-04-25T01:27:23.191 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:22 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:22.864+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:23.191 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:22 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:22.864+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:23.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:23 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:23.104+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:23.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:23 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:23.104+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:23.439 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:23 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:23.335+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:23.439 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:23 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:23.335+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:23.940 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:27:23 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4[43687]: 2024-04-25T01:27:23.689+0000 7f8bfc9b2700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:51.512067+0000 front 2024-04-25T01:25:51.512096+0000 (oldest deadline 2024-04-25T01:27:20.214610+0000) 2024-04-25T01:27:23.941 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:23 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:23.881+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:23.941 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:23 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:23.881+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:23.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:23 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:23.658+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:23.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:23 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:23.658+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:23.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:23 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:23.829+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:23.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:23 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:23.830+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:24.438 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:24 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:24.137+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:24.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:24 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:24.074+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:24.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:24 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:24.074+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:24.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:24 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:24.290+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:24.439 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:24 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:24.290+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:24.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:24 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:24.287+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:24.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:24 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:24.845+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:24.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:24 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:24.845+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:24.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:24 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:24.628+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:24.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:24 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:24.628+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:25.191 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:27:24 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4[43687]: 2024-04-25T01:27:24.717+0000 7f8bfc9b2700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:51.512067+0000 front 2024-04-25T01:25:51.512096+0000 (oldest deadline 2024-04-25T01:27:20.214610+0000) 2024-04-25T01:27:25.191 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:24 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:24.902+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:25.191 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:24 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:24.902+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:25.438 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:25 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:25.149+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:25.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:25 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:25.310+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:25.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:25 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:25.310+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:25.439 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:25 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:25.089+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:25.439 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:25 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:25.089+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:25.611 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:27:25 smithi031 bash[88163]: time="2024-04-25T01:27:25Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-3 in 30 seconds, resorting to SIGKILL" 2024-04-25T01:27:25.612 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:25 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:25.593+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:25.612 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:25 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:25.593+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:25.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:25 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:25.315+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:25.870 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:27:25 smithi031 podman[88163]: 2024-04-25 01:27:25.633277323 +0000 UTC m=+30.415812406 container died 6e15ece744ddf19f5033fff320cc718fa662cb4c161b297c5c2106fa77c401d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T01:27:26.137 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:25 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:25.890+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:26.137 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:25 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:25.890+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:26.190 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:27:25 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4[43687]: 2024-04-25T01:27:25.736+0000 7f8bfc9b2700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:51.512067+0000 front 2024-04-25T01:25:51.512096+0000 (oldest deadline 2024-04-25T01:27:20.214610+0000) 2024-04-25T01:27:26.191 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:25 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:25.944+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:26.191 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:25 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:25.944+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:26.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:26 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:26.136+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:26.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:26 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:26.279+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:26.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:26 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:26.279+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:26.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:26 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:26.105+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:26.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:26 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:26.105+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:26.500 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:27:26 smithi031 podman[88163]: 2024-04-25 01:27:26.158526143 +0000 UTC m=+30.941061224 container cleanup 6e15ece744ddf19f5033fff320cc718fa662cb4c161b297c5c2106fa77c401d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-3, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux ) 2024-04-25T01:27:26.517 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:27:26 smithi031 bash[88163]: ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-3 2024-04-25T01:27:26.517 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:27:26 smithi031 podman[88193]: 2024-04-25 01:27:26.384852552 +0000 UTC m=+0.744539846 container remove 6e15ece744ddf19f5033fff320cc718fa662cb4c161b297c5c2106fa77c401d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T01:27:26.517 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:27:26 smithi031 systemd[1]: ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.3.service: Main process exited, code=exited, status=137/n/a 2024-04-25T01:27:26.691 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:26 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:26.276+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:26.913 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:26 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:26.631+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:26.913 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:26 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:26.631+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:26.968 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:27:26 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4[43687]: 2024-04-25T01:27:26.712+0000 7f8bfc9b2700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:51.512067+0000 front 2024-04-25T01:25:51.512096+0000 (oldest deadline 2024-04-25T01:27:20.214610+0000) 2024-04-25T01:27:27.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:26 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:26.933+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:27.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:26 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:26.933+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:27.248 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:27:27 smithi031 podman[88278]: 2024-04-25 01:27:27.077666727 +0000 UTC m=+0.037055844 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:27:27.248 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:27:27 smithi031 podman[88278]: 2024-04-25 01:27:27.17701553 +0000 UTC m=+0.136404646 container create 01dd7f1272f6c6f3d8543782ce696f510a3a552f78f9791c8b9982a32da29ed3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9) 2024-04-25T01:27:27.285 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:26 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:26.989+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:27.285 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:26 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:26.989+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:27.437 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:27 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:27.250+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:27.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:27 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:27.250+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:27.438 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:27 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:27.179+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:27.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:27 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:27.096+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:27.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:27 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:27.096+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:27.656 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:27:27 smithi031 podman[88278]: 2024-04-25 01:27:27.455472001 +0000 UTC m=+0.414861162 container init 01dd7f1272f6c6f3d8543782ce696f510a3a552f78f9791c8b9982a32da29ed3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T01:27:27.656 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:27:27 smithi031 podman[88278]: 2024-04-25 01:27:27.467344318 +0000 UTC m=+0.426733435 container start 01dd7f1272f6c6f3d8543782ce696f510a3a552f78f9791c8b9982a32da29ed3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9) 2024-04-25T01:27:27.656 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:27:27 smithi031 podman[88278]: 2024-04-25 01:27:27.570967663 +0000 UTC m=+0.530356780 container attach 01dd7f1272f6c6f3d8543782ce696f510a3a552f78f9791c8b9982a32da29ed3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T01:27:27.664 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:27 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:27.305+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:27.913 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:27 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:27.675+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:27.913 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:27 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:27.675+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:27.940 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:27:27 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4[43687]: 2024-04-25T01:27:27.684+0000 7f8bfc9b2700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:51.512067+0000 front 2024-04-25T01:25:51.512096+0000 (oldest deadline 2024-04-25T01:27:20.214610+0000) 2024-04-25T01:27:28.246 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:27 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:27.995+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:28.247 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:27 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:27.995+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:28.248 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 01:27:27 smithi031 podman[88278]: 2024-04-25 01:27:27.935438642 +0000 UTC m=+0.894827843 container died 01dd7f1272f6c6f3d8543782ce696f510a3a552f78f9791c8b9982a32da29ed3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:27:28.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:27 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:27.971+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:28.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:27 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:27.971+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:28.438 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:28 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:28.140+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:28.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:28 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:28.219+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:28.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:28 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:28.219+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:28.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:28 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:28.115+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:28.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:28 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:28.115+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:28.639 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:28 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:28.267+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:28.645 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T01:27:28.745 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:27:28.745 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T01:27:28.746 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T01:27:28.746 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.4 2024-04-25T01:27:28.941 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:27:28 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4[43687]: 2024-04-25T01:27:28.660+0000 7f8bfc9b2700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:51.512067+0000 front 2024-04-25T01:25:51.512096+0000 (oldest deadline 2024-04-25T01:27:20.214610+0000) 2024-04-25T01:27:28.941 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:27:28 smithi040 systemd[1]: Stopping Ceph osd.4 for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:27:28.951 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:28 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:28.686+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:28.951 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:28 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:28.686+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:29.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:28 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:28.970+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:29.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:28 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:28.970+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:29.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:29 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:29.117+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:29.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:29 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:29.199+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:29.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:29 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:29.199+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:29.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:29 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:29.129+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:29.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:29 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:29.129+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:29.440 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:29 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:29.221+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:29.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:29 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:29.008+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:29.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:29 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:29.008+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:29.441 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:27:29 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4[43687]: 2024-04-25T01:27:29.050+0000 7f8bffbbd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:27:29.441 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:27:29 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4[43687]: 2024-04-25T01:27:29.050+0000 7f8bffbbd700 -1 osd.4 92 *** Got signal Terminated *** 2024-04-25T01:27:29.441 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:27:29 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4[43687]: 2024-04-25T01:27:29.050+0000 7f8bffbbd700 -1 osd.4 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:27:29.941 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:27:29 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4[43687]: 2024-04-25T01:27:29.661+0000 7f8bfc9b2700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:51.512067+0000 front 2024-04-25T01:25:51.512096+0000 (oldest deadline 2024-04-25T01:27:20.214610+0000) 2024-04-25T01:27:29.953 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:29 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:29.642+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:29.954 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:29 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:29.642+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:30.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:29 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:29.973+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:30.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:29 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:29.973+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:30.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:30 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:30.130+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:30.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:30 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:30.236+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:30.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:30 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:30.236+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:30.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:30 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:30.123+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:30.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:30 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:30.123+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:30.440 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:30 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:30.259+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:30.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:30 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:30.054+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:30.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:30 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:30.054+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:30.911 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:30 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:30.603+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:30.911 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:30 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:30.603+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:30.940 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:27:30 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4[43687]: 2024-04-25T01:27:30.696+0000 7f8bfc9b2700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:51.512067+0000 front 2024-04-25T01:25:51.512096+0000 (oldest deadline 2024-04-25T01:27:20.214610+0000) 2024-04-25T01:27:31.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:30 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:30.931+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:31.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:30 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:30.931+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:31.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:31 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:31.116+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:31.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:31 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:31.191+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:31.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:31 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:31.191+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:31.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:31 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:31.141+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:31.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:31 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:31.141+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:31.440 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:31 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:31.255+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:31.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:31 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:31.067+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:31.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:31 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:31.067+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:31.928 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:31 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:31.607+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:31.928 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:31 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:31.607+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:32.074 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:27:31 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4[43687]: 2024-04-25T01:27:31.726+0000 7f8bfc9b2700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:51.512067+0000 front 2024-04-25T01:25:51.512096+0000 (oldest deadline 2024-04-25T01:27:20.214610+0000) 2024-04-25T01:27:32.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:31 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:31.948+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:32.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:31 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:31.948+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:32.437 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:32 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:32.210+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:32.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:32 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:32.210+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:32.438 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:32 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:32.115+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:32.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:32 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:32.181+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:32.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:32 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:32.181+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:32.440 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:32 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:32.296+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:32.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:32 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:32.095+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:32.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:32 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:32.095+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:32.963 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:32 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:32.635+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:32.963 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:32 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:32.635+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:33.084 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:27:32 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4[43687]: 2024-04-25T01:27:32.748+0000 7f8bfc9b2700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:51.512067+0000 front 2024-04-25T01:25:51.512096+0000 (oldest deadline 2024-04-25T01:27:20.214610+0000) 2024-04-25T01:27:33.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:32 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:32.983+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:33.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:32 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:32.983+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:33.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:33 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:33.083+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:33.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:33 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:33.220+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:33.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:33 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:33.220+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:33.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:33 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:33.174+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:33.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:33 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:33.174+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:33.440 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:33 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:33.317+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:33.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:33 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:33.105+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:33.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:33 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:33.105+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:33.957 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:33 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:33.613+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:33.958 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:33 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:33.613+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:33.958 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:33 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:33.977+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:33.958 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:33 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:33.977+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:34.083 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:27:33 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4[43687]: 2024-04-25T01:27:33.725+0000 7f8bfc9b2700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:51.512067+0000 front 2024-04-25T01:25:51.512096+0000 (oldest deadline 2024-04-25T01:27:20.214610+0000) 2024-04-25T01:27:34.438 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:34 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:34.058+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:34.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:34 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:34.208+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:34.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:34 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:34.208+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:34.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:34 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:34.130+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:34.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:34 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:34.130+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:34.440 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:34 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:34.349+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:34.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:34 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:34.104+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:34.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:34 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:34.104+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:34.992 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:34 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:34.637+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:34.992 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:34 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:34.637+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:35.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:35 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:35.012+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:35.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:35 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:35.012+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:35.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:35 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:35.091+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:35.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:35 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:35.239+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:35.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:35 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:35.239+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:35.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:35 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:35.115+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:35.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:35 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:35.115+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:35.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:35 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:35.118+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:35.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:35 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:35.118+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:35.441 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:35 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:35.363+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:35.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:35 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:35.680+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:35.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:35 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:35.680+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:36.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:36 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:36.042+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:36.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:36 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:36.243+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:36.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:36 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:36.243+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:36.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:36 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:36.086+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:36.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:36 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:36.086+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:36.440 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:36 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:36.346+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:36.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:36 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:36.148+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:36.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:36 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:36.148+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:36.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:36 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:36.041+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:36.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:36 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:36.041+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:36.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:36 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:36.672+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:36.999 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:36 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:36.672+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:37.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:37 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:37.047+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:37.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:37 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:37.284+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:37.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:37 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:37.284+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:37.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:37 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:37.070+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:37.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:37 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:37.070+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:37.440 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:37 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:37.372+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:37.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:37 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:37.131+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:37.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:37 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:37.131+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:37.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:37 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:37.070+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:37.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:37 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:37.070+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:37.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:37 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:37.629+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:37.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:37 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:37.629+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:38.338 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:38 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:38.086+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:38.338 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:38 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:38.086+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:38.438 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:38 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:38.079+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:38.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:38 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:38.111+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:38.439 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:38 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:38.111+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:38.439 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:38 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:38.289+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:38.439 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:38 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:38.289+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:38.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:38 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:38.107+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:38.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:38 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:38.107+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:38.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:38 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:38.357+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:38.997 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:38 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:38.630+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:38.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:38 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:38.630+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:39.376 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:39 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:39.041+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:39.376 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:39 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:39.041+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:39.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:39 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:39.057+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:39.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:39 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:39.256+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:39.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:39 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:39.256+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:39.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:39 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:39.122+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:39.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:39 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:39.122+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:39.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:39 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:39.085+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:39.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:39 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:39.085+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:39.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:39 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:39.397+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:39.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:39 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:39.618+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:39.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:39 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:39.618+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:40.378 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:40 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:40.005+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:40.378 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:40 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:40.005+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:40.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:40 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:40.062+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:40.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:40 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:40.301+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:40.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:40 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:40.301+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:40.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:40 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:40.074+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:40.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:40 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:40.074+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:40.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:40 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:40.089+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:40.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:40 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:40.089+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:40.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:40 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:40.397+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:40.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:40 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:40.654+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:40.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:40 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:40.654+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:41.417 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:40 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:40.998+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:41.418 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:40 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:40.998+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:41.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:41 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:41.050+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:41.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:41 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:41.288+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:41.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:41 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:41.288+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:41.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:41 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:41.101+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:41.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:41 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:41.101+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:41.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:41 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:41.065+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:41.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:41 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:41.065+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:41.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:41 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:41.438+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:41.999 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:41 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:41.668+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:41.999 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:41 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:41.668+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:42.314 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:42 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:42.093+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:42.314 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:42 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:42.064+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:42.314 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:42 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:42.064+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:42.407 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:41 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:41.973+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:42.407 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:41 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:41.973+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:42.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:42 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:42.078+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:42.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:42 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:42.078+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:42.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:42 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:42.335+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:42.688 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:42 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:42.335+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:42.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:42 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:42.428+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:42.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:42 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:42.657+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:42.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:42 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:42.657+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:43.385 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:42 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:42.968+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:43.385 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:42 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:42.968+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:43.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:43 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:43.045+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:43.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:43 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:43.291+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:43.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:43 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:43.291+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:43.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:43 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:43.043+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:43.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:43 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:43.043+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:43.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:43 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:43.060+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:43.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:43 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:43.060+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:43.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:43 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:43.405+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:43.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:43 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:43.646+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:43.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:43 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:43.646+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:44.338 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:44 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:44.000+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:44.338 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:44 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:44.000+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:44.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:44 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:44.092+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:44.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:44 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:44.084+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:44.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:44 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:44.084+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:44.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:44 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:44.299+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:44.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:44 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:44.299+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:44.499 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:44 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:44.025+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:44.499 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:44 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:44.025+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:44.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:44 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:44.359+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:44.965 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:44 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:44.693+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:44.965 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:44 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:44.693+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:45.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:44 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:44.986+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:45.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:44 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:44.986+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:45.375 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:44 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:44.989+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:45.375 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:44 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:44.989+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:45.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:45 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:45.108+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:45.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:45 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:45.280+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:45.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:45 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:45.280+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:45.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:45 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:45.049+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:45.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:45 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:45.049+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:45.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:45 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:45.395+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:45.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:45 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:45.677+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:45.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:45 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:45.677+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:46.297 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:46 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:46.080+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:46.297 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:46 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:46.043+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:46.297 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:46 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:46.043+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:46.390 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:46 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:46.026+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:46.390 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:46 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:46.026+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:46.497 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:46 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:46.030+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:46.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:46 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:46.030+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:46.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:46 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:46.318+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:46.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:46 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:46.318+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:46.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:46 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:46.411+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:46.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:46 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:46.671+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:46.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:46 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:46.671+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:47.291 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:47 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:47.083+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:47.292 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:46 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:46.994+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:47.292 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:46 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:46.994+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:47.403 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:47 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:47.016+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:47.403 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:47 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:47.016+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:47.497 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:47 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:47.019+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:47.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:47 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:47.019+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:47.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:47 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:47.312+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:47.688 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:47 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:47.312+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:47.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:47 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:47.424+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:47.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:47 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:47.624+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:47.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:47 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:47.624+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:48.279 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:48 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:48.044+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:48.280 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:47 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:47.989+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:48.280 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:47 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:47.989+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:48.402 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:48 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:48.019+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:48.402 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:48 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:48.019+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:48.426 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:48 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:48.031+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:48.426 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:48 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:48.031+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:48.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:48 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:48.300+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:48.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:48 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:48.300+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:48.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:48 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:48.422+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:48.748 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:48 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:48.670+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:48.748 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:48 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:48.670+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:49.250 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:49 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:49.001+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:49.251 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:49 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:49.038+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:49.251 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:49 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:49.038+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:49.403 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:49 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:49.045+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:49.403 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:49 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:49.045+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:49.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:49 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:49.037+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:49.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:49 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:49.037+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:49.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:49 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:49.271+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:49.688 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:49 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:49.271+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:49.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:49 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:49.424+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:50.000 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:49 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:49.629+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:50.000 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:49 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:49.629+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:50.273 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:49 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:49.998+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:50.273 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:50 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:50.056+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:50.273 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:50 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:50.056+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:50.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:50 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:50.005+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:50.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:50 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:50.005+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:50.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:50 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:50.056+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:50.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:50 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:50.056+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:50.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:50 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:50.294+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:50.688 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:50 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:50.294+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:50.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:50 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:50.474+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:50.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:50 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:50.676+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:50.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:50 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:50.676+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:51.307 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:51 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:51.038+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:51.307 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:51 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:51.099+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:51.307 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:51 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:51.099+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:51.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:51 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:51.041+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:51.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:51 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:51.041+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:51.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:51 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:51.078+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:51.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:51 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:51.078+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:51.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:51 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:51.328+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:51.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:51 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:51.328+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:51.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:51 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:51.464+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:51.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:51 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:51.640+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:51.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:51 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:51.640+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:52.311 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:52 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:52.072+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:52.311 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:52 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:52.097+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:52.311 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:52 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:52.097+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:52.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:52 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:52.033+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:52.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:52 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:52.033+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:52.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:52 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:52.100+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:52.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:52 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:52.100+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:52.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:52 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:52.331+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:52.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:52 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:52.331+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:52.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:52 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:52.510+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:52.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:52 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:52.611+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:52.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:52 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:52.611+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:53.325 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:53 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:53.028+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:53.326 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:53 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:53.028+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:27:53.326 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:53 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:53.090+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:53.326 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:53 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:53.090+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:53.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:53 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:53.027+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:53.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:53 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:53.027+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:53.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:53 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:53.086+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:53.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:53 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:53.086+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:53.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:53 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:53.346+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:53.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:53 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:53.346+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:53.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:53 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:53.553+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:53.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:53 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:53.586+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:53.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:53 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:53.586+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:54.285 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:54 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:54.029+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:54.285 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:54 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:54.029+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:27:54.285 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:54 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:54.081+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:54.286 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:54 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:54.081+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:54.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:53 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:53.997+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:54.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:53 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:53.997+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:54.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:54 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:54.073+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:54.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:54 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:54.073+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:54.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:54 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:54.306+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:54.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:54 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:54.306+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:54.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:54 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:54.306+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.343792+0000 front 2024-04-25T01:26:22.343837+0000 (oldest deadline 2024-04-25T01:27:53.747970+0000) 2024-04-25T01:27:54.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:54 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:54.583+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:54.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:54 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:54.581+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:54.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:54 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:54.581+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:55.305 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:55 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:55.058+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:55.306 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:55 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:55.058+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:27:55.306 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:55 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:55.084+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:55.306 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:55 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:55.084+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:55.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:55 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:55.018+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:55.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:55 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:55.018+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:55.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:55 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:55.075+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:55.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:55 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:55.075+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:55.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:55 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:55.075+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:27:55.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:55 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:55.327+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:55.688 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:55 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:55.327+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:55.688 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:55 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:55.327+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.343792+0000 front 2024-04-25T01:26:22.343837+0000 (oldest deadline 2024-04-25T01:27:53.747970+0000) 2024-04-25T01:27:55.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:55 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:55.541+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:55.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:55 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:55.572+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:55.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:55 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:55.572+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:56.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:56 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:56.046+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:56.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:56 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:56.046+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:27:56.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:56 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:56.109+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:56.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:56 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:56.109+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:56.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:56 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:56.279+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:56.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:56 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:56.279+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:56.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:56 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:56.279+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.343792+0000 front 2024-04-25T01:26:22.343837+0000 (oldest deadline 2024-04-25T01:27:53.747970+0000) 2024-04-25T01:27:56.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:56 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:56.004+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:56.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:56 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:56.004+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:56.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:56 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:56.038+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:56.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:56 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:56.038+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:56.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:56 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:56.038+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:27:56.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:56 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:56.584+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:56.977 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:56 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:56.554+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:56.977 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:56 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:56.554+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:56.977 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:56 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:56.554+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:27:57.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:56 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:56.998+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:57.254 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:56 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:56.998+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:57.254 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:56 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:56.998+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:27:57.298 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:57 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:57.076+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:57.299 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:57 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:57.076+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:27:57.299 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:57 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:57.061+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:57.299 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:57 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:57.061+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:57.299 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:57 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:57.061+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:27:57.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:57 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:57.042+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:57.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:57 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:57.042+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:57.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:57 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:57.042+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:27:57.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:57 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:57.320+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:57.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:57 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:57.320+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:57.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:57 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:57.320+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.343792+0000 front 2024-04-25T01:26:22.343837+0000 (oldest deadline 2024-04-25T01:27:53.747970+0000) 2024-04-25T01:27:57.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:57 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:57.590+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:57.941 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:57 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:57.590+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:27:57.997 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:57 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:57.545+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:57.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:57 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:57.545+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:57.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:57 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:57.545+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:27:58.275 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:58 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:58.085+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:58.275 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:58 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:58.085+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:27:58.276 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:58 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:58.018+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:58.276 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:58 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:58.018+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:58.276 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:58 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:58.018+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:27:58.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:58 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:58.030+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:58.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:58 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:58.030+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:58.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:58 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:58.030+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:27:58.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:58 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:58.040+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:58.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:58 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:58.040+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:58.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:58 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:58.040+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:27:58.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:58 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:58.296+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:58.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:58 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:58.296+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:58.688 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:58 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:58.296+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.343792+0000 front 2024-04-25T01:26:22.343837+0000 (oldest deadline 2024-04-25T01:27:53.747970+0000) 2024-04-25T01:27:58.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:58 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:58.564+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:58.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:58 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:58.564+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:27:58.977 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:58 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:58.559+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:58.978 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:58 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:58.559+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:58.978 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:58 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:58.559+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:27:59.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:58 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:58.997+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:27:59.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:58 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:58.997+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:27:59.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:27:58 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:27:58.997+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:27:59.303 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:59 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:59.100+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:27:59.303 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:27:59 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:27:59.100+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:27:59.304 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:59 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:59.039+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:27:59.304 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:59 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:59.039+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:27:59.304 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:27:59 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:27:59.039+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:27:59.398 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:27:59 smithi040 bash[67760]: time="2024-04-25T01:27:59Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4 in 30 seconds, resorting to SIGKILL" 2024-04-25T01:27:59.398 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:27:59 smithi040 podman[67760]: 2024-04-25 01:27:59.174668961 +0000 UTC m=+30.220746274 container died a7ab7152ffad585511f6b35ad592a415976ef1c9463247090ace1fc499a687a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T01:27:59.399 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:59 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:59.043+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:27:59.399 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:59 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:59.043+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:27:59.399 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:27:59 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:27:59.043+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:27:59.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:59 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:59.324+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:27:59.688 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:59 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:59.324+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:27:59.688 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:27:59 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:27:59.324+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.343792+0000 front 2024-04-25T01:26:22.343837+0000 (oldest deadline 2024-04-25T01:27:53.747970+0000) 2024-04-25T01:27:59.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:59 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:59.546+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:27:59.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:27:59 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:27:59.546+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:27:59.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:59 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:59.571+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:27:59.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:59 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:59.571+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:27:59.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:27:59 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:27:59.571+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:00.069 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:27:59 smithi040 podman[67760]: 2024-04-25 01:27:59.731920491 +0000 UTC m=+30.777997802 container cleanup a7ab7152ffad585511f6b35ad592a415976ef1c9463247090ace1fc499a687a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-25T01:28:00.069 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:27:59 smithi040 bash[67760]: ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4 2024-04-25T01:28:00.069 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:27:59 smithi040 podman[67777]: 2024-04-25 01:27:59.978869426 +0000 UTC m=+0.802456056 container remove a7ab7152ffad585511f6b35ad592a415976ef1c9463247090ace1fc499a687a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T01:28:00.069 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:27:59 smithi040 systemd[1]: ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.4.service: Main process exited, code=exited, status=137/n/a 2024-04-25T01:28:00.304 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:00 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:00.075+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:00.304 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:00 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:00.075+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:00.304 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:00 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:00.075+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:00.304 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:00 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:00.087+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:00.304 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:00 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:00.087+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:00.407 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:28:00 smithi040 podman[67865]: 2024-04-25 01:28:00.209712341 +0000 UTC m=+0.018738929 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:28:00.407 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:28:00 smithi040 podman[67865]: 2024-04-25 01:28:00.312960898 +0000 UTC m=+0.121987462 container create 1b19df1223af2d3d6bc4f72e95eb0515586e4989c28c78451b18e08c632170dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T01:28:00.407 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:00 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:00.089+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:00.407 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:00 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:00.089+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:00.407 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:00 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:00.089+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:00.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:00 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:00.025+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:00.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:00 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:00.025+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:00.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:00 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:00.025+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:00.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:00 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:28:00.325+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:28:00.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:00 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:28:00.325+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:28:00.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:00 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:28:00.325+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.343792+0000 front 2024-04-25T01:26:22.343837+0000 (oldest deadline 2024-04-25T01:27:53.747970+0000) 2024-04-25T01:28:00.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:00 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:00.532+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:00.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:00 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:00.532+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:00.690 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:28:00 smithi040 podman[67865]: 2024-04-25 01:28:00.548554004 +0000 UTC m=+0.357580574 container init 1b19df1223af2d3d6bc4f72e95eb0515586e4989c28c78451b18e08c632170dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T01:28:00.691 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:28:00 smithi040 podman[67865]: 2024-04-25 01:28:00.551794766 +0000 UTC m=+0.360821338 container start 1b19df1223af2d3d6bc4f72e95eb0515586e4989c28c78451b18e08c632170dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:28:00.691 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:28:00 smithi040 podman[67865]: 2024-04-25 01:28:00.648648404 +0000 UTC m=+0.457674978 container attach 1b19df1223af2d3d6bc4f72e95eb0515586e4989c28c78451b18e08c632170dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T01:28:00.997 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:00 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:00.567+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:00.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:00 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:00.567+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:00.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:00 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:00.567+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:01.109 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 25 01:28:00 smithi040 podman[67865]: 2024-04-25 01:28:00.868175289 +0000 UTC m=+0.677201867 container died 1b19df1223af2d3d6bc4f72e95eb0515586e4989c28c78451b18e08c632170dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9) 2024-04-25T01:28:01.311 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:01 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:01.025+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:01.311 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:01 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:01.025+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:01.311 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:01 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:01.025+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:01.312 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:01 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:01.117+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:01.312 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:01 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:01.117+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:01.380 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:01 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:01.129+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:01.380 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:01 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:01.129+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:01.380 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:01 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:01.129+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:01.397 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T01:28:01.435 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:28:01.435 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T01:28:01.435 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T01:28:01.435 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.5 2024-04-25T01:28:01.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:01 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:01.052+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:01.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:01 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:01.052+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:01.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:01 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:01.052+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:01.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:01 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:28:01.332+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:28:01.688 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:01 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:28:01.332+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:28:01.688 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:01 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:28:01.332+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.343792+0000 front 2024-04-25T01:26:22.343837+0000 (oldest deadline 2024-04-25T01:27:53.747970+0000) 2024-04-25T01:28:01.688 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:01 smithi169 systemd[1]: Stopping Ceph osd.5 for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:28:01.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:01 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:01.558+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:01.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:01 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:01.558+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:01.976 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:01 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:28:01.723+0000 7fe9d84ad700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:28:01.976 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:01 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:28:01.723+0000 7fe9d84ad700 -1 osd.5 92 *** Got signal Terminated *** 2024-04-25T01:28:01.976 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:01 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:28:01.723+0000 7fe9d84ad700 -1 osd.5 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:28:01.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:01 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:01.571+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:01.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:01 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:01.571+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:01.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:01 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:01.571+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:02.367 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:02 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:02.140+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:02.367 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:02 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:02.140+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:02.367 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:01 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:01.997+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:02.368 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:01 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:01.997+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:02.368 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:01 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:01.997+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:02.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:02 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:02.144+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:02.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:02 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:02.144+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:02.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:02 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:02.144+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:02.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:02 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:02.039+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:02.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:02 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:02.039+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:02.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:02 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:02.039+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:02.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:02 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:28:02.381+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:28:02.688 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:02 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:28:02.381+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:28:02.688 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:02 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:28:02.381+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.343792+0000 front 2024-04-25T01:26:22.343837+0000 (oldest deadline 2024-04-25T01:27:53.747970+0000) 2024-04-25T01:28:02.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:02 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:02.529+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:02.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:02 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:02.529+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:02.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:02 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:02.611+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:02.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:02 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:02.611+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:02.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:02 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:02.611+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:03.325 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:03 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:03.102+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:03.325 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:03 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:03.102+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:03.326 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:03 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:03.019+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:03.326 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:03 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:03.020+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:03.326 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:03 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:03.020+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:03.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:03 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:03.189+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:03.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:03 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:03.189+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:03.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:03 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:03.189+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:03.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:03.031+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:03.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:03.031+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:03.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:03.031+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:03.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:03 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:28:03.346+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:28:03.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:03 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:28:03.346+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:28:03.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:03 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:28:03.346+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.343792+0000 front 2024-04-25T01:26:22.343837+0000 (oldest deadline 2024-04-25T01:27:53.747970+0000) 2024-04-25T01:28:03.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:03 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:03.559+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:03.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:03 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:03.559+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:03.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:03.629+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:03.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:03.629+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:03.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:03.629+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:04.355 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:04.091+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:04.356 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:04.091+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:04.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:04.056+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:04.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:04.056+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:04.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:04.056+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:04.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:04 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:04.148+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:04.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:04 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:04.148+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:04.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:04 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:04.148+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:04.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:04 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:04.024+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:04.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:04 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:04.024+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:04.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:04 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:04.024+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:04.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:28:04.376+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:28:04.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:28:04.376+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:28:04.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:28:04.376+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.343792+0000 front 2024-04-25T01:26:22.343837+0000 (oldest deadline 2024-04-25T01:27:53.747970+0000) 2024-04-25T01:28:04.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:04 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:04.574+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:04.941 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:04 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:04.574+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:04.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:04 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:04.630+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:04.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:04 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:04.630+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:04.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:04 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:04.630+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:05.405 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:05 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:05.099+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:05.405 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:05 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:05.099+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:05.405 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:05 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:05.047+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:05.406 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:05 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:05.047+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:05.406 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:05 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:05.047+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:05.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:05 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:05.145+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:05.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:05 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:05.145+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:05.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:05 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:05.145+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:05.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:05 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:05.058+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:05.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:05 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:05.058+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:05.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:05 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:05.058+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:05.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:05 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:28:05.426+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:28:05.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:05 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:28:05.426+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:28:05.687 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:05 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:28:05.426+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.343792+0000 front 2024-04-25T01:26:22.343837+0000 (oldest deadline 2024-04-25T01:27:53.747970+0000) 2024-04-25T01:28:05.942 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:05 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:05.586+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:05.942 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:05 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:05.586+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:05.997 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:05 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:05.670+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:05.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:05 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:05.670+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:05.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:05 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:05.670+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:06.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:06 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:06.077+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:06.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:06 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:06.077+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:06.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:06 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:06.053+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:06.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:06 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:06.053+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:06.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:06 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:06.053+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:06.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:06 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:06.172+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:06.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:06 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:06.172+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:06.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:06 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:06.172+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:06.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:06 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:06.105+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:06.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:06 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:06.105+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:06.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:06 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:06.105+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:06.937 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:06 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:28:06.464+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:18.333724+0000 front 2024-04-25T01:25:18.334300+0000 (oldest deadline 2024-04-25T01:26:51.438542+0000) 2024-04-25T01:28:06.937 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:06 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:28:06.464+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.739675+0000 front 2024-04-25T01:25:53.739796+0000 (oldest deadline 2024-04-25T01:27:22.343494+0000) 2024-04-25T01:28:06.938 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:06 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5[44433]: 2024-04-25T01:28:06.464+0000 7fe9d52a2700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.343792+0000 front 2024-04-25T01:26:22.343837+0000 (oldest deadline 2024-04-25T01:27:53.747970+0000) 2024-04-25T01:28:06.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:06 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:06.601+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:06.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:06 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:06.601+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:06.999 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:06 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:06.636+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:06.999 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:06 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:06.636+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:06.999 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:06 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:06.636+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:07.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:07 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:07.110+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:07.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:07 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:07.110+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:07.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:07 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:07.090+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:07.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:07 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:07.090+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:07.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:07 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:07.090+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:07.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:07 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:07.156+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:07.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:07 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:07.156+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:07.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:07 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:07.156+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:07.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:07 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:07.098+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:07.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:07 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:07.098+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:07.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:07 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:07.098+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:07.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:07 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:07.563+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:07.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:07 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:07.563+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:07.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:07 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:07.679+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:07.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:07 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:07.679+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:07.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:07 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:07.679+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:08.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:08 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:08.073+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:08.438 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:08 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:08.073+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:08.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:08 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:08.123+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:08.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:08 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:08.123+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:08.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:08 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:08.123+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:08.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:08 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:08.118+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:08.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:08 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:08.118+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:08.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:08 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:08.118+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:08.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:08 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:08.069+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:08.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:08 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:08.069+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:08.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:08 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:08.069+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:08.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:08 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:08.530+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:08.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:08 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:08.530+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:08.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:08 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:08.715+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:08.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:08 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:08.715+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:08.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:08 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:08.715+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:09.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:09 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:09.089+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:09.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:09 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:09.089+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:09.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:09 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:09.156+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:09.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:09 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:09.156+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:09.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:09 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:09.156+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:09.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:09 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:09.151+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:09.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:09 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:09.151+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:09.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:09 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:09.151+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:09.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:09 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:09.036+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:09.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:09 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:09.036+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:09.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:09 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:09.036+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:09.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:09 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:09.500+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:09.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:09 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:09.500+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:09.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:09 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:09.743+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:09.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:09 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:09.743+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:09.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:09 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:09.743+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:10.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:10 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:10.123+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:10.438 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:10 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:10.123+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:10.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:10 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:10.196+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:10.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:10 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:10.197+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:10.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:10 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:10.197+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:10.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:10 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:10.171+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:10.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:10 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:10.171+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:10.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:10 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:10.171+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:10.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:10 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:10.070+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:10.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:10 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:10.070+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:10.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:10 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:10.070+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:10.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:10 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:10.495+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:10.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:10 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:10.495+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:10.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:10 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:10.753+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:10.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:10 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:10.753+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:10.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:10 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:10.753+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:11.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:11 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:11.121+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:11.438 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:11 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:11.121+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:11.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:11 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:11.246+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:11.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:11 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:11.246+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:11.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:11 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:11.246+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:11.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:11 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:11.163+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:11.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:11 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:11.164+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:11.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:11 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:11.164+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:11.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:11 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:11.075+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:11.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:11 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:11.075+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:11.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:11 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:11.075+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:11.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:11 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:11.493+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:11.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:11 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:11.493+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:11.997 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:11 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:11.735+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:11.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:11 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:11.735+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:11.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:11 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:11.735+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:12.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:12 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:12.154+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:12.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:12 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:12.154+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:12.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:12 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:12.277+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:12.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:12 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:12.277+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:12.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:12 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:12.277+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:12.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:12 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:12.165+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:12.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:12 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:12.165+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:12.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:12 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:12.165+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:12.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:12 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:12.058+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:12.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:12 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:12.058+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:12.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:12 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:12.058+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:12.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:12 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:12.530+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:12.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:12 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:12.530+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:13.069 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:12 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:12.771+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:13.069 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:12 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:12.771+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:13.070 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:12 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:12.771+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:13.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:13 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:13.196+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:13.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:13 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:13.196+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:13.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:13 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:13.276+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:13.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:13 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:13.276+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:13.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:13 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:13.276+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:13.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:13 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:13.122+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:13.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:13 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:13.122+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:13.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:13 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:13.122+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:13.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:13 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:13.089+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:13.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:13 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:13.089+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:13.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:13 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:13.089+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:13.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:13 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:13.543+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:13.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:13 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:13.543+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:14.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:14 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:14.051+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:14.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:14 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:14.051+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:14.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:14 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:14.051+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:14.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:13 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:13.819+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:14.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:13 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:13.819+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:14.249 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:13 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:13.820+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:14.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:14 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:14.204+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:14.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:14 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:14.204+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:14.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:14 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:14.296+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:14.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:14 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:14.296+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:14.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:14 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:14.296+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:14.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:14 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:14.122+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:14.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:14 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:14.122+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:14.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:14 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:14.122+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:14.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:14 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:14.501+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:14.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:14 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:14.501+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:15.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:15 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:15.074+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:15.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:15 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:15.074+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:15.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:15 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:15.074+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:15.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:14 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:14.836+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:15.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:14 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:14.836+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:15.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:14 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:14.836+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:15.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:15 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:15.116+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:15.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:15 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:15.116+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:15.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:15 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:15.116+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:15.687 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:15 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:15.250+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:15.687 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:15 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:15.250+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:15.687 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:15 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:15.314+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:15.688 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:15 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:15.314+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:15.688 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:15 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:15.314+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:15.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:15 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:15.540+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:15.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:15 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:15.540+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:16.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:16 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:16.049+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:16.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:16 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:16.049+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:16.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:16 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:16.049+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:16.249 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:15 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:15.859+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:16.249 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:15 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:15.859+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:16.249 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:15 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:15.859+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:16.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:16 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:16.109+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:16.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:16 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:16.109+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:16.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:16 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:16.109+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:16.687 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:16 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:16.264+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:16.687 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:16 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:16.264+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:16.688 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:16 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:16.290+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:16.688 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:16 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:16.290+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:16.688 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:16 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:16.290+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:16.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:16 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:16.572+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:16.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:16 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:16.572+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:17.149 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:17 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:17.017+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:17.149 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:17 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:17.017+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:17.149 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:17 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:17.017+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:17.149 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:16 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:16.867+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:17.149 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:16 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:16.867+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:17.149 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:16 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:16.867+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:17.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:17 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:17.068+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:17.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:17 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:17.068+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:17.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:17 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:17.068+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:17.687 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:17 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:17.238+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:17.687 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:17 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:17.238+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:17.687 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:17 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:17.279+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:17.688 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:17 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:17.279+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:17.688 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:17 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:17.279+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:17.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:17 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:17.533+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:17.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:17 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:17.533+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:18.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:18 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:18.013+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:18.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:18 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:18.013+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:18.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:18 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:18.013+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:18.249 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:17 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:17.846+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:18.249 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:17 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:17.846+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:18.249 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:17 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:17.846+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:18.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:18 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:18.090+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:18.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:18 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:18.090+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:18.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:18 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:18.090+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:18.687 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:18 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:18.227+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:18.687 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:18 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:18.227+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:18.688 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:18 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:18.244+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:18.688 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:18 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:18.244+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:18.688 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:18 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:18.244+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:18.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:18 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:18.577+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:18.941 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:18 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:18.577+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:19.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:19 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:19.044+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:19.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:19 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:19.044+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:19.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:19 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:19.044+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:19.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:18 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:18.876+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:19.249 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:18 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:18.876+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:19.249 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:18 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:18.876+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:19.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:19 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:19.126+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:19.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:19 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:19.126+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:19.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:19 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:19.126+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:19.687 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:19 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:19.235+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:19.687 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:19 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:19.235+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:19.688 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:19 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:19.228+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:19.688 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:19 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:19.228+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:19.688 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:19 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:19.228+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:19.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:19 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:19.545+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:19.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:19 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:19.545+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:20.247 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:20 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:20.067+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:20.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:20 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:20.067+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:20.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:20 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:20.067+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:20.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:19 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:19.862+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:20.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:19 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:19.862+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:20.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:19 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:19.862+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:20.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:20 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:20.100+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:20.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:20 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:20.100+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:20.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:20 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:20.100+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:20.687 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:20 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:20.244+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:20.687 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:20 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:20.244+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:20.687 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:20 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:20.216+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:20.688 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:20 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:20.216+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:20.688 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:20 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:20.216+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:20.713 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:20 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:20.558+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:20.713 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:20 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:20.558+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:21.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:21 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:21.022+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:21.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:21 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:21.022+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:21.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:21 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:21.022+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:21.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:20 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:20.872+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:21.249 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:20 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:20.872+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:21.249 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:20 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:20.872+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:21.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:21 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:21.201+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:21.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:21 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:21.201+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:21.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:21 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:21.232+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:21.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:21 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:21.232+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:21.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:21 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:21.232+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:21.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:21 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:21.091+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:21.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:21 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:21.091+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:21.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:21 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:21.091+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:21.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:21 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:21.546+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:21.941 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:21 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:21.546+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:22.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:21 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:21.994+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:22.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:21 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:21.994+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:22.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:21 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:21.994+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:22.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:21 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:21.895+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:22.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:21 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:21.895+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:22.249 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:21 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:21.895+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:22.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:22 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:22.229+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:22.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:22 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:22.229+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:22.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:22 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:22.189+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:22.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:22 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:22.189+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:22.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:22 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:22.189+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:22.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:22 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:22.116+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:22.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:22 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:22.116+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:22.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:22 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:22.116+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:22.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:22 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:22.536+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:22.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:22 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:22.537+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:23.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:23 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:23.035+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:23.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:23 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:23.035+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:23.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:23 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:23.035+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:23.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:22 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:22.935+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:23.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:22 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:22.935+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:23.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:22 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:22.935+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:23.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:23 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:23.114+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:23.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:23 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:23.114+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:23.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:23 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:23.114+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:23.571 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:23 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:23.218+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:23.571 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:23 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:23.218+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:23.571 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:23 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:23.209+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:23.571 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:23 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:23.209+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:23.571 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:23 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:23.209+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:23.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:23 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:23.537+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:23.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:23 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:23.537+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:24.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:24 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:24.060+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:24.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:24 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:24.060+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:24.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:24 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:24.060+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:24.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:23 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:23.889+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:24.249 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:23 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:23.889+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:24.249 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:23 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:23.889+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:24.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:24 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:24.184+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:24.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:24 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:24.184+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:24.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:24 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:24.217+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:24.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:24 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:24.217+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:24.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:24 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:24.217+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:24.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:24 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:24.147+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:24.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:24 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:24.147+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:24.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:24 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:24.147+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:24.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:24 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:24.559+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:24.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:24 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:24.559+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:25.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:25 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:25.013+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:25.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:25 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:25.013+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:25.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:25 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:25.013+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:25.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:24 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:24.847+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:25.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:24 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:24.847+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:25.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:24 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:24.847+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:25.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:25 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:25.173+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:25.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:25 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:25.173+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:25.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:25 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:25.200+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:25.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:25 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:25.200+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:25.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:25 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:25.200+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:25.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:25 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:25.191+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:25.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:25 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:25.191+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:25.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:25 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:25.191+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:25.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:25 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:25.592+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:25.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:25 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:25.592+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:26.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:26 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:26.036+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:26.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:26 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:26.036+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:26.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:26 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:26.036+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:26.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:25 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:25.798+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:26.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:25 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:25.798+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:26.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:25 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:25.798+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:26.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:26 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:26.171+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:26.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:26 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:26.171+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:26.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:26 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:26.241+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:26.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:26 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:26.241+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:26.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:26 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:26.241+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:26.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:26 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:26.241+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:26.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:26 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:26.154+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:26.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:26 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:26.154+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:26.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:26 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:26.154+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:26.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:26 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:26.641+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:26.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:26 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:26.641+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:26.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:26 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:26.641+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:59.659613+0000 front 2024-04-25T01:26:59.659981+0000 (oldest deadline 2024-04-25T01:28:25.961632+0000) 2024-04-25T01:28:27.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:27 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:27.006+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:27.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:27 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:27.006+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:27.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:27 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:27.006+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:27.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:26 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:26.823+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:27.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:26 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:26.823+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:27.249 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:26 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:26.823+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:27.249 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:26 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:26.823+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:27.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:27 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:27.169+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:27.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:27 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:27.169+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:27.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:27 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:27.169+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:27.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:27 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:27.256+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:27.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:27 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:27.256+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:27.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:27 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:27.256+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:27.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:27 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:27.256+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:27.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:27 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:27.191+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:27.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:27 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:27.191+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:27.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:27 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:27.191+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:27.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:27 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:27.645+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:27.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:27 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:27.645+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:27.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:27 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:27.645+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:59.659613+0000 front 2024-04-25T01:26:59.659981+0000 (oldest deadline 2024-04-25T01:28:25.961632+0000) 2024-04-25T01:28:28.022 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:27 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:27.789+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:28.022 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:27 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:27.789+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:28.023 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:27 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:27.789+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:28.023 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:27 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:27.789+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:28.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:28 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:28.165+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:28.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:28 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:28.165+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:28.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:28 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:28.165+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:28.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:28 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:28.239+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:28.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:28 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:28.239+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:28.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:28 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:28.239+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:28.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:28 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:28.239+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:28.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:28 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:28.180+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:28.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:28 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:28.180+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:28.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:28 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:28.180+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:28.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:28 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:28.180+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:28.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:28 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:28.042+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:28.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:28 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:28.042+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:28.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:28 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:28.042+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:28.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:28 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:28.634+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:28.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:28 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:28.634+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:28.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:28 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:28.634+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:59.659613+0000 front 2024-04-25T01:26:59.659981+0000 (oldest deadline 2024-04-25T01:28:25.961632+0000) 2024-04-25T01:28:29.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:29 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:29.003+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:29.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:29 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:29.003+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:29.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:29 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:29.003+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:29.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:28 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:28.790+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:29.249 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:28 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:28.790+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:29.249 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:28 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:28.790+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:29.249 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:28 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:28.790+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:29.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:29 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:29.118+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:29.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:29 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:29.118+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:29.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:29 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:29.118+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:29.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:29 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:29.239+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:29.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:29 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:29.239+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:29.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:29 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:29.239+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:29.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:29 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:29.239+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:29.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:29 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:29.150+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:29.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:29 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:29.150+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:29.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:29 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:29.150+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:29.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:29 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:29.150+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:29.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:29 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:29.664+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:29.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:29 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:29.664+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:29.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:29 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:29.664+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:59.659613+0000 front 2024-04-25T01:26:59.659981+0000 (oldest deadline 2024-04-25T01:28:25.961632+0000) 2024-04-25T01:28:30.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:30 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:30.011+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:30.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:30 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:30.011+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:30.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:30 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:30.011+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:30.249 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:30 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:30.011+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.084392+0000 front 2024-04-25T01:27:00.084592+0000 (oldest deadline 2024-04-25T01:28:29.890159+0000) 2024-04-25T01:28:30.249 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:29 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:29.839+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:30.249 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:29 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:29.839+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:30.249 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:29 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:29.839+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:30.249 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:29 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:29.839+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:30.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:30 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:30.110+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:30.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:30 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:30.110+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:30.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:30 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:30.110+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:30.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:30 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:30.197+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:30.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:30 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:30.197+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:30.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:30 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:30.197+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:30.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:30 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:30.197+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:30.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:30 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:30.135+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:30.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:30 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:30.135+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:30.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:30 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:30.135+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:30.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:30 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:30.135+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:30.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:30 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:30.615+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:30.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:30 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:30.615+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:30.941 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:30 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:30.615+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:59.659613+0000 front 2024-04-25T01:26:59.659981+0000 (oldest deadline 2024-04-25T01:28:25.961632+0000) 2024-04-25T01:28:31.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:30 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:30.819+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:31.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:30 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:30.819+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:31.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:30 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:30.819+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:31.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:30 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:30.819+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:31.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:31 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:31.053+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:31.249 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:31 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:31.053+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:31.249 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:31 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:31.053+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:31.249 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:31 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:31.053+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.084392+0000 front 2024-04-25T01:27:00.084592+0000 (oldest deadline 2024-04-25T01:28:29.890159+0000) 2024-04-25T01:28:31.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:31 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:31.103+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:31.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:31 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:31.103+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:31.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:31 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:31.103+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:31.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:31 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:31.168+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:31.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:31 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:31.168+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:31.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:31 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:31.168+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:31.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:31 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:31.168+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:31.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:31 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:31.103+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:31.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:31 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:31.103+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:31.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:31 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:31.103+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:31.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:31 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:31.103+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:31.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:31 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:31.629+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:31.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:31 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:31.629+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:31.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:31 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:31.629+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:59.659613+0000 front 2024-04-25T01:26:59.659981+0000 (oldest deadline 2024-04-25T01:28:25.961632+0000) 2024-04-25T01:28:32.067 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:31 smithi169 bash[67915]: time="2024-04-25T01:28:31Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5 in 30 seconds, resorting to SIGKILL" 2024-04-25T01:28:32.067 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:31 smithi169 podman[67915]: 2024-04-25 01:28:31.843680325 +0000 UTC m=+30.200493777 container died 5e1d83861159cdb43bf41d9bf4f357d326b9fceb3de8245522076126c5d568bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9) 2024-04-25T01:28:32.077 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:31 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:31.843+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:32.077 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:31 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:31.843+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:32.078 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:31 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:31.843+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:32.078 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:31 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:31.843+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:32.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:32 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:32.137+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:32.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:32 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:32.137+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:32.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:32 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:32.137+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:32.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:32 smithi169 podman[67915]: 2024-04-25 01:28:32.172110314 +0000 UTC m=+30.528923760 container cleanup 5e1d83861159cdb43bf41d9bf4f357d326b9fceb3de8245522076126c5d568bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:28:32.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:32 smithi169 bash[67915]: ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5 2024-04-25T01:28:32.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:32 smithi169 podman[67946]: 2024-04-25 01:28:32.325727229 +0000 UTC m=+0.485734812 container remove 5e1d83861159cdb43bf41d9bf4f357d326b9fceb3de8245522076126c5d568bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:28:32.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:32 smithi169 systemd[1]: ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.5.service: Main process exited, code=exited, status=137/n/a 2024-04-25T01:28:32.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:32 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:32.192+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:32.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:32 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:32.192+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:32.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:32 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:32.192+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:32.439 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:32 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:32.192+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:32.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:32 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:32.090+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:32.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:32 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:32.090+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:32.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:32 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:32.090+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:32.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:32 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:32.090+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:32.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:32 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:32.098+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:32.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:32 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:32.098+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:32.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:32 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:32.098+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:32.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:32 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:32.098+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.084392+0000 front 2024-04-25T01:27:00.084592+0000 (oldest deadline 2024-04-25T01:28:29.890159+0000) 2024-04-25T01:28:32.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:32 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:32.630+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:32.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:32 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:32.630+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:32.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:32 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:32.630+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:59.659613+0000 front 2024-04-25T01:26:59.659981+0000 (oldest deadline 2024-04-25T01:28:25.961632+0000) 2024-04-25T01:28:33.022 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:32 smithi169 podman[68034]: 2024-04-25 01:28:32.741180315 +0000 UTC m=+0.112103588 container create 6dde24c41f32100351e904f57d3eca35079a942b3039b05ab6a51870fc0cee75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T01:28:33.022 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:32 smithi169 podman[68034]: 2024-04-25 01:28:32.647904563 +0000 UTC m=+0.018827838 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:28:33.022 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:32 smithi169 podman[68034]: 2024-04-25 01:28:32.968448959 +0000 UTC m=+0.339372233 container init 6dde24c41f32100351e904f57d3eca35079a942b3039b05ab6a51870fc0cee75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T01:28:33.022 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:32 smithi169 podman[68034]: 2024-04-25 01:28:32.971685911 +0000 UTC m=+0.342609191 container start 6dde24c41f32100351e904f57d3eca35079a942b3039b05ab6a51870fc0cee75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-25T01:28:33.078 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:32 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:32.815+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:33.078 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:32 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:32.815+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:33.078 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:32 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:32.815+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:33.078 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:32 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:32.815+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:33.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:33 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:33.141+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:33.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:33 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:33.141+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:33.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:33 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:33.141+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:33.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:33 smithi169 podman[68034]: 2024-04-25 01:28:33.043558337 +0000 UTC m=+0.414481620 container attach 6dde24c41f32100351e904f57d3eca35079a942b3039b05ab6a51870fc0cee75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T01:28:33.438 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 01:28:33 smithi169 podman[68034]: 2024-04-25 01:28:33.269122885 +0000 UTC m=+0.640046152 container died 6dde24c41f32100351e904f57d3eca35079a942b3039b05ab6a51870fc0cee75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9) 2024-04-25T01:28:33.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:33 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:33.165+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:33.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:33 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:33.165+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:33.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:33 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:33.165+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:33.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:33 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:33.165+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:33.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:33 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:33.103+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:33.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:33 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:33.103+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:33.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:33 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:33.103+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:33.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:33 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:33.103+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:33.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:33 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:33.098+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:33.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:33 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:33.098+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:33.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:33 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:33.098+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:33.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:33 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:33.098+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.084392+0000 front 2024-04-25T01:27:00.084592+0000 (oldest deadline 2024-04-25T01:28:29.890159+0000) 2024-04-25T01:28:33.767 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T01:28:33.804 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:28:33.804 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T01:28:33.804 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T01:28:33.804 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.6 2024-04-25T01:28:33.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:33 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:33.600+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:33.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:33 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:33.600+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:33.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:33 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:33.600+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:59.659613+0000 front 2024-04-25T01:26:59.659981+0000 (oldest deadline 2024-04-25T01:28:25.961632+0000) 2024-04-25T01:28:34.124 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:33 smithi031 systemd[1]: Stopping Ceph osd.6 for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:28:34.124 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:33 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:33.804+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:34.124 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:33 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:33.804+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:34.124 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:33 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:33.804+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:34.124 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:33 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:33.804+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:34.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:34 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:34.185+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:34.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:34 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:34.185+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:34.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:34 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:34.185+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:34.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:34 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:34.173+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:34.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:34 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:34.173+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:34.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:34 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:34.173+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:34.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:34 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:34.173+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:34.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:34 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:34.137+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:34.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:34 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:34.137+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:34.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:34 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:34.137+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:34.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:34 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:34.137+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:34.453 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:34 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:34.144+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:34.453 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:34 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:34.144+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:34.454 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:34 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:34.144+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:34.454 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:34 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:34.144+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.084392+0000 front 2024-04-25T01:27:00.084592+0000 (oldest deadline 2024-04-25T01:28:29.890159+0000) 2024-04-25T01:28:34.454 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:34 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:34.473+0000 7f4993a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:28:34.454 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:34 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:34.473+0000 7f4993a00700 -1 osd.6 92 *** Got signal Terminated *** 2024-04-25T01:28:34.454 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:34 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:34.473+0000 7f4993a00700 -1 osd.6 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:28:34.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:34 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:34.555+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:34.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:34 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:34.555+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:34.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:34 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:34.555+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:59.659613+0000 front 2024-04-25T01:26:59.659981+0000 (oldest deadline 2024-04-25T01:28:25.961632+0000) 2024-04-25T01:28:35.135 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:34 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:34.794+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:35.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:34 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:34.794+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:35.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:34 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:34.794+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:35.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:34 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:34.794+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:35.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:35 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:35.232+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:35.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:35 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:35.232+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:35.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:35 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:35.232+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:35.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:35 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:35.137+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:35.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:35 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:35.137+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:35.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:35 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:35.137+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:35.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:35 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:35.137+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:35.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:35 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:35.154+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:35.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:35 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:35.154+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:35.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:35 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:35.154+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:35.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:35 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:35.154+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:35.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:35 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:35.156+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:35.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:35 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:35.156+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:35.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:35 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:35.156+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:35.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:35 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:35.156+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.084392+0000 front 2024-04-25T01:27:00.084592+0000 (oldest deadline 2024-04-25T01:28:29.890159+0000) 2024-04-25T01:28:35.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:35 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:35.553+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:35.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:35 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:35.553+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:35.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:35 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:35.553+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:59.659613+0000 front 2024-04-25T01:26:59.659981+0000 (oldest deadline 2024-04-25T01:28:25.961632+0000) 2024-04-25T01:28:36.134 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:35 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:35.774+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:36.134 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:35 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:35.774+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:36.134 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:35 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:35.774+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:36.134 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:35 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:35.774+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:36.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:36 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:36.258+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:36.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:36 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:36.258+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:36.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:36 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:36.258+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:36.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:36 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:36.126+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:36.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:36 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:36.126+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:36.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:36 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:36.126+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:36.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:36 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:36.126+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:36.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:36 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:36.162+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:36.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:36 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:36.162+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:36.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:36 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:36.162+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:36.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:36 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:36.162+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:36.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:36 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:36.154+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:36.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:36 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:36.154+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:36.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:36 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:36.154+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:36.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:36 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:36.154+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.084392+0000 front 2024-04-25T01:27:00.084592+0000 (oldest deadline 2024-04-25T01:28:29.890159+0000) 2024-04-25T01:28:36.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:36 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:36.598+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:36.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:36 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:36.598+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:36.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:36 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:36.598+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:59.659613+0000 front 2024-04-25T01:26:59.659981+0000 (oldest deadline 2024-04-25T01:28:25.961632+0000) 2024-04-25T01:28:36.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:36 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:36.736+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:36.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:36 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:36.736+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:36.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:36 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:36.736+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:36.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:36 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:36.736+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:37.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:37 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:37.271+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:37.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:37 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:37.271+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:37.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:37 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:37.271+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:37.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:37 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:37.156+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:37.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:37 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:37.156+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:37.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:37 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:37.156+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:37.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:37 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:37.156+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:37.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:37 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:37.203+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:37.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:37 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:37.203+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:37.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:37 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:37.203+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:37.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:37 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:37.203+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:37.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:37 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:37.161+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:37.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:37 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:37.161+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:37.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:37 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:37.161+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:37.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:37 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:37.161+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.084392+0000 front 2024-04-25T01:27:00.084592+0000 (oldest deadline 2024-04-25T01:28:29.890159+0000) 2024-04-25T01:28:37.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:37 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:37.560+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:37.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:37 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:37.560+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:37.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:37 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:37.560+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:59.659613+0000 front 2024-04-25T01:26:59.659981+0000 (oldest deadline 2024-04-25T01:28:25.961632+0000) 2024-04-25T01:28:37.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:37 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:37.725+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:37.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:37 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:37.725+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:37.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:37 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:37.725+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:37.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:37 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:37.725+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:38.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:38 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:38.275+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:38.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:38 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:38.275+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:38.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:38 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:38.275+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:38.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:38 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:38.108+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:38.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:38 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:38.108+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:38.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:38 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:38.108+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:38.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:38 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:38.108+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:38.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:38 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:38.177+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:38.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:38 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:38.177+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:38.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:38 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:38.177+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:38.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:38 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:38.177+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:38.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:38 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:38.173+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:38.499 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:38 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:38.173+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:38.499 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:38 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:38.173+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:38.499 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:38 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:38.173+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.084392+0000 front 2024-04-25T01:27:00.084592+0000 (oldest deadline 2024-04-25T01:28:29.890159+0000) 2024-04-25T01:28:38.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:38 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:38.564+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:38.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:38 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:38.564+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:38.941 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:38 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:38.564+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:59.659613+0000 front 2024-04-25T01:26:59.659981+0000 (oldest deadline 2024-04-25T01:28:25.961632+0000) 2024-04-25T01:28:39.151 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:38 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:38.774+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:39.151 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:38 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:38.774+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:39.151 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:38 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:38.774+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:39.152 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:38 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:38.774+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:39.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:39 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:39.275+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:39.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:39 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:39.275+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:39.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:39 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:39.275+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:39.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:39 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:39.125+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:39.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:39 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:39.125+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:39.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:39 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:39.125+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:39.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:39 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:39.125+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:39.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:39 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:39.135+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:39.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:39 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:39.135+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:39.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:39 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:39.135+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:39.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:39 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:39.135+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:39.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:39 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:39.172+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:21.154418+0000 front 2024-04-25T01:25:21.153726+0000 (oldest deadline 2024-04-25T01:26:47.359857+0000) 2024-04-25T01:28:39.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:39 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:39.172+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.061518+0000 front 2024-04-25T01:25:52.061644+0000 (oldest deadline 2024-04-25T01:27:20.373529+0000) 2024-04-25T01:28:39.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:39 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:39.172+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:22.075200+0000 front 2024-04-25T01:26:22.076070+0000 (oldest deadline 2024-04-25T01:27:54.882252+0000) 2024-04-25T01:28:39.498 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:39 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6[55671]: 2024-04-25T01:28:39.172+0000 7f498ac00700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.084392+0000 front 2024-04-25T01:27:00.084592+0000 (oldest deadline 2024-04-25T01:28:29.890159+0000) 2024-04-25T01:28:39.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:39 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:39.554+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:39.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:39 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:39.554+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:39.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:39 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:39.554+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:59.659613+0000 front 2024-04-25T01:26:59.659981+0000 (oldest deadline 2024-04-25T01:28:25.961632+0000) 2024-04-25T01:28:40.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:40 smithi031 podman[88394]: 2024-04-25 01:28:40.031589953 +0000 UTC m=+5.751064567 container died 96a85b35987c9f8f4099efc0e35eb59584d8e23868c3111a9d5ce6bd0adc5614 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T01:28:40.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:39 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:39.819+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:40.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:39 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:39.819+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:40.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:39 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:39.819+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:40.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:39 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:39.819+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:40.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:40 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:40.257+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:40.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:40 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:40.257+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:40.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:40 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:40.257+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:40.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:40 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:40.147+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:40.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:40 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:40.147+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:40.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:40 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:40.147+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:40.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:40 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:40.147+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:40.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:40 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:40.168+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:40.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:40 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:40.168+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:40.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:40 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:40.168+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:40.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:40 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:40.168+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:40.751 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:40 smithi031 podman[88394]: 2024-04-25 01:28:40.589699899 +0000 UTC m=+6.309174506 container cleanup 96a85b35987c9f8f4099efc0e35eb59584d8e23868c3111a9d5ce6bd0adc5614 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240424) 2024-04-25T01:28:40.751 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:40 smithi031 bash[88394]: ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6 2024-04-25T01:28:40.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:40 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:40.511+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:40.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:40 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:40.511+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:40.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:40 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:40.511+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:59.659613+0000 front 2024-04-25T01:26:59.659981+0000 (oldest deadline 2024-04-25T01:28:25.961632+0000) 2024-04-25T01:28:41.248 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:40 smithi031 podman[88412]: 2024-04-25 01:28:40.848868737 +0000 UTC m=+0.801810869 container remove 96a85b35987c9f8f4099efc0e35eb59584d8e23868c3111a9d5ce6bd0adc5614 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T01:28:41.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:40 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:40.822+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:41.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:40 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:40.822+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:41.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:40 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:40.822+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:41.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:40 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:40.822+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:41.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:41 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:41.253+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:41.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:41 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:41.253+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:41.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:41 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:41.253+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:41.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:41 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:41.170+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:41.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:41 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:41.170+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:41.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:41 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:41.170+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:41.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:41 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:41.170+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:41.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:41 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:41.181+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:41.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:41 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:41.181+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:41.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:41 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:41.181+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:41.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:41 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:41.181+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:41.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:41 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:41.524+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:41.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:41 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:41.524+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:41.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:41 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:41.524+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:59.659613+0000 front 2024-04-25T01:26:59.659981+0000 (oldest deadline 2024-04-25T01:28:25.961632+0000) 2024-04-25T01:28:41.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:41 smithi031 podman[88500]: 2024-04-25 01:28:41.757774758 +0000 UTC m=+0.122479645 container create 047f12bc727ac52291caf1ddd179340bfafdd29ff5b05cb54e3d0351775cbae0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T01:28:41.998 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:41 smithi031 podman[88500]: 2024-04-25 01:28:41.668373848 +0000 UTC m=+0.033078725 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:28:41.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:41 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:41.781+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:41.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:41 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:41.781+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:41.999 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:41 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:41.781+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:41.999 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:41 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:41.781+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:42.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:42 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:42.300+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:42.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:42 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:42.300+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:42.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:42 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:42.300+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:42.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:42 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:42.135+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:42.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:42 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:42.135+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:42.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:42 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:42.135+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:42.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:42 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:42.135+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:42.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:42 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:42.172+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:42.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:42 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:42.172+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:42.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:42 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:42.172+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:42.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:42 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:42.172+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:42.499 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:42 smithi031 podman[88500]: 2024-04-25 01:28:42.056311452 +0000 UTC m=+0.421016372 container init 047f12bc727ac52291caf1ddd179340bfafdd29ff5b05cb54e3d0351775cbae0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6-deactivate, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-25T01:28:42.499 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:42 smithi031 podman[88500]: 2024-04-25 01:28:42.068628486 +0000 UTC m=+0.433333373 container start 047f12bc727ac52291caf1ddd179340bfafdd29ff5b05cb54e3d0351775cbae0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T01:28:42.499 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:42 smithi031 podman[88500]: 2024-04-25 01:28:42.135236162 +0000 UTC m=+0.499941084 container attach 047f12bc727ac52291caf1ddd179340bfafdd29ff5b05cb54e3d0351775cbae0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:28:42.862 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 01:28:42 smithi031 podman[88500]: 2024-04-25 01:28:42.542028243 +0000 UTC m=+0.906733163 container died 047f12bc727ac52291caf1ddd179340bfafdd29ff5b05cb54e3d0351775cbae0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-25T01:28:42.862 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:42 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:42.736+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:42.862 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:42 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:42.736+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:42.862 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:42 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:42.736+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:42.863 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:42 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:42.736+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:42.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:42 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:42.542+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:42.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:42 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:42.542+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:42.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:42 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:42.542+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:59.659613+0000 front 2024-04-25T01:26:59.659981+0000 (oldest deadline 2024-04-25T01:28:25.961632+0000) 2024-04-25T01:28:43.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:43 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:43.302+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:43.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:43 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:43.302+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:43.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:43 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:43.302+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:43.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:43 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:43.148+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:43.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:43 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:43.148+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:43.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:43 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:43.148+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:43.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:43 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:43.148+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:43.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:43 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:43.128+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:43.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:43 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:43.128+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:43.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:43 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:43.128+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:43.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:43 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:43.128+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:43.443 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T01:28:43.558 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:28:43.559 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T01:28:43.559 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T01:28:43.559 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.7 2024-04-25T01:28:43.874 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:43 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:43.538+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:43.874 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:43 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:43.538+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:43.874 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:43 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:43.538+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:59.659613+0000 front 2024-04-25T01:26:59.659981+0000 (oldest deadline 2024-04-25T01:28:25.961632+0000) 2024-04-25T01:28:43.874 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:43 smithi040 systemd[1]: Stopping Ceph osd.7 for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:28:43.874 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:43 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:43.894+0000 7fa7afc7b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:28:43.875 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:43 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:43.894+0000 7fa7afc7b700 -1 osd.7 92 *** Got signal Terminated *** 2024-04-25T01:28:43.875 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:43 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:43.894+0000 7fa7afc7b700 -1 osd.7 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:28:43.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:43 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:43.761+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:43.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:43 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:43.761+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:43.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:43 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:43.761+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:43.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:43 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:43.761+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:44.190 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:44 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:44.086+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:44.190 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:44 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:44.086+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:44.190 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:44 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:44.086+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:44.190 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:44 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:44.086+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:44.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:44 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:44.267+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:44.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:44 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:44.267+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:44.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:44 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:44.267+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:44.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:44 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:44.191+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:44.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:44 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:44.191+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:44.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:44 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:44.191+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:44.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:44 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:44.191+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:44.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:44 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:44.496+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:44.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:44 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:44.496+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:44.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:44 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:44.496+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:59.659613+0000 front 2024-04-25T01:26:59.659981+0000 (oldest deadline 2024-04-25T01:28:25.961632+0000) 2024-04-25T01:28:45.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:44 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:44.800+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:45.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:44 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:44.800+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:45.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:44 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:44.800+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:45.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:44 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:44.800+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:45.433 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:45 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:45.134+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:45.434 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:45 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:45.134+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:45.434 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:45 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:45.134+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:45.434 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:45 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:45.134+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:45.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:45 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:45.315+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:45.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:45 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:45.315+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:45.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:45 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:45.315+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:45.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:45 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:45.166+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:45.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:45 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:45.166+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:45.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:45 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:45.166+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:45.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:45 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:45.166+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:45.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:45 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:45.455+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:45.691 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:45 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:45.455+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:45.691 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:45 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:45.455+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:59.659613+0000 front 2024-04-25T01:26:59.659981+0000 (oldest deadline 2024-04-25T01:28:25.961632+0000) 2024-04-25T01:28:46.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:45 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:45.784+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:46.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:45 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:45.784+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:46.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:45 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:45.784+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:46.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:45 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:45.784+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:46.428 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:46 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:46.112+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:46.428 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:46 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:46.112+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:46.428 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:46 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:46.112+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:46.428 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:46 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:46.112+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:46.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:46 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:46.338+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:46.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:46 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:46.338+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:46.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:46 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:46.338+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:46.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:46 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:46.129+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:46.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:46 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:46.129+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:46.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:46 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:46.129+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:46.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:46 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:46.129+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:46.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:46 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:46.449+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:46.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:46 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:46.449+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:46.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:46 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:46.449+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:59.659613+0000 front 2024-04-25T01:26:59.659981+0000 (oldest deadline 2024-04-25T01:28:25.961632+0000) 2024-04-25T01:28:47.150 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:46 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:46.810+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:47.150 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:46 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:46.810+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:47.150 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:46 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:46.810+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:47.150 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:46 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:46.810+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:47.406 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:47 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:47.099+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:47.406 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:47 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:47.099+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:47.406 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:47 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:47.099+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:47.406 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:47 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:47.099+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:47.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:47 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:47.292+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:47.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:47 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:47.292+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:47.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:47 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:47.292+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:47.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:47 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:47.140+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:47.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:47 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:47.140+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:47.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:47 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:47.140+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:47.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:47 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:47.140+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:47.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:47 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:47.427+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:47.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:47 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:47.427+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:47.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:47 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:47.427+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:59.659613+0000 front 2024-04-25T01:26:59.659981+0000 (oldest deadline 2024-04-25T01:28:25.961632+0000) 2024-04-25T01:28:48.188 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:47 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:47.783+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:48.188 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:47 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:47.783+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:48.188 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:47 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:47.783+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:48.188 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:47 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:47.783+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:48.392 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:48 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:48.109+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:48.392 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:48 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:48.109+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:48.392 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:48 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:48.109+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:48.392 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:48 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:48.109+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:48.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:48 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:48.335+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:48.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:48 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:48.335+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:48.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:48 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:48.335+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:48.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:48 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:48.101+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:48.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:48 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:48.101+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:48.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:48 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:48.101+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:48.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:48 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:48.101+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:48.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:48 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:48.413+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:53.651272+0000 front 2024-04-25T01:25:53.651302+0000 (oldest deadline 2024-04-25T01:27:23.355184+0000) 2024-04-25T01:28:48.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:48 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:48.413+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.355559+0000 front 2024-04-25T01:26:23.355572+0000 (oldest deadline 2024-04-25T01:27:56.758511+0000) 2024-04-25T01:28:48.690 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:48 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7[50457]: 2024-04-25T01:28:48.413+0000 7fa7a8a68700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:59.659613+0000 front 2024-04-25T01:26:59.659981+0000 (oldest deadline 2024-04-25T01:28:25.961632+0000) 2024-04-25T01:28:48.880 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:48 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:48.807+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:48.880 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:48 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:48.807+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:48.880 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:48 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:48.807+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:48.880 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:48 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:48.807+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:49.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:49 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:49.289+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:49.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:49 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:49.289+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:49.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:49 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:49.289+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:49.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:49 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:49.086+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:49.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:49 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:49.086+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:49.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:49 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:49.086+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:49.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:49 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:49.086+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:49.440 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:49 smithi040 podman[67984]: 2024-04-25 01:28:49.336133803 +0000 UTC m=+5.562741230 container died 6554eb45317f4899558f94f422ea7c83cd63e6062f2e57c850014c110c5a8613 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T01:28:49.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:49 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:49.159+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:49.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:49 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:49.159+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:49.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:49 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:49.159+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:49.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:49 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:49.159+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:50.024 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:49 smithi040 podman[67984]: 2024-04-25 01:28:49.811694061 +0000 UTC m=+6.038301484 container cleanup 6554eb45317f4899558f94f422ea7c83cd63e6062f2e57c850014c110c5a8613 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:28:50.024 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:49 smithi040 bash[67984]: ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7 2024-04-25T01:28:50.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:49 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:49.791+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:50.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:49 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:49.791+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:50.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:49 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:49.791+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:50.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:49 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:49.791+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:50.355 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:50 smithi040 podman[68002]: 2024-04-25 01:28:50.04610808 +0000 UTC m=+0.711262825 container remove 6554eb45317f4899558f94f422ea7c83cd63e6062f2e57c850014c110c5a8613 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T01:28:50.355 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:50 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:50.155+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:50.356 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:50 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:50.155+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:50.356 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:50 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:50.155+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:50.356 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:50 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:50.155+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:50.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:50 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:50.266+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:50.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:50 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:50.266+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:50.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:50 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:50.266+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:50.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:50 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:50.119+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:50.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:50 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:50.119+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:50.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:50 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:50.119+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:50.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:50 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:50.119+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:50.622 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:50 smithi040 podman[68091]: 2024-04-25 01:28:50.281412258 +0000 UTC m=+0.018593951 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:28:50.622 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:50 smithi040 podman[68091]: 2024-04-25 01:28:50.522041381 +0000 UTC m=+0.259223064 container create 7397479c2c74e799df5c6ab1fc2fa70b91fb52e0f1365d751d66ec21efb1d18f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T01:28:50.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:50 smithi040 podman[68091]: 2024-04-25 01:28:50.749293985 +0000 UTC m=+0.486475668 container init 7397479c2c74e799df5c6ab1fc2fa70b91fb52e0f1365d751d66ec21efb1d18f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T01:28:50.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:50 smithi040 podman[68091]: 2024-04-25 01:28:50.752415745 +0000 UTC m=+0.489597434 container start 7397479c2c74e799df5c6ab1fc2fa70b91fb52e0f1365d751d66ec21efb1d18f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:28:50.940 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:50 smithi040 podman[68091]: 2024-04-25 01:28:50.832719534 +0000 UTC m=+0.569901217 container attach 7397479c2c74e799df5c6ab1fc2fa70b91fb52e0f1365d751d66ec21efb1d18f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:28:51.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:50 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:50.806+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:51.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:50 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:50.806+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:51.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:50 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:50.806+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:51.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:50 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:50.806+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:51.314 INFO:journalctl@ceph.osd.7.smithi040.stdout:Apr 25 01:28:51 smithi040 podman[68091]: 2024-04-25 01:28:51.050894559 +0000 UTC m=+0.788076253 container died 7397479c2c74e799df5c6ab1fc2fa70b91fb52e0f1365d751d66ec21efb1d18f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9) 2024-04-25T01:28:51.315 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:51 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:51.159+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:51.315 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:51 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:51.159+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:51.315 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:51 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:51.159+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:51.315 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:51 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:51.159+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:51.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:51 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:51.126+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:51.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:51 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:51.126+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:51.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:51 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:51.126+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:51.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:51 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:51.126+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:51.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:51 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:51.249+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:51.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:51 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:51.249+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:51.438 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:51 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:51.249+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:51.635 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T01:28:51.674 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:28:51.675 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T01:28:51.675 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T01:28:51.675 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.8 2024-04-25T01:28:51.977 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:51 smithi169 systemd[1]: Stopping Ceph osd.8 for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:28:52.240 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:52 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:51.999+0000 7fae7aa2f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:28:52.241 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:52 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:51.999+0000 7fae7aa2f700 -1 osd.8 92 *** Got signal Terminated *** 2024-04-25T01:28:52.241 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:52 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:51.999+0000 7fae7aa2f700 -1 osd.8 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:28:52.241 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:52 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:52.081+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:52.241 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:52 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:52.081+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:52.241 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:52 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:52.081+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:52.241 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:52 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:52.081+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:52.247 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:51 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:51.806+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:52.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:51 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:51.806+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:52.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:51 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:51.806+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:52.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:51 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:51.806+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:52.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:52 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:52.186+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:52.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:52 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:52.186+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:52.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:52 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:52.186+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:52.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:52 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:52.186+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:52.687 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:52 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:52.288+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:52.687 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:52 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:52.288+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:52.687 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:52 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:52.288+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:53.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:52 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:52.771+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:53.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:52 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:52.771+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:53.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:52 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:52.771+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:53.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:52 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:52.771+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:53.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:53 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:53.311+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:53.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:53 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:53.311+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:53.437 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:53 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:53.311+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:53.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:53 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:53.074+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:53.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:53 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:53.074+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:53.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:53 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:53.074+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:53.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:53 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:53.074+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:53.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:53 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:53.170+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:53.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:53 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:53.170+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:53.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:53 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:53.170+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:53.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:53 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:53.170+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:53.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:53 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:53.740+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:53.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:53 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:53.740+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:53.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:53 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:53.740+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:53.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:53 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:53.740+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:54.313 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:54 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:54.077+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:54.313 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:54 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:54.077+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:54.314 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:54 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:54.077+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:54.314 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:54 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:54.077+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:54.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:54 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:54.180+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:54.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:54 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:54.180+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:54.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:54 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:54.180+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:54.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:54 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:54.180+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:54.687 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:54 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:54.335+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:54.687 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:54 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:54.335+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:54.688 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:54 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:54.335+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:54.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:54 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:54.710+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:54.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:54 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:54.710+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:54.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:54 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:54.710+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:54.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:54 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:54.710+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:55.319 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:55 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:55.080+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:55.319 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:55 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:55.080+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:55.320 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:55 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:55.080+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:55.320 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:55 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:55.080+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:55.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:55 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:55.189+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:55.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:55 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:55.189+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:55.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:55 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:55.189+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:55.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:55 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:55.189+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:55.687 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:55 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:55.341+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:55.687 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:55 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:55.341+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:55.687 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:55 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:55.341+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:55.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:55 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:55.722+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:55.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:55 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:55.722+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:55.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:55 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:55.722+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:55.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:55 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:55.722+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:56.362 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:56 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:56.097+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:56.362 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:56 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:56.097+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:56.362 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:56 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:56.098+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:56.362 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:56 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:56.098+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:56.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:56 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:56.170+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:56.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:56 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:56.170+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:56.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:56 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:56.170+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:56.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:56 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:56.170+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:56.687 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:56 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:56.384+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.836769+0000 front 2024-04-25T01:25:52.836800+0000 (oldest deadline 2024-04-25T01:27:23.939676+0000) 2024-04-25T01:28:56.687 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:56 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:56.384+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:23.940029+0000 front 2024-04-25T01:26:23.940060+0000 (oldest deadline 2024-04-25T01:27:52.742614+0000) 2024-04-25T01:28:56.687 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:56 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8[51450]: 2024-04-25T01:28:56.384+0000 7fae77824700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:27:00.344178+0000 front 2024-04-25T01:27:00.344068+0000 (oldest deadline 2024-04-25T01:28:27.147313+0000) 2024-04-25T01:28:56.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:56 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:56.698+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:56.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:56 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:56.698+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:56.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:56 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:56.698+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:56.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:56 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:56.698+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:57.400 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:57 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:57.048+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:57.400 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:57 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:57.048+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:57.400 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:57 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:57.048+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:57.400 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:57 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:57.048+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:57.687 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:57 smithi169 podman[68142]: 2024-04-25 01:28:57.422400855 +0000 UTC m=+5.540409201 container died 423c4ed22be8d5d9f36fafce545c754450b970a1f3f3d83868e1397b060713f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T01:28:57.690 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:57 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:57.217+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:57.690 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:57 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:57.217+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:57.690 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:57 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:57.218+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:57.690 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:57 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:57.218+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:57.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:57 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:57.652+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:57.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:57 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:57.652+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:57.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:57 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:57.652+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:57.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:57 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:57.652+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:58.040 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:57 smithi169 podman[68142]: 2024-04-25 01:28:57.740737396 +0000 UTC m=+5.858745744 container cleanup 423c4ed22be8d5d9f36fafce545c754450b970a1f3f3d83868e1397b060713f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:28:58.040 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:57 smithi169 bash[68142]: ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8 2024-04-25T01:28:58.040 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:57 smithi169 podman[68171]: 2024-04-25 01:28:57.900321683 +0000 UTC m=+0.474773045 container remove 423c4ed22be8d5d9f36fafce545c754450b970a1f3f3d83868e1397b060713f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T01:28:58.295 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:58 smithi169 podman[68261]: 2024-04-25 01:28:58.213717443 +0000 UTC m=+0.018178045 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:28:58.295 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:58 smithi169 podman[68261]: 2024-04-25 01:28:58.318002761 +0000 UTC m=+0.122463405 container create 9dd696a48161fcb52d73767130f7c0a64e0719d00c0c50b077bc9653abcb0cda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T01:28:58.296 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:58 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:58.061+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:58.296 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:58 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:58.061+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:58.296 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:58 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:58.061+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:58.296 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:58 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:58.061+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:58.590 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:58 smithi169 podman[68261]: 2024-04-25 01:28:58.528954754 +0000 UTC m=+0.333415365 container init 9dd696a48161fcb52d73767130f7c0a64e0719d00c0c50b077bc9653abcb0cda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True) 2024-04-25T01:28:58.590 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:58 smithi169 podman[68261]: 2024-04-25 01:28:58.53245279 +0000 UTC m=+0.336913392 container start 9dd696a48161fcb52d73767130f7c0a64e0719d00c0c50b077bc9653abcb0cda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T01:28:58.590 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:58 smithi169 podman[68261]: 2024-04-25 01:28:58.612117268 +0000 UTC m=+0.416577872 container attach 9dd696a48161fcb52d73767130f7c0a64e0719d00c0c50b077bc9653abcb0cda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T01:28:58.690 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:58 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:58.253+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:58.690 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:58 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:58.253+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:58.690 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:58 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:58.253+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:58.690 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:58 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:58.253+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:58.937 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 01:28:58 smithi169 podman[68261]: 2024-04-25 01:28:58.830441281 +0000 UTC m=+0.634901884 container died 9dd696a48161fcb52d73767130f7c0a64e0719d00c0c50b077bc9653abcb0cda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T01:28:58.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:58 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:58.607+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:59.172 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:58 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:58.607+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:59.172 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:58 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:58.607+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:59.172 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:58 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:58.607+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:28:59.327 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:59 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:59.044+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:28:59.327 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:59 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:59.044+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:28:59.327 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:59 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:59.044+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:28:59.327 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:28:59 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:28:59.045+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:28:59.348 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T01:28:59.384 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:28:59.384 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T01:28:59.385 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T01:28:59.385 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.9 2024-04-25T01:28:59.569 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:59 smithi031 systemd[1]: Stopping Ceph osd.9 for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:28:59.690 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:59 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:59.268+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:28:59.690 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:59 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:59.268+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:28:59.690 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:59 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:59.268+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:28:59.690 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:28:59 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:28:59.268+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:28:59.947 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:59 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:59.591+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:28:59.947 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:59 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:59.591+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:28:59.947 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:59 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:59.591+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:28:59.948 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:28:59 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:28:59.591+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:29:00.247 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:00 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:00.124+0000 7fc706600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:29:00.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:00 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:00.124+0000 7fc706600700 -1 osd.9 92 *** Got signal Terminated *** 2024-04-25T01:29:00.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:00 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:00.124+0000 7fc706600700 -1 osd.9 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:29:00.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:00 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:00.010+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:29:00.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:00 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:00.010+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:29:00.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:00 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:00.010+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:29:00.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:00 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:00.010+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:29:00.690 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:00 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:00.226+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:29:00.690 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:00 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:00.226+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:29:00.690 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:00 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:00.226+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:29:00.691 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:00 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:00.226+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:29:00.691 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:00 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:00.226+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:31.635945+0000 front 2024-04-25T01:27:31.635965+0000 (oldest deadline 2024-04-25T01:28:59.638826+0000) 2024-04-25T01:29:00.997 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:00 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:00.592+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:29:00.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:00 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:00.592+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:29:00.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:00 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:00.592+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:29:00.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:00 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:00.592+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:29:01.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:01 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:01.029+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:29:01.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:01 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:01.029+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:29:01.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:01 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:01.029+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:29:01.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:01 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:01.029+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:29:01.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:01 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:01.029+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:32.010336+0000 front 2024-04-25T01:27:32.010529+0000 (oldest deadline 2024-04-25T01:29:00.112883+0000) 2024-04-25T01:29:01.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:01 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:01.204+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:29:01.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:01 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:01.204+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:29:01.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:01 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:01.204+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:29:01.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:01 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:01.204+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:29:01.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:01 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:01.204+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:31.635945+0000 front 2024-04-25T01:27:31.635965+0000 (oldest deadline 2024-04-25T01:28:59.638826+0000) 2024-04-25T01:29:01.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:01 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:01.549+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:29:01.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:01 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:01.549+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:29:01.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:01 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:01.549+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:29:01.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:01 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:01.549+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:29:02.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:02 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:02.063+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:29:02.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:02 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:02.063+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:29:02.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:02 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:02.063+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:29:02.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:02 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:02.063+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:29:02.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:02 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:02.063+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:32.010336+0000 front 2024-04-25T01:27:32.010529+0000 (oldest deadline 2024-04-25T01:29:00.112883+0000) 2024-04-25T01:29:02.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:02 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:02.207+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:29:02.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:02 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:02.207+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:29:02.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:02 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:02.207+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:29:02.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:02 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:02.207+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:29:02.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:02 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:02.207+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:31.635945+0000 front 2024-04-25T01:27:31.635965+0000 (oldest deadline 2024-04-25T01:28:59.638826+0000) 2024-04-25T01:29:02.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:02 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:02.531+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:29:02.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:02 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:02.531+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:29:02.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:02 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:02.531+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:29:02.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:02 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:02.531+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:29:02.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:02 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:02.531+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:28.862621+0000 front 2024-04-25T01:27:28.862845+0000 (oldest deadline 2024-04-25T01:29:01.866888+0000) 2024-04-25T01:29:03.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:03 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:03.083+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:29:03.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:03 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:03.083+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:29:03.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:03 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:03.083+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:29:03.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:03 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:03.083+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:29:03.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:03 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:03.083+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:32.010336+0000 front 2024-04-25T01:27:32.010529+0000 (oldest deadline 2024-04-25T01:29:00.112883+0000) 2024-04-25T01:29:03.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:03 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:03.192+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:29:03.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:03 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:03.192+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:29:03.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:03 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:03.192+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:29:03.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:03 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:03.192+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:29:03.441 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:03 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:03.192+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:31.635945+0000 front 2024-04-25T01:27:31.635965+0000 (oldest deadline 2024-04-25T01:28:59.638826+0000) 2024-04-25T01:29:03.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:03.548+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:29:03.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:03.548+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:29:03.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:03.548+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:29:03.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:03.548+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:29:03.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:03 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:03.548+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:28.862621+0000 front 2024-04-25T01:27:28.862845+0000 (oldest deadline 2024-04-25T01:29:01.866888+0000) 2024-04-25T01:29:04.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:04.069+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:29:04.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:04.069+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:29:04.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:04.069+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:29:04.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:04.069+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:29:04.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:04.069+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:32.010336+0000 front 2024-04-25T01:27:32.010529+0000 (oldest deadline 2024-04-25T01:29:00.112883+0000) 2024-04-25T01:29:04.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:04 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:04.192+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:29:04.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:04 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:04.192+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:29:04.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:04 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:04.192+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:29:04.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:04 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:04.192+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:29:04.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:04 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:04.192+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:31.635945+0000 front 2024-04-25T01:27:31.635965+0000 (oldest deadline 2024-04-25T01:28:59.638826+0000) 2024-04-25T01:29:04.998 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:04 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:04.586+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.423293+0000 front 2024-04-25T01:25:20.423394+0000 (oldest deadline 2024-04-25T01:26:46.531136+0000) 2024-04-25T01:29:04.999 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:04 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:04.586+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:52.735046+0000 front 2024-04-25T01:25:52.735111+0000 (oldest deadline 2024-04-25T01:27:20.141000+0000) 2024-04-25T01:29:04.999 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:04 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:04.586+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.842622+0000 front 2024-04-25T01:26:24.842571+0000 (oldest deadline 2024-04-25T01:27:55.750731+0000) 2024-04-25T01:29:04.999 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:04 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:04.586+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.653279+0000 front 2024-04-25T01:26:58.652567+0000 (oldest deadline 2024-04-25T01:28:25.960238+0000) 2024-04-25T01:29:04.999 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:04 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9[62602]: 2024-04-25T01:29:04.586+0000 7fc702800700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:28.862621+0000 front 2024-04-25T01:27:28.862845+0000 (oldest deadline 2024-04-25T01:29:01.866888+0000) 2024-04-25T01:29:05.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:05 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:05.068+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:29:05.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:05 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:05.068+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:29:05.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:05 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:05.068+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:29:05.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:05 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:05.068+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:29:05.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:05 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:05.068+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:32.010336+0000 front 2024-04-25T01:27:32.010529+0000 (oldest deadline 2024-04-25T01:29:00.112883+0000) 2024-04-25T01:29:05.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:05 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:05.156+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:29:05.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:05 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:05.156+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:29:05.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:05 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:05.156+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:29:05.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:05 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:05.156+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:29:05.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:05 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:05.156+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:31.635945+0000 front 2024-04-25T01:27:31.635965+0000 (oldest deadline 2024-04-25T01:28:59.638826+0000) 2024-04-25T01:29:06.208 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:05 smithi031 podman[88620]: 2024-04-25 01:29:05.788775513 +0000 UTC m=+5.930641109 container died ec9931861e589dee2f60968c7e02011929afe8e0fcb01c310385fdca8aca9078 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-25T01:29:06.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:06 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:06.118+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:29:06.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:06 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:06.118+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:29:06.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:06 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:06.118+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:29:06.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:06 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:06.118+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:29:06.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:06 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:06.118+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:32.010336+0000 front 2024-04-25T01:27:32.010529+0000 (oldest deadline 2024-04-25T01:29:00.112883+0000) 2024-04-25T01:29:06.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:06 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:06.118+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:29:06.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:06 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:06.118+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:29:06.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:06 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:06.118+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:29:06.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:06 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:06.118+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:29:06.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:06 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:06.118+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:31.635945+0000 front 2024-04-25T01:27:31.635965+0000 (oldest deadline 2024-04-25T01:28:59.638826+0000) 2024-04-25T01:29:06.498 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:06 smithi031 podman[88620]: 2024-04-25 01:29:06.356568932 +0000 UTC m=+6.498434522 container cleanup ec9931861e589dee2f60968c7e02011929afe8e0fcb01c310385fdca8aca9078 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:29:06.498 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:06 smithi031 bash[88620]: ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9 2024-04-25T01:29:06.997 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:06 smithi031 podman[88640]: 2024-04-25 01:29:06.616247422 +0000 UTC m=+0.805344545 container remove ec9931861e589dee2f60968c7e02011929afe8e0fcb01c310385fdca8aca9078 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-25T01:29:07.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:07 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:07.160+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:29:07.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:07 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:07.160+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:29:07.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:07 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:07.160+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:29:07.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:07 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:07.160+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:29:07.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:07 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:07.160+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:32.010336+0000 front 2024-04-25T01:27:32.010529+0000 (oldest deadline 2024-04-25T01:29:00.112883+0000) 2024-04-25T01:29:07.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:07 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:07.145+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:29:07.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:07 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:07.145+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:29:07.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:07 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:07.145+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:29:07.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:07 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:07.145+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:29:07.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:07 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:07.145+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:31.635945+0000 front 2024-04-25T01:27:31.635965+0000 (oldest deadline 2024-04-25T01:28:59.638826+0000) 2024-04-25T01:29:07.748 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:07 smithi031 podman[88729]: 2024-04-25 01:29:07.417200567 +0000 UTC m=+0.036097178 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:29:07.748 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:07 smithi031 podman[88729]: 2024-04-25 01:29:07.533671049 +0000 UTC m=+0.152567700 container create 22d0f5152429ea63ee913693526cf6e27226a2652fa5d0efb64b5f3ecfd00ed0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T01:29:08.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:07 smithi031 podman[88729]: 2024-04-25 01:29:07.836829527 +0000 UTC m=+0.455726198 container init 22d0f5152429ea63ee913693526cf6e27226a2652fa5d0efb64b5f3ecfd00ed0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T01:29:08.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:07 smithi031 podman[88729]: 2024-04-25 01:29:07.848425498 +0000 UTC m=+0.467322153 container start 22d0f5152429ea63ee913693526cf6e27226a2652fa5d0efb64b5f3ecfd00ed0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:29:08.248 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:07 smithi031 podman[88729]: 2024-04-25 01:29:07.92763646 +0000 UTC m=+0.546533115 container attach 22d0f5152429ea63ee913693526cf6e27226a2652fa5d0efb64b5f3ecfd00ed0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T01:29:08.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:08 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:08.183+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:29:08.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:08 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:08.183+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:29:08.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:08 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:08.183+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:29:08.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:08 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:08.183+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:29:08.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:08 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:08.183+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:32.010336+0000 front 2024-04-25T01:27:32.010529+0000 (oldest deadline 2024-04-25T01:29:00.112883+0000) 2024-04-25T01:29:08.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:08 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:08.191+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:29:08.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:08 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:08.191+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:29:08.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:08 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:08.191+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:29:08.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:08 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:08.191+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:29:08.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:08 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:08.191+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:31.635945+0000 front 2024-04-25T01:27:31.635965+0000 (oldest deadline 2024-04-25T01:28:59.638826+0000) 2024-04-25T01:29:08.747 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 01:29:08 smithi031 podman[88729]: 2024-04-25 01:29:08.320118632 +0000 UTC m=+0.939015306 container died 22d0f5152429ea63ee913693526cf6e27226a2652fa5d0efb64b5f3ecfd00ed0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True) 2024-04-25T01:29:09.169 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T01:29:09.286 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:29:09.286 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T01:29:09.286 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T01:29:09.287 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.10 2024-04-25T01:29:09.584 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:09 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:09.215+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:29:09.584 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:09 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:09.215+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:29:09.584 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:09 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:09.215+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:29:09.584 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:09 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:09.215+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:29:09.584 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:09 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:09.215+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:31.635945+0000 front 2024-04-25T01:27:31.635965+0000 (oldest deadline 2024-04-25T01:28:59.638826+0000) 2024-04-25T01:29:09.584 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:09 smithi040 systemd[1]: Stopping Ceph osd.10 for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:29:09.584 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:09 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:09.605+0000 7fc18be7b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:29:09.584 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:09 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:09.605+0000 7fc18be7b700 -1 osd.10 92 *** Got signal Terminated *** 2024-04-25T01:29:09.584 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:09 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:09.605+0000 7fc18be7b700 -1 osd.10 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:29:09.687 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:09 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:09.231+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:29:09.687 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:09 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:09.231+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:29:09.687 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:09 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:09.231+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:29:09.687 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:09 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:09.231+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:29:09.687 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:09 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:09.231+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:32.010336+0000 front 2024-04-25T01:27:32.010529+0000 (oldest deadline 2024-04-25T01:29:00.112883+0000) 2024-04-25T01:29:10.687 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:10 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:10.258+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:29:10.687 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:10 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:10.258+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:29:10.687 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:10 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:10.258+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:29:10.687 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:10 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:10.258+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:29:10.687 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:10 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:10.258+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:32.010336+0000 front 2024-04-25T01:27:32.010529+0000 (oldest deadline 2024-04-25T01:29:00.112883+0000) 2024-04-25T01:29:10.690 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:10 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:10.224+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:29:10.690 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:10 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:10.224+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:29:10.690 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:10 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:10.224+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:29:10.690 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:10 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:10.224+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:29:10.690 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:10 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:10.224+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:31.635945+0000 front 2024-04-25T01:27:31.635965+0000 (oldest deadline 2024-04-25T01:28:59.638826+0000) 2024-04-25T01:29:11.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:11 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:11.200+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:29:11.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:11 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:11.200+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:29:11.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:11 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:11.200+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:29:11.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:11 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:11.200+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:29:11.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:11 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:11.200+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:31.635945+0000 front 2024-04-25T01:27:31.635965+0000 (oldest deadline 2024-04-25T01:28:59.638826+0000) 2024-04-25T01:29:11.687 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:11 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:11.209+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:29:11.687 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:11 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:11.209+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:29:11.687 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:11 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:11.209+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:29:11.687 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:11 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:11.209+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:29:11.687 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:11 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:11.209+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:32.010336+0000 front 2024-04-25T01:27:32.010529+0000 (oldest deadline 2024-04-25T01:29:00.112883+0000) 2024-04-25T01:29:12.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:12 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:12.182+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:29:12.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:12 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:12.182+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:29:12.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:12 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:12.182+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:29:12.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:12 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:12.182+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:29:12.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:12 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:12.182+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:32.010336+0000 front 2024-04-25T01:27:32.010529+0000 (oldest deadline 2024-04-25T01:29:00.112883+0000) 2024-04-25T01:29:12.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:12 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:12.209+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:29:12.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:12 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:12.209+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:29:12.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:12 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:12.209+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:29:12.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:12 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:12.209+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:29:12.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:12 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:12.209+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:31.635945+0000 front 2024-04-25T01:27:31.635965+0000 (oldest deadline 2024-04-25T01:28:59.638826+0000) 2024-04-25T01:29:13.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:13 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:13.178+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:29:13.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:13 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:13.178+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:29:13.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:13 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:13.178+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:29:13.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:13 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:13.178+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:29:13.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:13 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:13.178+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:32.010336+0000 front 2024-04-25T01:27:32.010529+0000 (oldest deadline 2024-04-25T01:29:00.112883+0000) 2024-04-25T01:29:13.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:13 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:13.164+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:29:13.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:13 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:13.164+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:29:13.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:13 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:13.164+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:29:13.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:13 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:13.164+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:29:13.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:13 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:13.164+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:31.635945+0000 front 2024-04-25T01:27:31.635965+0000 (oldest deadline 2024-04-25T01:28:59.638826+0000) 2024-04-25T01:29:14.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:14 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:14.200+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:29:14.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:14 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:14.200+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:29:14.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:14 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:14.200+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:29:14.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:14 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:14.200+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:29:14.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:14 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:14.200+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:32.010336+0000 front 2024-04-25T01:27:32.010529+0000 (oldest deadline 2024-04-25T01:29:00.112883+0000) 2024-04-25T01:29:14.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:14 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:14.135+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:20.819628+0000 front 2024-04-25T01:25:20.819343+0000 (oldest deadline 2024-04-25T01:26:46.522016+0000) 2024-04-25T01:29:14.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:14 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:14.135+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.622713+0000 front 2024-04-25T01:25:50.622855+0000 (oldest deadline 2024-04-25T01:27:21.426903+0000) 2024-04-25T01:29:14.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:14 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:14.135+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:24.328067+0000 front 2024-04-25T01:26:24.328003+0000 (oldest deadline 2024-04-25T01:27:56.431875+0000) 2024-04-25T01:29:14.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:14 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:14.135+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.733016+0000 front 2024-04-25T01:26:58.732921+0000 (oldest deadline 2024-04-25T01:28:27.535010+0000) 2024-04-25T01:29:14.440 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:14 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10[57532]: 2024-04-25T01:29:14.135+0000 7fc184c68700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:31.635945+0000 front 2024-04-25T01:27:31.635965+0000 (oldest deadline 2024-04-25T01:28:59.638826+0000) 2024-04-25T01:29:15.390 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:15 smithi040 podman[68208]: 2024-04-25 01:29:15.118688773 +0000 UTC m=+5.615067517 container died 6aa70633137a5fd4591aaf6e191ef714343b83e38117630f4c7e73aceb23e757 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:29:15.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:15 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:15.187+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:29:15.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:15 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:15.187+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:29:15.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:15 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:15.187+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:29:15.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:15 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:15.187+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:29:15.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:15 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:15.187+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:32.010336+0000 front 2024-04-25T01:27:32.010529+0000 (oldest deadline 2024-04-25T01:29:00.112883+0000) 2024-04-25T01:29:15.690 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:15 smithi040 podman[68208]: 2024-04-25 01:29:15.413353037 +0000 UTC m=+5.909731781 container cleanup 6aa70633137a5fd4591aaf6e191ef714343b83e38117630f4c7e73aceb23e757 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:29:15.690 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:15 smithi040 bash[68208]: ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10 2024-04-25T01:29:15.690 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:15 smithi040 podman[68226]: 2024-04-25 01:29:15.639501098 +0000 UTC m=+0.524141803 container remove 6aa70633137a5fd4591aaf6e191ef714343b83e38117630f4c7e73aceb23e757 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T01:29:16.190 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:15 smithi040 podman[68315]: 2024-04-25 01:29:15.95680781 +0000 UTC m=+0.106549607 container create 69c7389cf858e004b1f5df77ba8d4422d5eae79b7a23908b6efe68b92350feea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux ) 2024-04-25T01:29:16.190 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:15 smithi040 podman[68315]: 2024-04-25 01:29:15.868795677 +0000 UTC m=+0.018537459 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:29:16.190 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:16 smithi040 podman[68315]: 2024-04-25 01:29:16.142746087 +0000 UTC m=+0.292487855 container init 69c7389cf858e004b1f5df77ba8d4422d5eae79b7a23908b6efe68b92350feea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T01:29:16.191 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:16 smithi040 podman[68315]: 2024-04-25 01:29:16.145878527 +0000 UTC m=+0.295620283 container start 69c7389cf858e004b1f5df77ba8d4422d5eae79b7a23908b6efe68b92350feea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-25T01:29:16.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:16 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:16.189+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:29:16.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:16 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:16.189+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:29:16.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:16 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:16.189+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:29:16.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:16 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:16.189+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:29:16.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:16 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:16.189+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:32.010336+0000 front 2024-04-25T01:27:32.010529+0000 (oldest deadline 2024-04-25T01:29:00.112883+0000) 2024-04-25T01:29:16.690 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:16 smithi040 podman[68315]: 2024-04-25 01:29:16.21760182 +0000 UTC m=+0.367343584 container attach 69c7389cf858e004b1f5df77ba8d4422d5eae79b7a23908b6efe68b92350feea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:29:16.690 INFO:journalctl@ceph.osd.10.smithi040.stdout:Apr 25 01:29:16 smithi040 podman[68315]: 2024-04-25 01:29:16.456395601 +0000 UTC m=+0.606137367 container died 69c7389cf858e004b1f5df77ba8d4422d5eae79b7a23908b6efe68b92350feea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T01:29:17.002 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T01:29:17.040 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:29:17.040 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T01:29:17.040 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T01:29:17.040 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.11 2024-04-25T01:29:17.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:17 smithi169 systemd[1]: Stopping Ceph osd.11 for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:29:17.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:17 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:17.170+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:29:17.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:17 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:17.170+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:29:17.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:17 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:17.170+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:29:17.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:17 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:17.171+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:29:17.438 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:17 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:17.171+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:32.010336+0000 front 2024-04-25T01:27:32.010529+0000 (oldest deadline 2024-04-25T01:29:00.112883+0000) 2024-04-25T01:29:17.937 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:17 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:17.552+0000 7f51d3ca3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:29:17.937 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:17 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:17.552+0000 7f51d3ca3700 -1 osd.11 92 *** Got signal Terminated *** 2024-04-25T01:29:17.937 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:17 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:17.552+0000 7f51d3ca3700 -1 osd.11 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:29:18.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:18 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:18.170+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:29:18.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:18 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:18.170+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:29:18.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:18 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:18.170+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:29:18.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:18 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:18.170+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:29:18.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:18 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:18.170+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:32.010336+0000 front 2024-04-25T01:27:32.010529+0000 (oldest deadline 2024-04-25T01:29:00.112883+0000) 2024-04-25T01:29:19.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:19 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:19.153+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:29:19.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:19 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:19.153+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:29:19.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:19 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:19.153+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:29:19.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:19 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:19.153+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:29:19.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:19 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:19.153+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:32.010336+0000 front 2024-04-25T01:27:32.010529+0000 (oldest deadline 2024-04-25T01:29:00.112883+0000) 2024-04-25T01:29:20.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:20 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:20.116+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:29:20.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:20 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:20.116+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:29:20.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:20 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:20.116+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:29:20.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:20 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:20.116+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:29:20.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:20 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:20.116+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:32.010336+0000 front 2024-04-25T01:27:32.010529+0000 (oldest deadline 2024-04-25T01:29:00.112883+0000) 2024-04-25T01:29:21.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:21 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:21.108+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:29:21.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:21 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:21.108+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:29:21.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:21 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:21.108+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:29:21.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:21 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:21.108+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:29:21.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:21 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:21.108+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:32.010336+0000 front 2024-04-25T01:27:32.010529+0000 (oldest deadline 2024-04-25T01:29:00.112883+0000) 2024-04-25T01:29:22.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:22 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:22.151+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T01:25:17.087828+0000 front 2024-04-25T01:25:17.088050+0000 (oldest deadline 2024-04-25T01:26:49.191398+0000) 2024-04-25T01:29:22.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:22 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:22.151+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6804 osd.1 since back 2024-04-25T01:25:50.792717+0000 front 2024-04-25T01:25:50.792695+0000 (oldest deadline 2024-04-25T01:27:20.596483+0000) 2024-04-25T01:29:22.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:22 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:22.151+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T01:26:25.197620+0000 front 2024-04-25T01:26:25.197598+0000 (oldest deadline 2024-04-25T01:27:56.402700+0000) 2024-04-25T01:29:22.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:22 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:22.151+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T01:26:58.703675+0000 front 2024-04-25T01:26:58.703740+0000 (oldest deadline 2024-04-25T01:28:25.708445+0000) 2024-04-25T01:29:22.437 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:22 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11[58241]: 2024-04-25T01:29:22.151+0000 7f51cca90700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.40:6812 osd.4 since back 2024-04-25T01:27:32.010336+0000 front 2024-04-25T01:27:32.010529+0000 (oldest deadline 2024-04-25T01:29:00.112883+0000) 2024-04-25T01:29:23.256 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:22 smithi169 podman[68365]: 2024-04-25 01:29:22.980356643 +0000 UTC m=+5.732410862 container died 1dbc2f0f360c796fcc8b7d3e0c5078f7acbf8ba6c3d4b86135350cc8ff7e7d70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9) 2024-04-25T01:29:23.565 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:23 smithi169 podman[68365]: 2024-04-25 01:29:23.369288763 +0000 UTC m=+6.121342988 container cleanup 1dbc2f0f360c796fcc8b7d3e0c5078f7acbf8ba6c3d4b86135350cc8ff7e7d70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5) 2024-04-25T01:29:23.565 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:23 smithi169 bash[68365]: ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11 2024-04-25T01:29:23.898 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:23 smithi169 podman[68380]: 2024-04-25 01:29:23.587150104 +0000 UTC m=+0.601337329 container remove 1dbc2f0f360c796fcc8b7d3e0c5078f7acbf8ba6c3d4b86135350cc8ff7e7d70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T01:29:24.187 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:23 smithi169 podman[68484]: 2024-04-25 01:29:23.92130737 +0000 UTC m=+0.094972114 container create 697851c6fbd4412594dd431605e0c974ef7883eef99c8e70fa88e68423ef6655 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:29:24.187 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:23 smithi169 podman[68484]: 2024-04-25 01:29:23.844600221 +0000 UTC m=+0.018264960 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:29:24.187 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:24 smithi169 podman[68484]: 2024-04-25 01:29:24.166502408 +0000 UTC m=+0.340167146 container init 697851c6fbd4412594dd431605e0c974ef7883eef99c8e70fa88e68423ef6655 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-25T01:29:24.187 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:24 smithi169 podman[68484]: 2024-04-25 01:29:24.169657345 +0000 UTC m=+0.343322090 container start 697851c6fbd4412594dd431605e0c974ef7883eef99c8e70fa88e68423ef6655 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:29:24.450 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:24 smithi169 podman[68484]: 2024-04-25 01:29:24.258427457 +0000 UTC m=+0.432092217 container attach 697851c6fbd4412594dd431605e0c974ef7883eef99c8e70fa88e68423ef6655 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T01:29:24.780 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:24 smithi169 podman[68484]: 2024-04-25 01:29:24.472655746 +0000 UTC m=+0.646320473 container died 697851c6fbd4412594dd431605e0c974ef7883eef99c8e70fa88e68423ef6655 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T01:29:25.050 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 01:29:25 smithi169 podman[68512]: 2024-04-25 01:29:25.045265398 +0000 UTC m=+0.563756473 container remove 697851c6fbd4412594dd431605e0c974ef7883eef99c8e70fa88e68423ef6655 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-25T01:29:25.055 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T01:29:25.092 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:29:25.093 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T01:29:25.093 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T01:29:25.093 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.a 2024-04-25T01:29:25.498 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 01:29:25 smithi031 systemd[1]: Stopping Ceph mds.a for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:29:25.872 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 01:29:25 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-a[69677]: 2024-04-25T01:29:25.712+0000 7fbe7be00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:29:25.872 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 01:29:25 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-a[69677]: 2024-04-25T01:29:25.712+0000 7fbe7be00700 -1 mds.a *** got signal Terminated *** 2024-04-25T01:29:29.748 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 01:29:29 smithi031 podman[88835]: 2024-04-25 01:29:29.48219584 +0000 UTC m=+3.898665293 container died 296222b64e937a87c4c6e55e737ea91d17e65b8ac07b45a9088d37cd85092f33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T01:29:30.132 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 01:29:30 smithi031 podman[88835]: 2024-04-25 01:29:30.130731188 +0000 UTC m=+4.547200635 container cleanup 296222b64e937a87c4c6e55e737ea91d17e65b8ac07b45a9088d37cd85092f33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T01:29:30.133 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 01:29:30 smithi031 bash[88835]: ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-a 2024-04-25T01:29:30.498 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 01:29:30 smithi031 podman[88860]: 2024-04-25 01:29:30.348706949 +0000 UTC m=+0.847782400 container remove 296222b64e937a87c4c6e55e737ea91d17e65b8ac07b45a9088d37cd85092f33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-a, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-25T01:29:30.876 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T01:29:30.994 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:29:30.994 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T01:29:30.994 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T01:29:30.994 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.d 2024-04-25T01:29:31.247 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 01:29:31 smithi031 systemd[1]: Stopping Ceph mds.d for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:29:31.997 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 01:29:31 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-d[68722]: 2024-04-25T01:29:31.590+0000 7fef9e800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:29:31.998 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 01:29:31 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-d[68722]: 2024-04-25T01:29:31.590+0000 7fef9e800700 -1 mds.d *** got signal Terminated *** 2024-04-25T01:29:35.497 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 01:29:35 smithi031 podman[88967]: 2024-04-25 01:29:35.263663782 +0000 UTC m=+3.803191785 container died 6d1e33ce69ac2e1912d54b8469f71bb32535411a9a88aac04ccfe450ab331fd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-25T01:29:35.997 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 01:29:35 smithi031 podman[88967]: 2024-04-25 01:29:35.859250461 +0000 UTC m=+4.398778503 container cleanup 6d1e33ce69ac2e1912d54b8469f71bb32535411a9a88aac04ccfe450ab331fd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9) 2024-04-25T01:29:35.997 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 01:29:35 smithi031 bash[88967]: ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-d 2024-04-25T01:29:36.435 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 01:29:36 smithi031 podman[88981]: 2024-04-25 01:29:36.143956499 +0000 UTC m=+0.865389206 container remove 6d1e33ce69ac2e1912d54b8469f71bb32535411a9a88aac04ccfe450ab331fd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T01:29:36.473 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T01:29:36.588 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:29:36.589 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T01:29:36.589 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T01:29:36.589 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.g 2024-04-25T01:29:36.997 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 01:29:36 smithi031 systemd[1]: Stopping Ceph mds.g for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:29:37.497 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 01:29:37 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-g[69332]: 2024-04-25T01:29:37.245+0000 7f4bb0a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:29:37.498 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 01:29:37 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-g[69332]: 2024-04-25T01:29:37.245+0000 7f4bb0a00700 -1 mds.g *** got signal Terminated *** 2024-04-25T01:29:41.497 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 01:29:41 smithi031 podman[89086]: 2024-04-25 01:29:41.247627187 +0000 UTC m=+4.135065832 container died 1ac2c36f4ed6af11793c288c16c85788ebaafc4a4ed9979db019e75147f2ef8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T01:29:41.872 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 01:29:41 smithi031 podman[89086]: 2024-04-25 01:29:41.887919458 +0000 UTC m=+4.775358113 container cleanup 1ac2c36f4ed6af11793c288c16c85788ebaafc4a4ed9979db019e75147f2ef8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T01:29:41.873 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 01:29:41 smithi031 bash[89086]: ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-g 2024-04-25T01:29:42.497 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 01:29:42 smithi031 podman[89100]: 2024-04-25 01:29:42.126805986 +0000 UTC m=+0.865530832 container remove 1ac2c36f4ed6af11793c288c16c85788ebaafc4a4ed9979db019e75147f2ef8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-g, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T01:29:42.604 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T01:29:42.721 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:29:42.722 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T01:29:42.722 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T01:29:42.722 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.j 2024-04-25T01:29:43.247 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 01:29:42 smithi031 systemd[1]: Stopping Ceph mds.j for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:29:43.747 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 01:29:43 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-j[68356]: 2024-04-25T01:29:43.373+0000 7f1efc600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:29:43.747 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 01:29:43 smithi031 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-j[68356]: 2024-04-25T01:29:43.373+0000 7f1efc600700 -1 mds.j *** got signal Terminated *** 2024-04-25T01:29:47.581 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 01:29:47 smithi031 podman[89203]: 2024-04-25 01:29:47.278170903 +0000 UTC m=+4.045585600 container died 8ed92e04f5832057f65651f2e67214486e99ab5608eae12727beb0ebc2e13460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-04-25T01:29:47.997 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 01:29:47 smithi031 podman[89203]: 2024-04-25 01:29:47.733117607 +0000 UTC m=+4.500532301 container cleanup 8ed92e04f5832057f65651f2e67214486e99ab5608eae12727beb0ebc2e13460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-j, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:29:47.997 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 01:29:47 smithi031 bash[89203]: ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-j 2024-04-25T01:29:48.497 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 01:29:48 smithi031 podman[89218]: 2024-04-25 01:29:48.092396372 +0000 UTC m=+0.798262651 container remove 8ed92e04f5832057f65651f2e67214486e99ab5608eae12727beb0ebc2e13460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-j, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T01:29:48.664 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T01:29:48.776 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:29:48.776 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T01:29:48.776 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T01:29:48.776 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.b 2024-04-25T01:29:49.082 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 01:29:48 smithi040 systemd[1]: Stopping Ceph mds.b for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:29:49.440 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 01:29:49 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-b[63226]: 2024-04-25T01:29:49.104+0000 7fbcccf45700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:29:49.440 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 01:29:49 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-b[63226]: 2024-04-25T01:29:49.104+0000 7fbcccf45700 -1 mds.b *** got signal Terminated *** 2024-04-25T01:29:52.365 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 01:29:52 smithi040 podman[68420]: 2024-04-25 01:29:52.134390574 +0000 UTC m=+3.140020751 container died 2d7279fa8895ce38cb77ccf3f7c158f1c010d6ef6520c87d86911d02468fda6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T01:29:52.690 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 01:29:52 smithi040 podman[68420]: 2024-04-25 01:29:52.554007501 +0000 UTC m=+3.559637721 container cleanup 2d7279fa8895ce38cb77ccf3f7c158f1c010d6ef6520c87d86911d02468fda6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9) 2024-04-25T01:29:52.690 INFO:journalctl@ceph.mds.b.smithi040.stdout:Apr 25 01:29:52 smithi040 bash[68420]: ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-b 2024-04-25T01:29:52.902 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T01:29:52.939 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:29:52.939 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T01:29:52.939 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T01:29:52.939 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.e 2024-04-25T01:29:53.440 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 01:29:53 smithi040 systemd[1]: Stopping Ceph mds.e for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:29:53.440 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 01:29:53 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-e[62606]: 2024-04-25T01:29:53.299+0000 7f0f5d706700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:29:53.440 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 01:29:53 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-e[62606]: 2024-04-25T01:29:53.299+0000 7f0f5d706700 -1 mds.e *** got signal Terminated *** 2024-04-25T01:29:56.394 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 01:29:56 smithi040 podman[68552]: 2024-04-25 01:29:56.144490885 +0000 UTC m=+2.951114248 container died 69a334d0808fae89763a7f6ce362f012cd85f1779a5c65557fd01d30b503ce75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9) 2024-04-25T01:29:56.690 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 01:29:56 smithi040 podman[68552]: 2024-04-25 01:29:56.499392774 +0000 UTC m=+3.306016128 container cleanup 69a334d0808fae89763a7f6ce362f012cd85f1779a5c65557fd01d30b503ce75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-25T01:29:56.690 INFO:journalctl@ceph.mds.e.smithi040.stdout:Apr 25 01:29:56 smithi040 bash[68552]: ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-e 2024-04-25T01:29:56.869 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T01:29:56.903 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:29:56.903 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T01:29:56.903 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T01:29:56.903 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.h 2024-04-25T01:29:57.190 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 01:29:57 smithi040 systemd[1]: Stopping Ceph mds.h for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:29:57.690 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 01:29:57 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-h[62917]: 2024-04-25T01:29:57.244+0000 7f8e49466700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:29:57.690 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 01:29:57 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-h[62917]: 2024-04-25T01:29:57.244+0000 7f8e49466700 -1 mds.h *** got signal Terminated *** 2024-04-25T01:30:00.415 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 01:30:00 smithi040 podman[68673]: 2024-04-25 01:30:00.140363193 +0000 UTC m=+2.991630231 container died 99bde325bbc07f81ce80ab61bd2bcbd301353d746952fa734deaf92d685a5562 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:30:00.690 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 01:30:00 smithi040 podman[68673]: 2024-04-25 01:30:00.528488783 +0000 UTC m=+3.379755827 container cleanup 99bde325bbc07f81ce80ab61bd2bcbd301353d746952fa734deaf92d685a5562 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-25T01:30:00.980 INFO:journalctl@ceph.mds.h.smithi040.stdout:Apr 25 01:30:00 smithi040 bash[68673]: ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-h 2024-04-25T01:30:00.982 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T01:30:01.016 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:30:01.017 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T01:30:01.017 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T01:30:01.017 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.k 2024-04-25T01:30:01.335 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 01:30:01 smithi040 systemd[1]: Stopping Ceph mds.k for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:30:01.690 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 01:30:01 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-k[62286]: 2024-04-25T01:30:01.356+0000 7f309e796700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:30:01.690 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 01:30:01 smithi040 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-k[62286]: 2024-04-25T01:30:01.356+0000 7f309e796700 -1 mds.k *** got signal Terminated *** 2024-04-25T01:30:03.468 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 01:30:03 smithi040 podman[68794]: 2024-04-25 01:30:03.225347281 +0000 UTC m=+1.960808400 container died b6248292272ea2abf2921c4efede399990005aef985fbc328e1be1309488a7ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-k, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T01:30:03.818 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 01:30:03 smithi040 podman[68794]: 2024-04-25 01:30:03.590256761 +0000 UTC m=+2.325717880 container cleanup b6248292272ea2abf2921c4efede399990005aef985fbc328e1be1309488a7ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T01:30:03.818 INFO:journalctl@ceph.mds.k.smithi040.stdout:Apr 25 01:30:03 smithi040 bash[68794]: ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-k 2024-04-25T01:30:03.956 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T01:30:03.989 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:30:03.989 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T01:30:03.989 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T01:30:03.989 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.c 2024-04-25T01:30:04.268 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 01:30:04 smithi169 systemd[1]: Stopping Ceph mds.c for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:30:04.545 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 01:30:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-c[63241]: 2024-04-25T01:30:04.426+0000 7fe6ef8e9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:30:04.545 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 01:30:04 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-c[63241]: 2024-04-25T01:30:04.426+0000 7fe6ef8e9700 -1 mds.c *** got signal Terminated *** 2024-04-25T01:30:08.437 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 01:30:08 smithi169 podman[68591]: 2024-04-25 01:30:08.133208366 +0000 UTC m=+3.925080503 container died a80419ae6cb96211dd12912a0d598e0fbca776e0c0a443a289fb9b222c0b9723 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:30:08.873 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 01:30:08 smithi169 podman[68591]: 2024-04-25 01:30:08.536306341 +0000 UTC m=+4.328178484 container cleanup a80419ae6cb96211dd12912a0d598e0fbca776e0c0a443a289fb9b222c0b9723 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-c, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9) 2024-04-25T01:30:08.873 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 01:30:08 smithi169 bash[68591]: ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-c 2024-04-25T01:30:08.873 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 01:30:08 smithi169 podman[68618]: 2024-04-25 01:30:08.779724561 +0000 UTC m=+0.638448516 container remove a80419ae6cb96211dd12912a0d598e0fbca776e0c0a443a289fb9b222c0b9723 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T01:30:08.890 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T01:30:08.925 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:30:08.925 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T01:30:08.925 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T01:30:08.925 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.f 2024-04-25T01:30:09.186 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 01:30:09 smithi169 systemd[1]: Stopping Ceph mds.f for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:30:09.687 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 01:30:09 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-f[62933]: 2024-04-25T01:30:09.291+0000 7f032b136700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:30:09.687 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 01:30:09 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-f[62933]: 2024-04-25T01:30:09.291+0000 7f032b136700 -1 mds.f *** got signal Terminated *** 2024-04-25T01:30:12.361 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 01:30:12 smithi169 podman[68727]: 2024-04-25 01:30:12.130529909 +0000 UTC m=+2.958158728 container died b6396cd0d5014a2a1aa4667f226c71c4cd019f3dc0fc335636fab72325fc4c15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-f, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T01:30:12.686 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 01:30:12 smithi169 podman[68727]: 2024-04-25 01:30:12.549980114 +0000 UTC m=+3.377608929 container cleanup b6396cd0d5014a2a1aa4667f226c71c4cd019f3dc0fc335636fab72325fc4c15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T01:30:12.687 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 01:30:12 smithi169 bash[68727]: ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-f 2024-04-25T01:30:12.946 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T01:30:12.967 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 01:30:12 smithi169 podman[68741]: 2024-04-25 01:30:12.726686668 +0000 UTC m=+0.590217110 container remove b6396cd0d5014a2a1aa4667f226c71c4cd019f3dc0fc335636fab72325fc4c15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424) 2024-04-25T01:30:12.967 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 01:30:12 smithi169 systemd[1]: ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.f.service: Deactivated successfully. 2024-04-25T01:30:12.967 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 01:30:12 smithi169 systemd[1]: Stopped Ceph mds.f for 95849b94-029f-11ef-bc93-c7b262605968. 2024-04-25T01:30:12.967 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 01:30:12 smithi169 systemd[1]: ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.f.service: Consumed 8.079s CPU time. 2024-04-25T01:30:12.979 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:30:12.979 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T01:30:12.979 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T01:30:12.979 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.i 2024-04-25T01:30:13.290 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 01:30:13 smithi169 systemd[1]: Stopping Ceph mds.i for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:30:13.686 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 01:30:13 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-i[63552]: 2024-04-25T01:30:13.312+0000 7f17565b7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:30:13.687 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 01:30:13 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-i[63552]: 2024-04-25T01:30:13.312+0000 7f17565b7700 -1 mds.i *** got signal Terminated *** 2024-04-25T01:30:16.373 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 01:30:16 smithi169 podman[68848]: 2024-04-25 01:30:16.138058278 +0000 UTC m=+2.920658787 container died d04b8c94532444375d32c168575c9add64ab3f756e19337ff806455de59f3c55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-i, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-25T01:30:16.686 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 01:30:16 smithi169 podman[68848]: 2024-04-25 01:30:16.537531457 +0000 UTC m=+3.320131974 container cleanup d04b8c94532444375d32c168575c9add64ab3f756e19337ff806455de59f3c55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-i, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T01:30:16.687 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 01:30:16 smithi169 bash[68848]: ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-i 2024-04-25T01:30:16.856 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T01:30:16.887 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:30:16.888 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T01:30:16.888 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T01:30:16.888 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.l 2024-04-25T01:30:17.186 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 01:30:16 smithi169 systemd[1]: Stopping Ceph mds.l for 95849b94-029f-11ef-bc93-c7b262605968... 2024-04-25T01:30:17.686 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 01:30:17 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-l[62629]: 2024-04-25T01:30:17.215+0000 7f3f95057700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:30:17.687 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 01:30:17 smithi169 ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-l[62629]: 2024-04-25T01:30:17.215+0000 7f3f95057700 -1 mds.l *** got signal Terminated *** 2024-04-25T01:30:20.414 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 01:30:20 smithi169 podman[68970]: 2024-04-25 01:30:20.094002086 +0000 UTC m=+2.972706901 container died 5ac256c12852a2a3f5af0e0af802795ddd72b34f530033f084d93cd4a084165d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-l, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T01:30:20.686 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 01:30:20 smithi169 podman[68970]: 2024-04-25 01:30:20.517212305 +0000 UTC m=+3.395917161 container cleanup 5ac256c12852a2a3f5af0e0af802795ddd72b34f530033f084d93cd4a084165d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-l, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:30:20.687 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 01:30:20 smithi169 bash[68970]: ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-l 2024-04-25T01:30:20.934 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T01:30:20.954 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 01:30:20 smithi169 podman[68984]: 2024-04-25 01:30:20.726351251 +0000 UTC m=+0.624421340 container remove 5ac256c12852a2a3f5af0e0af802795ddd72b34f530033f084d93cd4a084165d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-95849b94-029f-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9) 2024-04-25T01:30:20.954 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 01:30:20 smithi169 systemd[1]: ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.l.service: Deactivated successfully. 2024-04-25T01:30:20.954 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 01:30:20 smithi169 systemd[1]: Stopped Ceph mds.l for 95849b94-029f-11ef-bc93-c7b262605968. 2024-04-25T01:30:20.954 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 01:30:20 smithi169 systemd[1]: ceph-95849b94-029f-11ef-bc93-c7b262605968@mds.l.service: Consumed 1.051s CPU time. 2024-04-25T01:30:20.964 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:30:20.964 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T01:30:20.964 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 95849b94-029f-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T01:30:21.290 INFO:teuthology.orchestra.run.smithi031.stdout:Deleting cluster with fsid: 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:30:26.309 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 95849b94-029f-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T01:30:26.508 INFO:teuthology.orchestra.run.smithi040.stdout:Deleting cluster with fsid: 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:30:29.059 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 95849b94-029f-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T01:30:29.249 INFO:teuthology.orchestra.run.smithi169.stdout:Deleting cluster with fsid: 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:30:31.539 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T01:30:31.636 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T01:30:31.669 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T01:30:31.694 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T01:30:31.697 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672295/remote/smithi031/crash 2024-04-25T01:30:31.697 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/crash -- . 2024-04-25T01:30:31.807 INFO:teuthology.orchestra.run.smithi031.stderr:tar: /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T01:30:31.807 INFO:teuthology.orchestra.run.smithi031.stderr:tar: Error is not recoverable: exiting now 2024-04-25T01:30:31.813 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672295/remote/smithi040/crash 2024-04-25T01:30:31.814 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/crash -- . 2024-04-25T01:30:31.855 INFO:teuthology.orchestra.run.smithi040.stderr:tar: /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T01:30:31.855 INFO:teuthology.orchestra.run.smithi040.stderr:tar: Error is not recoverable: exiting now 2024-04-25T01:30:31.856 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672295/remote/smithi169/crash 2024-04-25T01:30:31.857 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/crash -- . 2024-04-25T01:30:31.887 INFO:teuthology.orchestra.run.smithi169.stderr:tar: /var/lib/ceph/95849b94-029f-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T01:30:31.887 INFO:teuthology.orchestra.run.smithi169.stderr:tar: Error is not recoverable: exiting now 2024-04-25T01:30:31.889 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T01:30:31.889 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-25T01:30:32.000 INFO:tasks.cephadm:Compressing logs... 2024-04-25T01:30:32.000 DEBUG:teuthology.orchestra.run.smithi031:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T01:30:32.005 DEBUG:teuthology.orchestra.run.smithi040:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T01:30:32.007 DEBUG:teuthology.orchestra.run.smithi169:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T01:30:32.032 INFO:teuthology.orchestra.run.smithi169.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T01:30:32.033 INFO:teuthology.orchestra.run.smithi169.stderr:: No such file or directory 2024-04-25T01:30:32.033 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.log 2024-04-25T01:30:32.033 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T01:30:32.033 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T01:30:32.033 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T01:30:32.033 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T01:30:32.033 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.f.log: /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.audit.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T01:30:32.033 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T01:30:32.033 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T01:30:32.033 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T01:30:32.034 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mon.c.log: 86.3% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T01:30:32.035 INFO:teuthology.orchestra.run.smithi169.stderr: 90.2% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T01:30:32.036 INFO:teuthology.orchestra.run.smithi040.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T01:30:32.036 INFO:teuthology.orchestra.run.smithi040.stderr:: No such file or directory 2024-04-25T01:30:32.036 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.log 2024-04-25T01:30:32.036 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T01:30:32.036 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T01:30:32.036 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T01:30:32.036 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T01:30:32.036 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.7.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T01:30:32.041 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T01:30:32.041 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T01:30:32.041 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T01:30:32.042 INFO:teuthology.orchestra.run.smithi040.stderr: 86.3% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T01:30:32.042 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.10.log 90.2% 2024-04-25T01:30:32.042 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T01:30:32.042 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T01:30:32.042 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T01:30:32.042 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.cephadm.log: 82.3% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T01:30:32.042 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T01:30:32.044 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.e.log: 92.7% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T01:30:32.045 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T01:30:32.060 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T01:30:32.060 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T01:30:32.060 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T01:30:32.060 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.cephadm.log: 82.3% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T01:30:32.060 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T01:30:32.061 INFO:teuthology.orchestra.run.smithi031.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-25T01:30:32.062 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T01:30:32.062 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T01:30:32.063 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T01:30:32.064 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.log 2024-04-25T01:30:32.065 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.11.log: 92.9% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T01:30:32.065 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T01:30:32.066 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T01:30:32.066 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T01:30:32.067 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T01:30:32.068 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.g.log: 86.3% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T01:30:32.068 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T01:30:32.069 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T01:30:32.070 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.3.log: 90.0% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T01:30:32.070 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T01:30:32.072 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T01:30:32.072 INFO:teuthology.orchestra.run.smithi031.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T01:30:32.073 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mgr.x.log: 95.0% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T01:30:32.073 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T01:30:32.074 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T01:30:32.076 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mon.a.log: /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T01:30:32.076 INFO:teuthology.orchestra.run.smithi031.stderr: 94.8% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T01:30:32.077 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T01:30:32.077 INFO:teuthology.orchestra.run.smithi031.stderr: 82.5% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T01:30:32.078 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T01:30:32.079 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T01:30:32.083 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T01:30:32.084 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.k.log: 95.0% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T01:30:32.085 INFO:teuthology.orchestra.run.smithi031.stderr: 94.9% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T01:30:32.088 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.9.log: 94.7% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T01:30:32.097 INFO:teuthology.orchestra.run.smithi031.stderr: 91.0% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T01:30:32.114 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T01:30:32.115 INFO:teuthology.orchestra.run.smithi169.stderr: 95.0% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T01:30:32.277 INFO:teuthology.orchestra.run.smithi031.stderr: 90.2% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T01:30:32.328 INFO:teuthology.orchestra.run.smithi040.stderr: 95.5% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T01:30:32.455 INFO:teuthology.orchestra.run.smithi169.stderr: 91.7% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T01:30:32.495 INFO:teuthology.orchestra.run.smithi169.stderr: 95.5% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T01:30:32.513 INFO:teuthology.orchestra.run.smithi040.stderr: 91.7% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T01:30:32.556 INFO:teuthology.orchestra.run.smithi040.stderr: 94.9% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T01:30:33.241 INFO:teuthology.orchestra.run.smithi031.stderr: 90.3% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T01:30:33.993 INFO:teuthology.orchestra.run.smithi169.stderr: 95.6% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T01:30:35.093 INFO:teuthology.orchestra.run.smithi169.stderr: 94.4% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T01:30:37.160 INFO:teuthology.orchestra.run.smithi040.stderr: 94.2% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T01:30:44.361 INFO:teuthology.orchestra.run.smithi040.stderr: 93.4% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T01:30:44.839 INFO:teuthology.orchestra.run.smithi031.stderr: 93.1% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T01:30:46.503 INFO:teuthology.orchestra.run.smithi169.stderr: 93.2% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T01:30:47.129 INFO:teuthology.orchestra.run.smithi169.stderr: 93.3% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T01:30:48.020 INFO:teuthology.orchestra.run.smithi040.stderr: 93.2% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T01:30:50.089 INFO:teuthology.orchestra.run.smithi169.stderr: 93.2% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T01:30:51.068 INFO:teuthology.orchestra.run.smithi031.stderr: 93.1% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T01:30:53.007 INFO:teuthology.orchestra.run.smithi169.stderr: 93.1% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T01:30:53.009 INFO:teuthology.orchestra.run.smithi169.stderr: 2024-04-25T01:30:53.009 INFO:teuthology.orchestra.run.smithi169.stderr:real 0m20.988s 2024-04-25T01:30:53.010 INFO:teuthology.orchestra.run.smithi169.stderr:user 1m11.748s 2024-04-25T01:30:53.010 INFO:teuthology.orchestra.run.smithi169.stderr:sys 0m2.562s 2024-04-25T01:30:53.838 INFO:teuthology.orchestra.run.smithi040.stderr: 93.1% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T01:30:54.343 INFO:teuthology.orchestra.run.smithi031.stderr: 92.9% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T01:30:55.488 INFO:teuthology.orchestra.run.smithi040.stderr: 93.2% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T01:30:55.490 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-25T01:30:55.490 INFO:teuthology.orchestra.run.smithi040.stderr:real 0m23.467s 2024-04-25T01:30:55.490 INFO:teuthology.orchestra.run.smithi040.stderr:user 1m17.357s 2024-04-25T01:30:55.490 INFO:teuthology.orchestra.run.smithi040.stderr:sys 0m2.488s 2024-04-25T01:30:57.823 INFO:teuthology.orchestra.run.smithi031.stderr: 92.9% -- replaced with /var/log/ceph/95849b94-029f-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T01:30:57.828 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-04-25T01:30:57.828 INFO:teuthology.orchestra.run.smithi031.stderr:real 0m25.791s 2024-04-25T01:30:57.828 INFO:teuthology.orchestra.run.smithi031.stderr:user 1m15.457s 2024-04-25T01:30:57.828 INFO:teuthology.orchestra.run.smithi031.stderr:sys 0m5.821s 2024-04-25T01:30:57.829 INFO:tasks.cephadm:Archiving logs... 2024-04-25T01:30:57.829 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672295/remote/smithi031/log 2024-04-25T01:30:57.830 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T01:31:09.703 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672295/remote/smithi040/log 2024-04-25T01:31:09.704 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T01:31:20.943 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672295/remote/smithi169/log 2024-04-25T01:31:20.944 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T01:31:31.459 INFO:tasks.cephadm:Removing cluster... 2024-04-25T01:31:31.459 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 95849b94-029f-11ef-bc93-c7b262605968 --force 2024-04-25T01:31:31.785 INFO:teuthology.orchestra.run.smithi031.stdout:Deleting cluster with fsid: 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:31:32.652 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 95849b94-029f-11ef-bc93-c7b262605968 --force 2024-04-25T01:31:32.855 INFO:teuthology.orchestra.run.smithi040.stdout:Deleting cluster with fsid: 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:31:33.223 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 95849b94-029f-11ef-bc93-c7b262605968 --force 2024-04-25T01:31:33.403 INFO:teuthology.orchestra.run.smithi169.stdout:Deleting cluster with fsid: 95849b94-029f-11ef-bc93-c7b262605968 2024-04-25T01:31:33.812 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T01:31:33.812 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T01:31:33.873 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T01:31:33.891 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T01:31:33.906 INFO:tasks.cephadm:Teardown complete 2024-04-25T01:31:33.906 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T01:31:33.917 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T01:31:33.917 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T01:31:33.924 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T01:31:33.934 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T01:31:34.087 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T01:31:34.097 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T01:31:34.097 DEBUG:teuthology.orchestra.run.smithi031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:31:34.106 DEBUG:teuthology.orchestra.run.smithi040:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:31:34.108 DEBUG:teuthology.orchestra.run.smithi169:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:31:34.122 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-04-25T01:31:34.122 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 1: ntpq: command not found 2024-04-25T01:31:34.126 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T01:31:34.126 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-04-25T01:31:34.126 INFO:teuthology.orchestra.run.smithi169.stdout:^- hv01.front.sepia.ceph.com 2 6 377 18 +1147us[+1157us] +/- 572ms 2024-04-25T01:31:34.126 INFO:teuthology.orchestra.run.smithi169.stdout:^* hv02.front.sepia.ceph.com 3 6 377 15 -174us[ -164us] +/- 54ms 2024-04-25T01:31:34.126 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 +231us[ +241us] +/- 61ms 2024-04-25T01:31:34.127 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:31:34.127 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T01:31:34.127 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2024-04-25T01:31:34.128 INFO:teuthology.orchestra.run.smithi040.stdout:^- hv01.front.sepia.ceph.com 2 6 377 17 +1158us[+1223us] +/- 572ms 2024-04-25T01:31:34.128 INFO:teuthology.orchestra.run.smithi040.stdout:^* hv02.front.sepia.ceph.com 3 6 377 17 -145us[ -81us] +/- 54ms 2024-04-25T01:31:34.128 INFO:teuthology.orchestra.run.smithi040.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 16 +248us[ +248us] +/- 61ms 2024-04-25T01:31:34.128 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:31:34.155 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-04-25T01:31:34.164 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T01:31:34.164 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-04-25T01:31:34.165 INFO:teuthology.orchestra.run.smithi031.stdout:^- hv01.front.sepia.ceph.com 2 6 377 19 +1169us[+1228us] +/- 572ms 2024-04-25T01:31:34.165 INFO:teuthology.orchestra.run.smithi031.stdout:^* hv02.front.sepia.ceph.com 3 6 377 18 -155us[ -95us] +/- 54ms 2024-04-25T01:31:34.165 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 +247us[ +247us] +/- 61ms 2024-04-25T01:31:34.165 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:31:34.166 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T01:31:34.176 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T01:31:34.177 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T01:31:34.187 DEBUG:teuthology.orchestra.run.smithi031:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T01:31:34.381 DEBUG:teuthology.orchestra.run.smithi040:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T01:31:34.453 DEBUG:teuthology.orchestra.run.smithi169:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T01:31:34.519 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T01:31:34.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:31:34.620 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T01:31:34.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:31:34.653 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T01:31:34.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:31:34.679 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T01:31:34.692 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T01:31:34.702 INFO:teuthology.task.internal:Duration was 2148.682561 seconds 2024-04-25T01:31:34.702 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T01:31:34.712 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T01:31:34.712 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T01:31:34.720 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T01:31:34.723 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T01:31:34.759 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T01:31:34.763 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T01:31:34.868 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T01:31:35.233 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T01:31:35.233 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T01:31:35.233 DEBUG:teuthology.orchestra.run.smithi031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T01:31:35.314 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi040.front.sepia.ceph.com 2024-04-25T01:31:35.314 DEBUG:teuthology.orchestra.run.smithi040:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T01:31:35.347 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T01:31:35.347 DEBUG:teuthology.orchestra.run.smithi169:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T01:31:35.377 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T01:31:35.377 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T01:31:35.385 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T01:31:35.390 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T01:31:35.474 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T01:31:35.474 DEBUG:teuthology.orchestra.run.smithi031:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T01:31:35.480 DEBUG:teuthology.orchestra.run.smithi040:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T01:31:35.483 DEBUG:teuthology.orchestra.run.smithi169:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T01:31:35.977 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T01:31:35.989 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T01:31:35.989 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T01:31:36.075 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T01:31:36.101 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T01:31:36.125 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T01:31:36.137 DEBUG:teuthology.orchestra.run.smithi031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:31:36.145 DEBUG:teuthology.orchestra.run.smithi040:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:31:36.148 DEBUG:teuthology.orchestra.run.smithi169:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:31:36.169 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = core 2024-04-25T01:31:36.187 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = core 2024-04-25T01:31:36.242 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = core 2024-04-25T01:31:36.289 DEBUG:teuthology.orchestra.run.smithi031:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:31:36.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:31:36.342 DEBUG:teuthology.orchestra.run.smithi040:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:31:36.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:31:36.357 DEBUG:teuthology.orchestra.run.smithi169:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:31:36.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:31:36.372 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T01:31:36.385 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T01:31:36.385 DEBUG:teuthology.misc:Transferring archived files from smithi031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672295/remote/smithi031 2024-04-25T01:31:36.386 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T01:31:36.497 DEBUG:teuthology.misc:Transferring archived files from smithi040:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672295/remote/smithi040 2024-04-25T01:31:36.497 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T01:31:36.535 DEBUG:teuthology.misc:Transferring archived files from smithi169:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672295/remote/smithi169 2024-04-25T01:31:36.535 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T01:31:36.571 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T01:31:36.571 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T01:31:36.579 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T01:31:36.581 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T01:31:36.632 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T01:31:36.645 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T01:31:36.646 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T01:31:36.657 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T01:31:36.657 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T01:31:36.664 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T01:31:36.667 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T01:31:36.680 INFO:teuthology.orchestra.run.smithi040.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 01:31 /home/ubuntu/cephtest 2024-04-25T01:31:36.681 INFO:teuthology.orchestra.run.smithi169.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 01:31 /home/ubuntu/cephtest 2024-04-25T01:31:36.714 INFO:teuthology.orchestra.run.smithi031.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 01:31 /home/ubuntu/cephtest 2024-04-25T01:31:36.717 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T01:31:36.729 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T01:31:36.758 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/ffsb}} duration: 2148.6825606822968 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-25T01:31:36.758 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T01:31:36.839 INFO:teuthology.run:pass